[overture] Re: Building v24 on Blue Gene P

Hi Joe,

Really you will be divorcing Overture and CG from perl, which is doable.  
Stripping calls from Overture is easy, you just need to hack/replace 
Ogshow/OvertureParser.C when building it on the BG/P backend.  Generating 
perl-less command files on your mac will not be that hard though it may involve 
some adjustments to the way overture's command interpreter records the playback 
commands.  It may also be possible to do it with a simple perl script that 
reads a command file line by line, interprets each line ending in a semi-colon 
and string-interpolates the remaining lines, outputting the interpolated lines 
to a perl-less command file.

bpp should not be an issue.  It is used to generate code at compile time (only 
when bpp files have changed) and is never executed at run time.  If executing 
bpp on the frontend when compiling cg is an issue you can just 'touch' all the 
source files so that the bpp dependencies are not activated in the build 
system.  If that does not work then the build system will have to be adjusted 
to ignore regeneration of bpp generated source code. 

Regards,
Kyle
 
________________________________________
From: overture-bounce@xxxxxxxxxxxxx [overture-bounce@xxxxxxxxxxxxx] On Behalf 
Of J. P. Bernstein [astro@xxxxxxx]
Sent: Wednesday, March 28, 2012 11:43 AM
To: overture@xxxxxxxxxxxxx
Subject: [overture] Re: Building v24 on Blue Gene P

Hello Kyle,

I have confirmed with ALCF management that building perl on the backend of the 
Argonne BG/P is not and will not be supported by the facility due to overhead 
issues.

What we are really after is having CG built on the BG/P backend. To us, the 
most sensible path is to build Overture somewhere else (like my Mac laptop, 
which has been done), and use it to generate perl-less command files to pass to 
a CG build on the BG/P backend. Thus, I would like to divorce CG from Overture 
to the greatest extent possible. How realistic is that goal, from the 
perspective of the Overture developers?

The immediate issue that arises with the above is CG's dependency on BPP. How 
much does CG depended on the extended capabilities of BPP, relative to CPP. In 
other words, would building BPP on it's own or reverting CG to work with CPP 
make more sense?

Best,

Joe

----- Original Message -----
From: "Kyle K. Chand" <chand1@xxxxxxxx>
To: overture@xxxxxxxxxxxxx
Sent: Monday, March 5, 2012 4:24:19 PM
Subject: [overture] Re: Building v24 on Blue Gene P

Hi Joe,

Mechanically, it would not be hard to replace the perl interpreter in
OvertureParser.C with the python interpreter (or any other
interpreter, check out Lua!).
However, it may be a great deal of effort to translate all our command
files from perl to python.  We do not use perl for regular
expressions, we use perl for two
purposes:
1) writing perl code to parameterize and generate input to our grid
generator and solvers, and
2) utilize perl's "string interpolation" mechanism to turn command
file lines like:
$N=11;
number of grid lines
$N $N
into
number of grid lines
11 11

Lately we have been using 1) a lot more, often writing perl loops and
including perl modules to parameterize and automate input generation
and make things generally more maintainable.
The mechanism represented by 2) is how 1) eventually gets used.  So,
it would be a lot of work to re-write many of our more recent (and
future) command files to use another interpreted language.

I think the most efficient path would be to strip perl from the BG
port of the code.  Then use Overture on some other machine to generate
perl-less, pure text, input for the problems you want to run.  One
could even
write a (perl :-) script that reads a command file and sends each line
ending with a semi-colon to the interpreter and every other line for
string interpolation, echoing the interpolated line to a "stripped"
command file.  You would not even need another copy of Overture for
that.

Best regards,
Kyle




mailto: chand1@xxxxxxxx
phoneto: (925) 422 7740



On Mar 5, 2012, at 12:09 PM, J. P. Bernstein wrote:

> Thank you very much, Kyle, for those details.
>
> While I am digesting that, would you care to comment on the notion
> of replacing perl functionality with python? I am not familiar
> enough with python or Overture to know, of the top of my head, if
> python offers the capability needed.
>
> I ask because python is available on the back end of the BG/P at
> Argonne. Might such a transformation be of interest to you and the
> Overture team? My job description would allow me to engage
> considerably on such a task.
>
> Best,
>
> Joe
>
> On Mar 5, 2012, at 12:42 PM, Kyle K. Chand wrote:
>
>> Hi Joe,
>>
>> Well, I have some bad news regarding perl.  Perl is really an
>> integral part of how we set up problems both for the
>> grid generator as well as CG solver runs.  Using Overture w/o perl
>> is kind of like editing C++ files with ed in a terminal window; you
>> can do it but it is not really suggested :-)  Overture's graphics/
>> interpreter interface actually starts a perl interpreter
>> and the text commands are first sent to the interpreter to either
>> get string interpolation performed or perl statements evaluated.
>> You can see an example of how this is used in $Overture/sampleGrids/
>> squareArg.cmd, which parameterizes an ogen
>> grid generator command file using perl statements.
>>
>> Now, it is probably possible to build Overture without the perl
>> interpreter linked in, you will have to change OvertureParser.C as
>> well
>> as the bin/Makefile to remove references to perl.  Then, however,
>> you will need to strip perl from any command files you plan to use.
>>
>> The best way to go would be to compile Perl for BGP.  However, if
>> that is not possible for some reason, here is another way to go:
>> 1) Build a "full" version of Overture somewhere you can generate
>> grids and test things out
>> 2) Modify this "full" version to write the "plain text" command
>> files after the perl processing is completed, run any command files
>> you want through it to strip the perl
>> 3) Build a "perl-less" version for BGP on which you can run perl-
>> stripped cg command files
>>
>> The easiest way to "strip" command files of perl might be to
>> intercept the interpolated strings before they get sent to the rest
>> of Overture
>> from OvertureParser, I'll have to think about the least-intrusive
>> way to do that.
>>
>> Regards,
>> Kyle
>> PS, I will try to get to your Mac question later today.
>>
>> mailto: chand1@xxxxxxxx
>> phoneto: (925) 422 7740
>>
>>
>>
>> On Mar 2, 2012, at 5:58 PM, J. P. Bernstein wrote:
>>
>>> Hello Bill,
>>>
>>> Thank you and Kyle for the quick replies to my emails.
>>>
>>> Warning headed. I am on assignment from the Argonne Leadership
>>> Computing
>>> Facility (ALCF) on this effort. There is significant interest at
>>> ALCF to
>>> at least run the CG solvers on BGP (not necessarily ogen).
>>> Nevertheless,
>>> I will share your sentiment with my assigners.
>>>
>>> A question about the perl situation: my initial assessment
>>> suggested to
>>> me that perl was needed only for configuring the build and running
>>> the
>>> tests. Since perl is available on the front end, the configuration
>>> part
>>> should work. Part of my assignment is converting the perl test
>>> scripts
>>> to, e.g., python. However, your message below makes me doubt my
>>> assessment. Are the perl libs, which, as you know, are not
>>> available on
>>> the back end, needed at run time beyond the test scripts?
>>>
>>> Best,
>>>
>>> Joe
>>>
>>> On 3/2/2012 6:08 PM, Bill Henshaw wrote:
>>>> Hi Joe,
>>>> At one time I did get Overture compiled on the BGP front end at ANL
>>>> with xlc but I was left
>>>> with the issue that the perl libraries were not available with
>>>> BGP. We
>>>> could work around
>>>> this if necessary but I prefer not too if possible.
>>>>
>>>> That said, experience tells me that since we have not run
>>>> Overture on
>>>> BGP then you
>>>> are likely to have problems that will be very hard for you to fix.
>>>> Therefore this is just a warning
>>>> that the path you are on may lead to great misery and despair ...
>>>>
>>>> ...Bill
>>>>
>>>>
>>>> J. P. Bernstein wrote:
>>>>> Hello,
>>>>>
>>>>> I am trying to build Overture.v24 on Blue Gene P (BG/P).
>>>>> In an attempt to add a BG/P configuration for A++, I looked for
>>>>> the
>>>>> following config file mentioned in the A++ README file:
>>>>> A++/config/config.options
>>>>> but was not able to find the file or the directory mentioned in
>>>>> the
>>>>> A++ directory tree.
>>>>>
>>>>> I was able to build A++ without errors with the following
>>>>> configure
>>>>> command:
>>>>>
>>>>> ./configure --with-CC=mpixlc_r --with-CFLAGS="-qarch=450d -
>>>>> qtune=450"
>>>>> --with-CXX=mpixlcxx_r --with-CXXFLAGS="-qarch=450d -qtune=450"
>>>>> --prefix=`pwd`
>>>>>
>>>>> However, the flags did not seem to be passed:
>>>>>
>>>>> mpixlc_r -DHAVE_CONFIG_H -I. -I. -I../.. -I../../include
>>>>> -c
>>>>> `test -f common_func.c || echo './'`common_func.c
>>>>>
>>>>> mpixlcxx_r           -o resize  resize.o
>>>>> -L/home/jpbernst/Code/Overture/A++P++-0.8.0/A++/install/lib  -
>>>>> L../src
>>>>> -lApp -lApp_static -lApp  -lm
>>>>>
>>>>> Is this a known issue?
>>>>>
>>>>> Returning to the BG/P config issue, I tried to hack :
>>>>>
>>>>> config/A++P++_common_autoconfig.m4
>>>>> config.sub
>>>>>
>>>>> in order to add a Bg/P setup without success. Is there
>>>>> documentation
>>>>> available for adding a machine config? I was not able to find
>>>>> any on
>>>>> the Overture webpage.
>>>>>
>>>>> Best,
>>>>>
>>>>> Joe Bernstein
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>

Other related posts: