[overture] Re: Building v24 on Blue Gene P

  • From: "Kyle K. Chand" <chand1@xxxxxxxx>
  • To: "overture@xxxxxxxxxxxxx" <overture@xxxxxxxxxxxxx>
  • Date: Mon, 5 Mar 2012 14:24:19 -0800

Hi Joe,
Mechanically, it would not be hard to replace the perl interpreter in OvertureParser.C with the python interpreter (or any other interpreter, check out Lua!). However, it may be a great deal of effort to translate all our command files from perl to python. We do not use perl for regular expressions, we use perl for two 
purposes:
1) writing perl code to parameterize and generate input to our grid generator and solvers, and 2) utilize perl's "string interpolation" mechanism to turn command file lines like: 
$N=11;
number of grid lines
$N $N
into
number of grid lines
11 11
Lately we have been using 1) a lot more, often writing perl loops and including perl modules to parameterize and automate input generation and make things generally more maintainable. The mechanism represented by 2) is how 1) eventually gets used. So, it would be a lot of work to re-write many of our more recent (and future) command files to use another interpreted language.
I think the most efficient path would be to strip perl from the BG port of the code. Then use Overture on some other machine to generate perl-less, pure text, input for the problems you want to run. One could even write a (perl :-) script that reads a command file and sends each line ending with a semi-colon to the interpreter and every other line for string interpolation, echoing the interpolated line to a "stripped" command file. You would not even need another copy of Overture for that. 

Best regards,
Kyle




mailto: chand1@xxxxxxxx
phoneto: (925) 422 7740



On Mar 5, 2012, at 12:09 PM, J. P. Bernstein wrote:


Thank you very much, Kyle, for those details.
While I am digesting that, would you care to comment on the notion of replacing perl functionality with python? I am not familiar enough with python or Overture to know, of the top of my head, if python offers the capability needed.
I ask because python is available on the back end of the BG/P at Argonne. Might such a transformation be of interest to you and the Overture team? My job description would allow me to engage considerably on such a task. 

Best,

Joe

On Mar 5, 2012, at 12:42 PM, Kyle K. Chand wrote:


Hi Joe,
Well, I have some bad news regarding perl. Perl is really an integral part of how we set up problems both for the grid generator as well as CG solver runs. Using Overture w/o perl is kind of like editing C++ files with ed in a terminal window; you can do it but it is not really suggested :-) Overture's graphics/ interpreter interface actually starts a perl interpreter and the text commands are first sent to the interpreter to either get string interpolation performed or perl statements evaluated. You can see an example of how this is used in $Overture/sampleGrids/ squareArg.cmd, which parameterizes an ogen 
grid generator command file using perl statements.
Now, it is probably possible to build Overture without the perl interpreter linked in, you will have to change OvertureParser.C as well as the bin/Makefile to remove references to perl. Then, however, you will need to strip perl from any command files you plan to use.
The best way to go would be to compile Perl for BGP. However, if that is not possible for some reason, here is another way to go: 1) Build a "full" version of Overture somewhere you can generate grids and test things out 2) Modify this "full" version to write the "plain text" command files after the perl processing is completed, run any command files you want through it to strip the perl 3) Build a "perl-less" version for BGP on which you can run perl- stripped cg command files
The easiest way to "strip" command files of perl might be to intercept the interpolated strings before they get sent to the rest of Overture from OvertureParser, I'll have to think about the least-intrusive way to do that. 

Regards,
Kyle
PS, I will try to get to your Mac question later today.

mailto: chand1@xxxxxxxx
phoneto: (925) 422 7740



On Mar 2, 2012, at 5:58 PM, J. P. Bernstein wrote:


Hello Bill,

Thank you and Kyle for the quick replies to my emails.
Warning headed. I am on assignment from the Argonne Leadership Computing Facility (ALCF) on this effort. There is significant interest at ALCF to at least run the CG solvers on BGP (not necessarily ogen). Nevertheless, 
I will share your sentiment with my assigners.
A question about the perl situation: my initial assessment suggested to me that perl was needed only for configuring the build and running the tests. Since perl is available on the front end, the configuration part should work. Part of my assignment is converting the perl test scripts 
to, e.g., python. However, your message below makes me doubt my
assessment. Are the perl libs, which, as you know, are not available on 
the back end, needed at run time beyond the test scripts?

Best,

Joe

On 3/2/2012 6:08 PM, Bill Henshaw wrote:

Hi Joe,
At one time I did get Overture compiled on the BGP front end at ANL
with xlc but I was left
with the issue that the perl libraries were not available with BGP. We 
could work around
this if necessary but I prefer not too if possible.
That said, experience tells me that since we have not run Overture on 
BGP then you
are likely to have problems that will be very hard for you to fix.
Therefore this is just a warning
that the path you are on may lead to great misery and despair ...

...Bill


J. P. Bernstein wrote:

Hello,

I am trying to build Overture.v24 on Blue Gene P (BG/P).
In an attempt to add a BG/P configuration for A++, I looked for the 
following config file mentioned in the A++ README file:
A++/config/config.options
but was not able to find the file or the directory mentioned in the 
A++ directory tree.
I was able to build A++ without errors with the following configure 
command:
./configure --with-CC=mpixlc_r --with-CFLAGS="-qarch=450d - qtune=450" 
--with-CXX=mpixlcxx_r --with-CXXFLAGS="-qarch=450d -qtune=450"
--prefix=`pwd`

However, the flags did not seem to be passed:
mpixlc_r -DHAVE_CONFIG_H -I. -I. -I../.. -I../../include -c 
`test -f common_func.c || echo './'`common_func.c

mpixlcxx_r           -o resize  resize.o
-L/home/jpbernst/Code/Overture/A++P++-0.8.0/A++/install/lib - L../src 
-lApp -lApp_static -lApp  -lm

Is this a known issue?

Returning to the BG/P config issue, I tried to hack :

config/A++P++_common_autoconfig.m4
config.sub
in order to add a Bg/P setup without success. Is there documentation available for adding a machine config? I was not able to find any on 
the Overture webpage.

Best,

Joe Bernstein

Other related posts:

  1. Home
  2. overture
  3. Archive
  4. 03-2012