Hi Joe,
Really you will be divorcing Overture and CG from perl, which is doable.
Stripping calls from Overture is easy, you just need to hack/replace
Ogshow/OvertureParser.C when building it on the BG/P backend. Generating
perl-less command files on your mac will not be that hard though it may involve
some adjustments to the way overture's command interpreter records the playback
commands. It may also be possible to do it with a simple perl script that
reads a command file line by line, interprets each line ending in a semi-colon
and string-interpolates the remaining lines, outputting the interpolated lines
to a perl-less command file.
bpp should not be an issue. It is used to generate code at compile time (only
when bpp files have changed) and is never executed at run time. If executing
bpp on the frontend when compiling cg is an issue you can just 'touch' all the
source files so that the bpp dependencies are not activated in the build
system. If that does not work then the build system will have to be adjusted
to ignore regeneration of bpp generated source code.
Regards,
Kyle
________________________________________
From: overture-bounce@xxxxxxxxxxxxx [overture-bounce@xxxxxxxxxxxxx] On Behalf
Of J. P. Bernstein [astro@xxxxxxx]
Sent: Wednesday, March 28, 2012 11:43 AM
To: overture@xxxxxxxxxxxxx
Subject: [overture] Re: Building v24 on Blue Gene P
Hello Kyle,
I have confirmed with ALCF management that building perl on the backend of the
Argonne BG/P is not and will not be supported by the facility due to overhead
issues.
What we are really after is having CG built on the BG/P backend. To us, the
most sensible path is to build Overture somewhere else (like my Mac laptop,
which has been done), and use it to generate perl-less command files to pass to
a CG build on the BG/P backend. Thus, I would like to divorce CG from Overture
to the greatest extent possible. How realistic is that goal, from the
perspective of the Overture developers?
The immediate issue that arises with the above is CG's dependency on BPP. How
much does CG depended on the extended capabilities of BPP, relative to CPP. In
other words, would building BPP on it's own or reverting CG to work with CPP
make more sense?
Best,
Joe
----- Original Message -----
From: "Kyle K. Chand"<chand1@xxxxxxxx>
To: overture@xxxxxxxxxxxxx
Sent: Monday, March 5, 2012 4:24:19 PM
Subject: [overture] Re: Building v24 on Blue Gene P
Hi Joe,
Mechanically, it would not be hard to replace the perl interpreter in
OvertureParser.C with the python interpreter (or any other
interpreter, check out Lua!).
However, it may be a great deal of effort to translate all our command
files from perl to python. We do not use perl for regular
expressions, we use perl for two
purposes:
1) writing perl code to parameterize and generate input to our grid
generator and solvers, and
2) utilize perl's "string interpolation" mechanism to turn command
file lines like:
$N=11;
number of grid lines
$N $N
into
number of grid lines
11 11
Lately we have been using 1) a lot more, often writing perl loops and
including perl modules to parameterize and automate input generation
and make things generally more maintainable.
The mechanism represented by 2) is how 1) eventually gets used. So,
it would be a lot of work to re-write many of our more recent (and
future) command files to use another interpreted language.
I think the most efficient path would be to strip perl from the BG
port of the code. Then use Overture on some other machine to generate
perl-less, pure text, input for the problems you want to run. One
could even
write a (perl :-) script that reads a command file and sends each line
ending with a semi-colon to the interpreter and every other line for
string interpolation, echoing the interpolated line to a "stripped"
command file. You would not even need another copy of Overture for
that.
Best regards,
Kyle
mailto: chand1@xxxxxxxx
phoneto: (925) 422 7740
On Mar 5, 2012, at 12:09 PM, J. P. Bernstein wrote:
Thank you very much, Kyle, for those details.
While I am digesting that, would you care to comment on the notion
of replacing perl functionality with python? I am not familiar
enough with python or Overture to know, of the top of my head, if
python offers the capability needed.
I ask because python is available on the back end of the BG/P at
Argonne. Might such a transformation be of interest to you and the
Overture team? My job description would allow me to engage
considerably on such a task.
Best,
Joe
On Mar 5, 2012, at 12:42 PM, Kyle K. Chand wrote:
Hi Joe,
Well, I have some bad news regarding perl. Perl is really an
integral part of how we set up problems both for the
grid generator as well as CG solver runs. Using Overture w/o perl
is kind of like editing C++ files with ed in a terminal window; you
can do it but it is not really suggested :-) Overture's graphics/
interpreter interface actually starts a perl interpreter
and the text commands are first sent to the interpreter to either
get string interpolation performed or perl statements evaluated.
You can see an example of how this is used in $Overture/sampleGrids/
squareArg.cmd, which parameterizes an ogen
grid generator command file using perl statements.
Now, it is probably possible to build Overture without the perl
interpreter linked in, you will have to change OvertureParser.C as
well
as the bin/Makefile to remove references to perl. Then, however,
you will need to strip perl from any command files you plan to use.
The best way to go would be to compile Perl for BGP. However, if
that is not possible for some reason, here is another way to go:
1) Build a "full" version of Overture somewhere you can generate
grids and test things out
2) Modify this "full" version to write the "plain text" command
files after the perl processing is completed, run any command files
you want through it to strip the perl
3) Build a "perl-less" version for BGP on which you can run perl-
stripped cg command files
The easiest way to "strip" command files of perl might be to
intercept the interpolated strings before they get sent to the rest
of Overture
from OvertureParser, I'll have to think about the least-intrusive
way to do that.
Regards,
Kyle
PS, I will try to get to your Mac question later today.
mailto: chand1@xxxxxxxx
phoneto: (925) 422 7740
On Mar 2, 2012, at 5:58 PM, J. P. Bernstein wrote:
Hello Bill,
Thank you and Kyle for the quick replies to my emails.
Warning headed. I am on assignment from the Argonne Leadership
Computing
Facility (ALCF) on this effort. There is significant interest at
ALCF to
at least run the CG solvers on BGP (not necessarily ogen).
Nevertheless,
I will share your sentiment with my assigners.
A question about the perl situation: my initial assessment
suggested to
me that perl was needed only for configuring the build and running
the
tests. Since perl is available on the front end, the configuration
part
should work. Part of my assignment is converting the perl test
scripts
to, e.g., python. However, your message below makes me doubt my
assessment. Are the perl libs, which, as you know, are not
available on
the back end, needed at run time beyond the test scripts?
Best,
Joe
On 3/2/2012 6:08 PM, Bill Henshaw wrote:
Hi Joe,
At one time I did get Overture compiled on the BGP front end at ANL
with xlc but I was left
with the issue that the perl libraries were not available with
BGP. We
could work around
this if necessary but I prefer not too if possible.
That said, experience tells me that since we have not run
Overture on
BGP then you
are likely to have problems that will be very hard for you to fix.
Therefore this is just a warning
that the path you are on may lead to great misery and despair ...
...Bill
J. P. Bernstein wrote:
Hello,
I am trying to build Overture.v24 on Blue Gene P (BG/P).
In an attempt to add a BG/P configuration for A++, I looked for
the
following config file mentioned in the A++ README file:
A++/config/config.options
but was not able to find the file or the directory mentioned in
the
A++ directory tree.
I was able to build A++ without errors with the following
configure
command:
./configure --with-CC=mpixlc_r --with-CFLAGS="-qarch=450d -
qtune=450"
--with-CXX=mpixlcxx_r --with-CXXFLAGS="-qarch=450d -qtune=450"
--prefix=`pwd`
However, the flags did not seem to be passed:
mpixlc_r -DHAVE_CONFIG_H -I. -I. -I../.. -I../../include
-c
`test -f common_func.c || echo './'`common_func.c
mpixlcxx_r -o resize resize.o
-L/home/jpbernst/Code/Overture/A++P++-0.8.0/A++/install/lib -
L../src
-lApp -lApp_static -lApp -lm
Is this a known issue?
Returning to the BG/P config issue, I tried to hack :
config/A++P++_common_autoconfig.m4
config.sub
in order to add a Bg/P setup without success. Is there
documentation
available for adding a machine config? I was not able to find
any on
the Overture webpage.
Best,
Joe Bernstein
