[overture] Parallel Moving Grid Memory Usage
- From: Kyle Hord <kmhord01@xxxxxxxxx>
- To: overture@xxxxxxxxxxxxx
- Date: Wed, 13 Jun 2012 13:55:59 -0400
Dr. Henshaw,
I've been benchmarking the parallel moving grid code in v24b, and I noticed
the memory usage is somewhat erratic. I tested a simple 2D pitch and plunge
case with 100k grid points. When 1 processor was specified, the average
memory usage per core was 168 mb. For 2 processors, the average per core
was 653 mb, then for 4 – 970 mb, then 8 – 1933 mb. While I do expect memory
usage to go up per core when increasing the number of cores used, this
seems somewhat excessive. When moving grids is turned off, I did not see
this increase. I saw this problem with a fresh installation of 24b as with
a slightly modified version of 24b with different moving grids kinematics.
We’ve been able to replicate this run away memory usage on two our servers
(possibly a third).
I believe this may be related to an earlier error I was receiving on
another server of ours. When running moving grids, I would receive a “*Fatal
error in PMPI_Isend: Internal MPI error!, error stack:” *and cgins would
crash. I believe this is related to the cores of the machine running out of
memory resulting in the crash. I’ve run Valgrind to find memory leaks, but
no success. I never found a solution to this problem.
Thanks for any input,
Kyle
Other related posts:
