[FLASH-USERS] Understanding FLASH memory requirements

[Ryan Farber]

Hi Klaus,

Thanks for your response! I think I've got a handle on things now, but I
typed out my thought process below in case it helps other people in the
future.

> grid_space_gb = total_grid_vars*MAXBLOCKS*procs*(nxb+2*NGUARD)**NDIM /
> > bytes_per_GB
>
> It seems you have forgotten a factor of 8, for size of a double precision
> real in bytes.
>
> Good point! So I actually get ~730 GB memory required, though I'm
overestimating since you mentioned that NFLUXES doesn't require a full
block's worth of memory (why is that?). Conversely, there's some overhead
due to AMR that I'm neglecting. I understand now that rss*procs (~300 GB)
doesn't track all the memory that is being used but it's sensible then that
the memory required is greater than rss*procs.

Conversely, vsize*procs ~ 1476GB is a bit greater than the RAM available to
30 nodes (1440 GB), so I must be dipping into swap for even the "good" 360
procs, 30 nodes, maxblocks=500 run.

On the other hand, in the 180 procs, 15 nodes, maxblocks=1000 case,
vsize*procs ~ 1270GB whereas I should only have 720 GB of RAM available, so
it makes sense that it crashed (even from the memory calculation estimate
~730 GB point of view).

I realized I had previously tried one additional node (16) by restarting
from chk 0000 to bypass initialization (takes ~ 5 hours, rather a long
time). I don't know why that would be a problem, but in case it is, I'm
trying again with 16 nodes (180 procs, maxblocks=1000) no restart since the
memory requirement estimation suggests 16 nodes should be sufficient.

I actually didn't have any subsequent memory numbers output in the log file
(until I did a restart that is and it was there similar to what I posted
last time). I found that io_memoryReport.F90 is responsible for that log
file output so I'll call that every step in Driver_evolveFlash (my problem
directory's copy) to see how much it changes between initialization and
evolution.

Best,
--------
Ryan Farber
Graduate Student
University of Michigan, Ann Arbor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20190426/271bd95b/attachment.htm>