[FLASH-USERS] Understanding FLASH memory requirements

[Klaus Weide]

On Fri, 26 Apr 2019, Ryan Farber wrote:

> Hi Klaus,
> 
> Thanks for your response! I think I've got a handle on things now, but I
> typed out my thought process below in case it helps other people in the
> future.
> 
> > grid_space_gb = total_grid_vars*MAXBLOCKS*procs*(nxb+2*NGUARD)**NDIM /
> > > bytes_per_GB
> >
> > It seems you have forgotten a factor of 8, for size of a double precision
> > real in bytes.
> >
> > Good point! So I actually get ~730 GB memory required, though I'm
> overestimating since you mentioned that NFLUXES doesn't require a full
> block's worth of memory (why is that?). 

The flux arrays that are used directly by the Hydro unit are only 
declared / allocated for one block [they don't have something like 
',...,MAXBLOCKS)' in the dimensions], so they don't count here.

There are various arrays, both inside the PARAMESH code and the 
higher-level Grid unit code, to store fluxes in connection with flux 
correction (flux conservation). Those do scale with MAXBLOCKS. But they 
are not declared for all cells, but only (more or less) for cell faces at 
block faces. See the following files for the gory details (search for 
NFLUXES or nfluxes):

  amr_initialize.F90
  Grid/GridMain/paramesh/Grid_data.F90
  Grid/GridMain/paramesh/Grid_init.F90



(There is a line

  #define NVARS_TOTAL (NUNK_VARS + (NFACE_VARS * 3) + NFLUXES)

in Flash.h but that doesn't seem to be very useful - I don't see 
NVARS_TOTAL actually being used for anything.)


Klaus