[FLASH-USERS] FLASH memory usage: Static, Dynamic and Machine Specific

[Norbert Flocke]

It uses much more memory and maxes out at about Lmax = 8 due to numerical 
instabilities. This was the main reason for developing a new multipole 
solver.

On Thu, 11 Aug 2016, Tomasz Plewa wrote:

> Why not to use the original multipole solver? It is not really any less 
> accurate.
>
> Tomek
> --
> On 08/11/16 12:41, Klaus Weide wrote:
>> On Wed, 10 Aug 2016, Rahul Kashyap wrote:
>> 
>>> Yes, I forgot to mention that I'm using new multipole implementation with
>>> 60 poles.
>>> 
>>> I have attached a small txt files with short summary on three runs which
>>> very well describes my problem. 1024 proc have been used for all runs with
>>> fixed lrefinemax and base blocks. I get three differenet error for three
>>> different maxblocks value.
>>> 
>>> My understanding was that reasonable use of maxblocks avoids any such
>>> memory failures.
>> Rahul,
>> 
>> It appears that the total memory required by
>>
>>    PARAMESH Grid + multipole solver ( + Particles + other units )
>> 
>> is just too much; I suspect that this is PRIMARILY due to the memory
>> requirements of the Multipole(_new) solver.
>> 
>> There are several large arrays allocated, see in particular statements
>> like
>>
>>    allocate (gr_mpoleScratch (1:gr_mpoleMaxLM,1:gr_mpoleMaxQ  ),...)
>> 
>> in gr_mpoleAllocateRadialArrays.F90, where gr_mpoleMaxQ may be very large
>> and gr_mpoleMaxLM ~ gr_mpoleMaxL ** 2 ( == 60**2 in your case?).
>> 
>> Unfortunately this memory is required for each process, you cannot reduce
>> this by running on more procs.
>> 
>> It makes sense in general to try to reduce the memory required by the
>> PARAMESH Grid by lowering maxblocks, but this can go only so far;
>> maxblocks has to leave room for "a few" more blocks than the number
>> actually required by the distributed grid. These additional slots
>> are needed for temporary storage during processing by some internal
>> PARAMESH routines for things like block redistribution. I don't
>> know of a reliable way to predict how low "a few" can go in a given
>> case, so this has to be determined empirically. Apparently,
>> this was too low in your maxblocks=5 case.
>> 
>> It may be possible to tweak the maxblocks value further, possibly in
>> connection with also modifying the values of maxblocks_alloc and
>> maxblocks_tr (see amr_initialize.F90 and paramesh_dimensions.F90),
>> in order to allow the Grid domain initialization to proceed with
>> maxblocks < 10; but this may then still not give you enough free memory
>> for the multipole solver (and particles, etc).
>> 
>> So you should investigate ways to lower the memory requirements of the
>> Poisson solver; you may have to lower the resolution (not sure which
>> runtime parameters to change), or perhaps use a different implementation.
>> 
>> Klaus
>
>