[FLASH-USERS] FLASH memory usage: Static, Dynamic and Machine Specific

[Tomasz Plewa]

This may not be the case knowing Rahul's past work.

Tomek
--
On 08/11/16 13:21, Carlo Graziani wrote:
> I agree that multipole self-gravity is not a good choice for "general"
> mass distributions.  But for distributions with large central mass concentrations --
> basically, stars -- multipole self-gravity is clearly an optimal choice that should
> lead to a better balance of accuracy and computational efficiency than multigrid.
>
> Carlo
>
> On 08/11/2016 12:10 PM, Tomasz Plewa wrote:
>> Accuracy and functionality does not necessarily come together. If one
>> really needs accurate solutions for general mass distributions, there is
>> no more accurate method than the multigrid.
>>
>> Tomek
>> --
>> On 08/11/16 13:00, Carlo Graziani wrote:
>>> Actually, the old solver is considerably less accurate than the new one,
>>> particularly for large multipole cutoff values.  In fact, the old solver -- the
>>> old algorithm, for that matter -- fails altogether to converge in l. The new
>>> solver does in fact converge.
>>>
>>> Furthermore, the optimization of the center of the expansion implemented in the new
>>> solver is designed to accelerate the convergence with l, so that it should not
>>> be necessary to go to lmax=60 in most cases.  All this is described in
>>> https://arxiv.org/abs/1307.3135.
>>>
>>> Note also that if you're running a 3D problem, the array sizes scale as (lmax+1)^2,
>>> so part of your problem may be due to choosing an excessive lmax.  You could
>>> try cutting it down a bit, to see if the problem fits.
>>>
>>> Carlo
>>>
>>> On 08/11/2016 11:47 AM, Tomasz Plewa wrote:
>>>> Why not to use the original multipole solver? It is not really any less
>>>> accurate.
>>>>
>>>> Tomek
>>>> --
>>>> On 08/11/16 12:41, Klaus Weide wrote:
>>>>> On Wed, 10 Aug 2016, Rahul Kashyap wrote:
>>>>>
>>>>>> Yes, I forgot to mention that I'm using new multipole implementation with
>>>>>> 60 poles.
>>>>>>
>>>>>> I have attached a small txt files with short summary on three runs which
>>>>>> very well describes my problem. 1024 proc have been used for all runs with
>>>>>> fixed lrefinemax and base blocks. I get three differenet error for three
>>>>>> different maxblocks value.
>>>>>>
>>>>>> My understanding was that reasonable use of maxblocks avoids any such
>>>>>> memory failures.
>>>>> Rahul,
>>>>>
>>>>> It appears that the total memory required by
>>>>>
>>>>>      PARAMESH Grid + multipole solver ( + Particles + other units )
>>>>>
>>>>> is just too much; I suspect that this is PRIMARILY due to the memory
>>>>> requirements of the Multipole(_new) solver.
>>>>>
>>>>> There are several large arrays allocated, see in particular statements
>>>>> like
>>>>>
>>>>>      allocate (gr_mpoleScratch (1:gr_mpoleMaxLM,1:gr_mpoleMaxQ  ),...)
>>>>>
>>>>> in gr_mpoleAllocateRadialArrays.F90, where gr_mpoleMaxQ may be very large
>>>>> and gr_mpoleMaxLM ~ gr_mpoleMaxL ** 2 ( == 60**2 in your case?).
>>>>>
>>>>> Unfortunately this memory is required for each process, you cannot reduce
>>>>> this by running on more procs.
>>>>>
>>>>> It makes sense in general to try to reduce the memory required by the
>>>>> PARAMESH Grid by lowering maxblocks, but this can go only so far;
>>>>> maxblocks has to leave room for "a few" more blocks than the number
>>>>> actually required by the distributed grid. These additional slots
>>>>> are needed for temporary storage during processing by some internal
>>>>> PARAMESH routines for things like block redistribution. I don't
>>>>> know of a reliable way to predict how low "a few" can go in a given
>>>>> case, so this has to be determined empirically. Apparently,
>>>>> this was too low in your maxblocks=5 case.
>>>>>
>>>>> It may be possible to tweak the maxblocks value further, possibly in
>>>>> connection with also modifying the values of maxblocks_alloc and
>>>>> maxblocks_tr (see amr_initialize.F90 and paramesh_dimensions.F90),
>>>>> in order to allow the Grid domain initialization to proceed with
>>>>> maxblocks < 10; but this may then still not give you enough free memory
>>>>> for the multipole solver (and particles, etc).
>>>>>
>>>>> So you should investigate ways to lower the memory requirements of the
>>>>> Poisson solver; you may have to lower the resolution (not sure which
>>>>> runtime parameters to change), or perhaps use a different implementation.
>>>>>
>>>>> Klaus
>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: tplewa.vcf
Type: text/x-vcard
Size: 344 bytes
Desc: not available
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20160811/c6e52831/attachment-0001.vcf>