<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<tt>I suggested to use the multigrid solver. It works for any
realistic problem.<br>
<br>
Tomek<br>
</tt>--<br>
<div class="moz-cite-prefix">On 08/11/16 13:06, Sean M. Couch wrote:<br>
</div>
<blockquote
cite="mid:etPan.57acb0a9.3e9008b4.1003f@flash.uchicago.edu"
type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style>body{font-family:Helvetica,Arial;font-size:13px}</style>
<div>
<blockquote type="cite" class="clean_bq" style="font-family:
Helvetica, Arial; font-size: 13px; font-style: normal;
font-variant-caps: normal; font-weight: normal;
letter-spacing: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
<span>
<div>
<div>Why not to use the original multipole solver? It is
not really any less<span class="Apple-converted-space"> </span><br>
accurate.<span class="Apple-converted-space"> </span></div>
</div>
</span></blockquote>
</div>
<p>Oh? Then I guess we shouldn’t have bothered to write this
whole paper about how much more accurate (and efficient) it is: <a
moz-do-not-send="true"
href="http://adsabs.harvard.edu/abs/2013ApJ...778..181C">http://adsabs.harvard.edu/abs/2013ApJ...778..181C</a>.
And going back to the old solver doesn’t address Rahul’s
problem at all. </p>
<p>Rahul, in the reference paper, we go to very high L_max, but in
2D only. Perhaps try setting mpole_3DAxisymmetry to .TRUE. in
your .par file. This is likely not an ultimate fix since you
need non-axisymmetric gravity, I assume, for your problem but it
could tell you if it is indeed the memory requirements of the
mpole arrays.</p>
<p><br>
</p>
<p>Sean</p>
<p><br>
</p>
<div>
<blockquote type="cite" class="clean_bq" style="font-family:
Helvetica, Arial; font-size: 13px; font-style: normal;
font-variant-caps: normal; font-weight: normal;
letter-spacing: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
<span>
<div>
<div>--<span class="Apple-converted-space"> </span><br>
On 08/11/16 12:41, Klaus Weide wrote:<span
class="Apple-converted-space"> </span><br>
> On Wed, 10 Aug 2016, Rahul Kashyap wrote:<span
class="Apple-converted-space"> </span><br>
><span class="Apple-converted-space"> </span><br>
>> Yes, I forgot to mention that I'm using new
multipole implementation with<span
class="Apple-converted-space"> </span><br>
>> 60 poles.<span class="Apple-converted-space"> </span><br>
>><span class="Apple-converted-space"> </span><br>
>> I have attached a small txt files with short
summary on three runs which<span
class="Apple-converted-space"> </span><br>
>> very well describes my problem. 1024 proc have
been used for all runs with<span
class="Apple-converted-space"> </span><br>
>> fixed lrefinemax and base blocks. I get three
differenet error for three<span
class="Apple-converted-space"> </span><br>
>> different maxblocks value.<span
class="Apple-converted-space"> </span><br>
>><span class="Apple-converted-space"> </span><br>
>> My understanding was that reasonable use of
maxblocks avoids any such<span
class="Apple-converted-space"> </span><br>
>> memory failures.<span
class="Apple-converted-space"> </span><br>
> Rahul,<span class="Apple-converted-space"> </span><br>
><span class="Apple-converted-space"> </span><br>
> It appears that the total memory required by<span
class="Apple-converted-space"> </span><br>
><span class="Apple-converted-space"> </span><br>
> PARAMESH Grid + multipole solver ( + Particles +
other units )<span class="Apple-converted-space"> </span><br>
><span class="Apple-converted-space"> </span><br>
> is just too much; I suspect that this is PRIMARILY
due to the memory<span class="Apple-converted-space"> </span><br>
> requirements of the Multipole(_new) solver.<span
class="Apple-converted-space"> </span><br>
><span class="Apple-converted-space"> </span><br>
> There are several large arrays allocated, see in
particular statements<span class="Apple-converted-space"> </span><br>
> like<span class="Apple-converted-space"> </span><br>
><span class="Apple-converted-space"> </span><br>
> allocate (gr_mpoleScratch
(1:gr_mpoleMaxLM,1:gr_mpoleMaxQ ),...)<span
class="Apple-converted-space"> </span><br>
><span class="Apple-converted-space"> </span><br>
> in gr_mpoleAllocateRadialArrays.F90, where
gr_mpoleMaxQ may be very large<span
class="Apple-converted-space"> </span><br>
> and gr_mpoleMaxLM ~ gr_mpoleMaxL ** 2 ( == 60**2 in
your case?).<span class="Apple-converted-space"> </span><br>
><span class="Apple-converted-space"> </span><br>
> Unfortunately this memory is required for each
process, you cannot reduce<span
class="Apple-converted-space"> </span><br>
> this by running on more procs.<span
class="Apple-converted-space"> </span><br>
><span class="Apple-converted-space"> </span><br>
> It makes sense in general to try to reduce the
memory required by the<span
class="Apple-converted-space"> </span><br>
> PARAMESH Grid by lowering maxblocks, but this can
go only so far;<span class="Apple-converted-space"> </span><br>
> maxblocks has to leave room for "a few" more blocks
than the number<span class="Apple-converted-space"> </span><br>
> actually required by the distributed grid. These
additional slots<span class="Apple-converted-space"> </span><br>
> are needed for temporary storage during processing
by some internal<span class="Apple-converted-space"> </span><br>
> PARAMESH routines for things like block
redistribution. I don't<span
class="Apple-converted-space"> </span><br>
> know of a reliable way to predict how low "a few"
can go in a given<span class="Apple-converted-space"> </span><br>
> case, so this has to be determined empirically.
Apparently,<span class="Apple-converted-space"> </span><br>
> this was too low in your maxblocks=5 case.<span
class="Apple-converted-space"> </span><br>
><span class="Apple-converted-space"> </span><br>
> It may be possible to tweak the maxblocks value
further, possibly in<span class="Apple-converted-space"> </span><br>
> connection with also modifying the values of
maxblocks_alloc and<span class="Apple-converted-space"> </span><br>
> maxblocks_tr (see amr_initialize.F90 and
paramesh_dimensions.F90),<span
class="Apple-converted-space"> </span><br>
> in order to allow the Grid domain initialization to
proceed with<span class="Apple-converted-space"> </span><br>
> maxblocks < 10; but this may then still not give
you enough free memory<span
class="Apple-converted-space"> </span><br>
> for the multipole solver (and particles, etc).<span
class="Apple-converted-space"> </span><br>
><span class="Apple-converted-space"> </span><br>
> So you should investigate ways to lower the memory
requirements of the<span class="Apple-converted-space"> </span><br>
> Poisson solver; you may have to lower the
resolution (not sure which<span
class="Apple-converted-space"> </span><br>
> runtime parameters to change), or perhaps use a
different implementation.<span
class="Apple-converted-space"> </span><br>
><span class="Apple-converted-space"> </span><br>
> Klaus<span class="Apple-converted-space"> </span><br>
<br>
</div>
</div>
</span></blockquote>
</div>
</blockquote>
<br>
</body>
</html>