[FLASH-USERS] How to generate big checkpoint files in FLASH

[Zheng Yuan]

yes... I run FLASH using 32*32 dim for 200 iterations. The compression 
ratios of the first 20 iterations are extreme high. How much time should 
I simulate in order to get the 'fully developed turbulence field'? (My 
major is computer science. Sorry for this naive question..)

Best
Zheng

On 3/2/16 11:56 AM, Dean Townsley wrote:
> Zheng,
>
> Also, if you are testing compression algorithms, you should consider 
> the physical state of the data you are compressing.
>
> The initial conditions or state after only a few time steps of 
> something like the Sod problem or Sedov will be very easily 
> compressible because there are large regions on the grid that are in a 
> spatially uniform state.  Conversely, a fully developed turbulence 
> field (e.g. StirTurb run for a decent fraction of a turnover time) is 
> likely to be harder to compress.
>
> Dean
>
> On 03/02/2016 11:47 AM, Zheng Yuan wrote:
>> Thanks Klaus! I will use 3D instead of 2D. Someone also suggested 
>> that larger values of lrefine_min & lrefine_max results in larger 
>> checkpoint file sizes. I will use 3D & larger Irefine_min&max, while 
>> keeping nxb*nyb*nzb = 8*8*8.
>>
>> Best
>> Zheng
>>
>> On 3/2/16 11:23 AM, Klaus Weide wrote:
>>> On Tue, 1 Mar 2016, Zheng Yuan wrote:
>>>
>>>> Dear all,
>>>>
>>>> My goal is to run FLASH using 1K MPI processes and get large 
>>>> checkpoint files
>>>> (500 GB each. These large checkpoint files will be used as the test 
>>>> data of a
>>>> parallel data compression algorithm proposed by our group.). Does 
>>>> anybody know
>>>> how to set the parameters to run FLASH in large scale and get large 
>>>> checkpoint
>>>> files?
>>> Hello,
>>>
>>> If you really "just" want large checkpoint files, you should start 
>>> with a 3D
>>> setup!
>>>
>>>
>>>> Currently, I am trying to run FLASH/Sedov to generate large 
>>>> checkpoint files.
>>>> However, FLASH reports an error when I increase the dimension of 
>>>> the grid.
>>>>
>>>> I configured FLASH using:
>>>>
>>>> ./setup Sedov -auto +pnetcdf -objdir=sedov_2d 
>>>> -parfile=sedov_io_69b_2d.par -2d
>>>> -nxb=64 -nyb=64 -maxblocks=200
>>>>
>>>> To run more iterations, I changed the 'nend' parameter to 10 in
>>>> sedov_2d/flash.par and make sedov_2d
>>>>
>>>> I run flash using command:
>>>>
>>>> mpirun -n 4 ./flash4
>>>>
>>>>
>>>> FLASH terminate at the third iteration. The error is:
>>>   *** Wrote plotfile to sedov_2d_6lev_ncmpi_plt_cnt_0001 ****
>>>         n          t         dt  (         x, y,          z) |  
>>> dt_hydro
>>>         1 2.0000E-10 2.5000E-05  ( 4.993E-01,  4.875E-01, 0.000E+00) 
>>> |  1.265E-05
>>>   *** Wrote checkpoint file to sedov_2d_6lev_ncmpi_chk_0001 ****
>>>   *** Wrote plotfile to sedov_2d_6lev_ncmpi_plt_cnt_0002 ****
>>>         2 5.0000E-05 2.5000E-05  ( 5.134E-01,  5.017E-01, 0.000E+00) 
>>> |  8.673E-06
>>>   *** Wrote checkpoint file to sedov_2d_6lev_ncmpi_chk_0002 ****
>>>   *** Wrote plotfile to sedov_2d_6lev_ncmpi_plt_cnt_0003 ****
>>>         3 1.0000E-04 2.5000E-05  ( 5.139E-01,  4.988E-01, 0.000E+00) 
>>> |  6.407E-07
>>>   *** Wrote checkpoint file to sedov_2d_6lev_ncmpi_chk_0003 ****
>>>      Nonconvergence in subroutine rieman
>>>   .....
>>>
>>>
>>> By increasing NXB and NYB, you have increased the resolution. This
>>> requires a smaller timestep for stability, see "dt_hydro" column.  But
>>> your flash.par fixes dtmin and dtmax at 2.5e-5, the direct effect of
>>> this can be seen in the "dt" column. So you are forcing FLASH to do
>>> Hydro advances with a time step that is too large for stability.
>>> So it is not surprising that the simulation fails after a few steps.
>>>
>>> Basically, aking your runtime parameter "dtmin" much smaller should
>>> get you a simulation that runs.
>>>
>>> Btw, a more common way to increase resolution (thus file sizes) would
>>> be to increase lrefine_in and/or lrefine_max and/or Nblock{X,Y,Z},
>>> rather than keep increasing the block size.
>>>
>>> Klaus
>>
>>
>