<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><br class=""></div><div class="">Hi Zheng,</div><div class=""><br class=""></div><div class="">the Sedov test is not really a good test to check file compression. As pointed out earlier, there are large patches that have the same values of density, velocity, etc., so that will compress well. In contrast, a fully-developed turbulent field of density, velocity, pressure, etc will be very hard to compress. I recommend to use the turbulence stirring unit to create FLASH output files with fully-developed turbulence and try your compression algorithm on that. You can simply look into Section 16.3 in the user guide and specifically into 16.3.2-16.3.4, which contains a unit test setup that you can modify to create high-resolution (big) output files. I’m also happy to make some of my own existing checkpoint files available—the biggest files that I’m currently producing with this setup are 19 TB from a simulation with 10000^3 grid cells.</div><div class=""><br class=""></div><div class="">Kind regards</div><div class="">Christoph</div><div class=""><br class=""></div><div class=""><div class="">—</div><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">Christoph Federrath</div><div class=""><a href="http://www.mso.anu.edu.au/~chfeder" class="">http://www.mso.anu.edu.au/~chfeder</a></div></div></div></div><div class=""><br class=""></div><div class=""><blockquote type="cite" class=""><div class="">On 3 Mar 2016, at 05:14, Zheng Yuan <<a href="mailto:zhengyuan2014@u.northwestern.edu" class="">zhengyuan2014@u.northwestern.edu</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="">yes... I run FLASH using 32*32 dim for 200 iterations. The compression ratios of the first 20 iterations are extreme high. How much time should I simulate in order to get the 'fully developed turbulence field'? (My major is computer science. Sorry for this naive question..)<br class=""><br class="">Best<br class="">Zheng<br class=""><br class="">On 3/2/16 11:56 AM, Dean Townsley wrote:<br class=""><blockquote type="cite" class="">Zheng,<br class=""><br class="">Also, if you are testing compression algorithms, you should consider the physical state of the data you are compressing.<br class=""><br class="">The initial conditions or state after only a few time steps of something like the Sod problem or Sedov will be very easily compressible because there are large regions on the grid that are in a spatially uniform state. Conversely, a fully developed turbulence field (e.g. StirTurb run for a decent fraction of a turnover time) is likely to be harder to compress.<br class=""><br class="">Dean<br class=""><br class="">On 03/02/2016 11:47 AM, Zheng Yuan wrote:<br class=""><blockquote type="cite" class="">Thanks Klaus! I will use 3D instead of 2D. Someone also suggested that larger values of lrefine_min & lrefine_max results in larger checkpoint file sizes. I will use 3D & larger Irefine_min&max, while keeping nxb*nyb*nzb = 8*8*8.<br class=""><br class="">Best<br class="">Zheng<br class=""><br class="">On 3/2/16 11:23 AM, Klaus Weide wrote:<br class=""><blockquote type="cite" class="">On Tue, 1 Mar 2016, Zheng Yuan wrote:<br class=""><br class=""><blockquote type="cite" class="">Dear all,<br class=""><br class="">My goal is to run FLASH using 1K MPI processes and get large checkpoint files<br class="">(500 GB each. These large checkpoint files will be used as the test data of a<br class="">parallel data compression algorithm proposed by our group.). Does anybody know<br class="">how to set the parameters to run FLASH in large scale and get large checkpoint<br class="">files?<br class=""></blockquote>Hello,<br class=""><br class="">If you really "just" want large checkpoint files, you should start with a 3D<br class="">setup!<br class=""><br class=""><br class=""><blockquote type="cite" class="">Currently, I am trying to run FLASH/Sedov to generate large checkpoint files.<br class="">However, FLASH reports an error when I increase the dimension of the grid.<br class=""><br class="">I configured FLASH using:<br class=""><br class="">./setup Sedov -auto +pnetcdf -objdir=sedov_2d -parfile=sedov_io_69b_2d.par -2d<br class="">-nxb=64 -nyb=64 -maxblocks=200<br class=""><br class="">To run more iterations, I changed the 'nend' parameter to 10 in<br class="">sedov_2d/flash.par and make sedov_2d<br class=""><br class="">I run flash using command:<br class=""><br class="">mpirun -n 4 ./flash4<br class=""><br class=""><br class="">FLASH terminate at the third iteration. The error is:<br class=""></blockquote> *** Wrote plotfile to sedov_2d_6lev_ncmpi_plt_cnt_0001 ****<br class=""> n t dt ( x, y, z) | dt_hydro<br class=""> 1 2.0000E-10 2.5000E-05 ( 4.993E-01, 4.875E-01, 0.000E+00) | 1.265E-05<br class=""> *** Wrote checkpoint file to sedov_2d_6lev_ncmpi_chk_0001 ****<br class=""> *** Wrote plotfile to sedov_2d_6lev_ncmpi_plt_cnt_0002 ****<br class=""> 2 5.0000E-05 2.5000E-05 ( 5.134E-01, 5.017E-01, 0.000E+00) | 8.673E-06<br class=""> *** Wrote checkpoint file to sedov_2d_6lev_ncmpi_chk_0002 ****<br class=""> *** Wrote plotfile to sedov_2d_6lev_ncmpi_plt_cnt_0003 ****<br class=""> 3 1.0000E-04 2.5000E-05 ( 5.139E-01, 4.988E-01, 0.000E+00) | 6.407E-07<br class=""> *** Wrote checkpoint file to sedov_2d_6lev_ncmpi_chk_0003 ****<br class=""> Nonconvergence in subroutine rieman<br class=""> .....<br class=""><br class=""><br class="">By increasing NXB and NYB, you have increased the resolution. This<br class="">requires a smaller timestep for stability, see "dt_hydro" column. But<br class="">your flash.par fixes dtmin and dtmax at 2.5e-5, the direct effect of<br class="">this can be seen in the "dt" column. So you are forcing FLASH to do<br class="">Hydro advances with a time step that is too large for stability.<br class="">So it is not surprising that the simulation fails after a few steps.<br class=""><br class="">Basically, aking your runtime parameter "dtmin" much smaller should<br class="">get you a simulation that runs.<br class=""><br class="">Btw, a more common way to increase resolution (thus file sizes) would<br class="">be to increase lrefine_in and/or lrefine_max and/or Nblock{X,Y,Z},<br class="">rather than keep increasing the block size.<br class=""><br class="">Klaus<br class=""></blockquote><br class=""><br class=""></blockquote><br class=""></blockquote><br class=""></div></div></blockquote></div><br class=""></body></html>