[FLASH-USERS] Strange discrepancy in Dark Matter structure of cosmological simulation

Sat Sep 7 10:20:12 EDT 2013

Dear FLASH users and devs,

We have been conducting some comparisons of cosmological simulations conducted with FLASH 4.0.1, Gadget-2 and Enzo. We set up a cosmological volume of 60 Mpc/h box size with Planck cosmological parameters and evolve these initial conditions in all three codes. For the grid based codes (FLASH, Enzo) we used 256^3 resolution, and for Gadget-2 we used 256^3 DM particles and 256^3 gas particles.

In doing so we noticed that the dark matter structure FLASH shows a significant difference to both Enzo and Gadget-2, while those two codes show practically perfect agreement. I have attached some projection plots at redshift z=0 to show you the differences. As you can see, it seems as if the DM structure is less evolved in FLASH. To see, if this is discrepancy is specific to FLASH 4.0.1, we also tried FLASH 3.1, but there is no change to be seen there.

Since this simulation basically only includes cosmology, self-gravity, active particles and hydro, we can only make out three possible points, which could influence the DM evolution: the cosmology, the gravity or the LeapfrogCosmo unit.

We tried to check the cosmology unit by comparing the dt/dz, z, t and tend values and the timesteps that the simulation takes, but everything is in very good agreement with our calculations and also Enzo. So at least in terms of general cosmology there doesn't seem to be a problem there.

For checking the gravity unit, we switched from Multigrid to bhtree, but this did change absolutely nothing - the agreement between the two gravity units is very good. So we are inclined to exclude the gravity unit as the culprit.

This leaves us basically with the LeapFrogCosmo unit as the only point left, where things could go wrong.

I have checked everything that I can think of in terms of reading the ICs wrongly into FLASH, but all my checks turn out well (overall mass, min/max/avg density, particle masses, crit dens,...). To further exclude me as a source of error, a colleague also started with a FLASH 4.0.1 vanilla version from scratch and implemented the IC reading himself, but this also leads to the same results. The ICs have been filled in by assuming both distances and masses to be comoving cgs, but with the configured cosmology (aka not "over h").

Since the structures seemed less evolved to us, we decided to try running the simulation into the future and set redshift z = -0.5 as the final z. The result is given as the sim1-future* attachment, and to our surprise, at this point the FLASH structures match the other two codes structures (with those at z=0).

We would be thankful for any insights into this, since we are at our wits end.

Cheers,
Josef

The attachments can also be found at http://astro-staff.uibk.ac.at/~josef/work/comparison/flash-users/ together with the initial condition gadget-2 snapshot file ics_* and the resulting FLASH checkpoint 0000 file for simulation 1.

Attachments:
sim1_*   ... the simulation 1 comparison projection plots
sim2_*   ... the simulation 2 comparison projection plots
sim1-future_*   ... simulation 1 evolved until z = -0.5
flash.par   ... simulation 1 flash.par with comments removed

----------------------------------------------
Dr. Josef Stöckl
Institute of Astro- and Particle Physics
Technikerstrasse 25/8
6020 Innsbruck
Austria

Phone:  +43 512 507 52020
E-Mail: josef.stoeckl at uibk.ac.at
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20130907/997d54ad/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sim1_enzo_Projection_z_Density.png
Type: image/png
Size: 449833 bytes
Desc: sim1_enzo_Projection_z_Density.png
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20130907/997d54ad/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sim1_enzo_Projection_z_dm_density.png
Type: image/png
Size: 478218 bytes
Desc: sim1_enzo_Projection_z_dm_density.png
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20130907/997d54ad/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sim1_flash4_Projection_z_Density.png
Type: image/png
Size: 399180 bytes
Desc: sim1_flash4_Projection_z_Density.png
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20130907/997d54ad/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sim1_flash4_Projection_z_pden.png
Type: image/png
Size: 419489 bytes
Desc: sim1_flash4_Projection_z_pden.png
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20130907/997d54ad/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sim1_gadget_Projection_z_Density.png
Type: image/png
Size: 640116 bytes
Desc: sim1_gadget_Projection_z_Density.png
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20130907/997d54ad/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sim1_gadget_Projection_z_dm_density.png
Type: image/png
Size: 622341 bytes
Desc: sim1_gadget_Projection_z_dm_density.png
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20130907/997d54ad/attachment-0005.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sim1-future_flash4_Projection_z_Density.png
Type: image/png
Size: 422247 bytes
Desc: sim1-future_flash4_Projection_z_Density.png
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20130907/997d54ad/attachment-0006.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sim1-future_flash4_Projection_z_pden.png
Type: image/png
Size: 490337 bytes
Desc: sim1-future_flash4_Projection_z_pden.png
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20130907/997d54ad/attachment-0007.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sim2_enzo_Projection_z_Density.png
Type: image/png
Size: 489382 bytes
Desc: sim2_enzo_Projection_z_Density.png
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20130907/997d54ad/attachment-0008.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sim2_enzo_Projection_z_dm_density.png
Type: image/png
Size: 482896 bytes
Desc: sim2_enzo_Projection_z_dm_density.png
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20130907/997d54ad/attachment-0009.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sim2_flash4_Projection_z_Density.png
Type: image/png
Size: 410936 bytes
Desc: sim2_flash4_Projection_z_Density.png
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20130907/997d54ad/attachment-0010.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sim2_flash4_Projection_z_pden.png
Type: image/png
Size: 493500 bytes
Desc: sim2_flash4_Projection_z_pden.png
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20130907/997d54ad/attachment-0011.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sim2_gadget_Projection_z_Density.png
Type: image/png
Size: 643983 bytes
Desc: sim2_gadget_Projection_z_Density.png
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20130907/997d54ad/attachment-0012.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sim2_gadget_Projection_z_dm_density.png
Type: image/png
Size: 625775 bytes
Desc: sim2_gadget_Projection_z_dm_density.png
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20130907/997d54ad/attachment-0013.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: flash.par
Type: application/octet-stream
Size: 4406 bytes
Desc: flash.par
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20130907/997d54ad/attachment.obj>