[FLASH-USERS] flash-users Digest, Vol 71, Issue 3
Chalence Safranek-Shrader
ctss at astro.as.utexas.edu
Mon Sep 9 15:51:43 EDT 2013
Hi Josef,
This is indeed very interesting --- thank you for finding this potential
error. Along the same lines as John ZuHone's comment, It would be useful to
redo the simulations without hydro (dark matter only) to conclusively
exclude hydro as the culprit. Also, have you compared the halo mass
functions from all three codes with analytic predictions?
What kind of refinement schemes are you using for the FLASH and Enzo runs?
It seems like you had refinement turned on in some of the FLASH
projections. Turning all refinement/derefinement off would provide a very
clean comparison between the FLASH and Enzo runs.
Chalence
On Mon, Sep 9, 2013 at 2:28 AM, <flash-users-request at flash.uchicago.edu>wrote:
> Send flash-users mailing list submissions to
> flash-users at flash.uchicago.edu
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://flash.uchicago.edu/mailman/listinfo/flash-users
> or, via email, send a message with subject or body 'help' to
> flash-users-request at flash.uchicago.edu
>
> You can reach the person managing the list at
> flash-users-owner at flash.uchicago.edu
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of flash-users digest..."
>
>
> Today's Topics:
>
> 1. Re: Strange discrepancy in Dark Matter structure of
> cosmological simulation (Mateusz Ruszkowski)
> 2. Re: Strange discrepancy in Dark Matter structure of
> cosmological simulation (St?ckl)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sun, 8 Sep 2013 21:31:19 -0400
> From: Mateusz Ruszkowski <mateuszr at umich.edu>
> Subject: Re: [FLASH-USERS] Strange discrepancy in Dark Matter
> structure of cosmological simulation
> To: John ZuHone <jzuhone at milkyway.gsfc.nasa.gov>
> Cc: "flash-users at flash.uchicago.edu" <flash-users at flash.uchicago.edu>,
> "St?ckl, Josef" <Josef.Stoeckl at uibk.ac.at>
> Message-ID: <FEAD890D-5AB8-4D6C-BE90-34CC4BB52548 at umich.edu>
> Content-Type: text/plain; charset="iso-8859-1"
>
>
>
>
> Hi Josef,
>
> > To see, if this is discrepancy is specific to FLASH 4.0.1, we also tried
> FLASH 3.1, but there is no change to be seen there.
>
> Just to be 100% sure, could you please clarify this? Do you mean no
> change/difference between FLASH3.1 and Enzo/Gadget runs, or no difference
> between FLASH4 and FLASH3.1 (in which case FLASH3.1 gave you the result
> that differed from your Gadget/Enzo runs).
>
> Did your Enzo test include gas? I agree with John and I would be curious
> to see the results of the Zeldovich pancake tests.
>
> Thanks,
> Mateusz
>
>
> On Sep 8, 2013, at 9:48 AM, John ZuHone wrote:
>
> > Josef,
> >
> > First, thank you for the very thorough investigation you've done of this
> problem.
> >
> > I did have a few thoughts on maybe how to try to weed this one out:
> >
> > 1) I wonder a little bit about this:
> >
> >> Since this simulation basically only includes cosmology, self-gravity,
> active particles and hydro, we can only make out three possible points,
> which could influence the DM evolution: the cosmology, the gravity or the
> LeapfrogCosmo unit.
> >
> >
> > Why not hydro? Certainly the baryonic component is very small by mass,
> but in some collapsed regions the effect would not be completely
> insignificant. I'm guessing you're right, but I would just throw that one
> out there for consideration.
> >
> > 2) Which Driver unit are you using? Split or Unsplit? Have you tried
> this with the other one?
> >
> > 3) Could you try setting up the Zeldovich pancake problem (without
> hydro) in all three codes? This should provide a clean, unambiguous point
> of comparison.
> >
> > 4) Thanks for sending the flash.par, but could you also send along the
> setup_units and setup_call files? I'd like to see what units are being
> employed.
> >
> > I think that somewhere between these we could get to the bottom of this.
> >
> > Best,
> >
> > John
> >
> > On Sep 7, 2013, at 10:20 AM, St?ckl, Josef <Josef.Stoeckl at uibk.ac.at>
> wrote:
> >
> >> Dear FLASH users and devs,
> >>
> >> We have been conducting some comparisons of cosmological simulations
> conducted with FLASH 4.0.1, Gadget-2 and Enzo. We set up a cosmological
> volume of 60 Mpc/h box size with Planck cosmological parameters and evolve
> these initial conditions in all three codes. For the grid based codes
> (FLASH, Enzo) we used 256^3 resolution, and for Gadget-2 we used 256^3 DM
> particles and 256^3 gas particles.
> >>
> >> In doing so we noticed that the dark matter structure FLASH shows a
> significant difference to both Enzo and Gadget-2, while those two codes
> show practically perfect agreement. I have attached some projection plots
> at redshift z=0 to show you the differences. As you can see, it seems as if
> the DM structure is less evolved in FLASH. To see, if this is discrepancy
> is specific to FLASH 4.0.1, we also tried FLASH 3.1, but there is no change
> to be seen there.
> >>
> >> Since this simulation basically only includes cosmology, self-gravity,
> active particles and hydro, we can only make out three possible points,
> which could influence the DM evolution: the cosmology, the gravity or the
> LeapfrogCosmo unit.
> >>
> >> We tried to check the cosmology unit by comparing the dt/dz, z, t and
> tend values and the timesteps that the simulation takes, but everything is
> in very good agreement with our calculations and also Enzo. So at least in
> terms of general cosmology there doesn't seem to be a problem there.
> >>
> >> For checking the gravity unit, we switched from Multigrid to bhtree,
> but this did change absolutely nothing - the agreement between the two
> gravity units is very good. So we are inclined to exclude the gravity unit
> as the culprit.
> >>
> >> This leaves us basically with the LeapFrogCosmo unit as the only point
> left, where things could go wrong.
> >>
> >> I have checked everything that I can think of in terms of reading the
> ICs wrongly into FLASH, but all my checks turn out well (overall mass,
> min/max/avg density, particle masses, crit dens,...). To further exclude me
> as a source of error, a colleague also started with a FLASH 4.0.1 vanilla
> version from scratch and implemented the IC reading himself, but this also
> leads to the same results. The ICs have been filled in by assuming both
> distances and masses to be comoving cgs, but with the configured cosmology
> (aka not "over h").
> >>
> >> Since the structures seemed less evolved to us, we decided to try
> running the simulation into the future and set redshift z = -0.5 as the
> final z. The result is given as the sim1-future* attachment, and to our
> surprise, at this point the FLASH structures match the other two codes
> structures (with those at z=0).
> >>
> >> We would be thankful for any insights into this, since we are at our
> wits end.
> >>
> >> Cheers,
> >> Josef
> >>
> >> The attachments can also be found at
> http://astro-staff.uibk.ac.at/~josef/work/comparison/flash-users/together with the initial condition gadget-2 snapshot file ics_* and the
> resulting FLASH checkpoint 0000 file for simulation 1.
> >>
> >> Attachments:
> >> sim1_* ... the simulation 1 comparison projection plots
> >> sim2_* ... the simulation 2 comparison projection plots
> >> sim1-future_* ... simulation 1 evolved until z = -0.5
> >> flash.par ... simulation 1 flash.par with comments removed
> >>
> >>
> >> ----------------------------------------------
> >> Dr. Josef St?ckl
> >> Institute of Astro- and Particle Physics
> >> Technikerstrasse 25/8
> >> 6020 Innsbruck
> >> Austria
> >>
> >> Phone: +43 512 507 52020
> >> E-Mail: josef.stoeckl at uibk.ac.at
> >>
> <sim1_enzo_Projection_z_Density.png><sim1_enzo_Projection_z_dm_density.png><sim1_flash4_Projection_z_Density.png><sim1_flash4_Projection_z_pden.png><sim1_gadget_Projection_z_Density.png><sim1_gadget_Projection_z_dm_density.png><sim1-future_flash4_Projection_z_Density.png><sim1-future_flash4_Projection_z_pden.png><sim2_enzo_Projection_z_Density.png><sim2_enzo_Projection_z_dm_density.png><sim2_flash4_Projection_z_Density.png><sim2_flash4_Projection_z_pden.png><sim2_gadget_Projection_z_Density.png><sim2_gadget_Projection_z_dm_density.png><flash.par>
> >
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://flash.uchicago.edu/pipermail/flash-users/attachments/20130908/05ceea60/attachment-0001.html
>
> ------------------------------
>
> Message: 2
> Date: Mon, 9 Sep 2013 07:28:08 +0000
> From: St?ckl, Josef <Josef.Stoeckl at uibk.ac.at>
> Subject: Re: [FLASH-USERS] Strange discrepancy in Dark Matter
> structure of cosmological simulation
> To: Mateusz Ruszkowski <mateuszr at umich.edu>, John ZuHone
> <jzuhone at milkyway.gsfc.nasa.gov>
> Cc: "flash-users at flash.uchicago.edu" <flash-users at flash.uchicago.edu>
> Message-ID:
> <95922201DF825541A3CE40DA24AEE5CA017A525F at XMBX3.uibk.ac.at>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi Mateusz,
>
> Yes, sorry for that ambiguous phrasing.
>
> I meant that the FLASH 3.1 result is the same as the FLASH 4.0.1 result,
> i.e. both being different from the Enzo/Gadget runs.
>
> Yes, Enzo also did include Gas. I can provide you also with
> Temperature/... plots, if you are interested.
>
> For the Pancake test: I ran it in 3D with 256^3 resolution (in particles
> and cells) yesterday, and have uploaded the resulting projections at
> http://astro-staff.uibk.ac.at/~josef/work/comparison/pancake/flash4/ ,
> but I'm not sure, which type of plot would be most useful to compare
> (slices, projections,...?). I will try to convert the initial checkpoint it
> into Gadget-2 format, to get the same ICs for the runs with Enzo & Gadget.
>
> Thanks and cheers,
> Josef
>
>
> ________________________________
> Von: Mateusz Ruszkowski [mateuszr at umich.edu]
> Gesendet: Montag, 09. September 2013 03:31
> An: John ZuHone
> Cc: St?ckl, Josef; flash-users at flash.uchicago.edu
> Betreff: Re: [FLASH-USERS] Strange discrepancy in Dark Matter structure of
> cosmological simulation
>
>
>
>
> Hi Josef,
>
> To see, if this is discrepancy is specific to FLASH 4.0.1, we also tried
> FLASH 3.1, but there is no change to be seen there.
>
> Just to be 100% sure, could you please clarify this? Do you mean no
> change/difference between FLASH3.1 and Enzo/Gadget runs, or no difference
> between FLASH4 and FLASH3.1 (in which case FLASH3.1 gave you the result
> that differed from your Gadget/Enzo runs).
>
> Did your Enzo test include gas? I agree with John and I would be curious
> to see the results of the Zeldovich pancake tests.
>
> Thanks,
> Mateusz
>
>
> On Sep 8, 2013, at 9:48 AM, John ZuHone wrote:
>
> Josef,
>
> First, thank you for the very thorough investigation you've done of this
> problem.
>
> I did have a few thoughts on maybe how to try to weed this one out:
>
> 1) I wonder a little bit about this:
>
> Since this simulation basically only includes cosmology, self-gravity,
> active particles and hydro, we can only make out three possible points,
> which could influence the DM evolution: the cosmology, the gravity or the
> LeapfrogCosmo unit.
>
> Why not hydro? Certainly the baryonic component is very small by mass, but
> in some collapsed regions the effect would not be completely insignificant.
> I'm guessing you're right, but I would just throw that one out there for
> consideration.
>
> 2) Which Driver unit are you using? Split or Unsplit? Have you tried this
> with the other one?
>
> 3) Could you try setting up the Zeldovich pancake problem (without hydro)
> in all three codes? This should provide a clean, unambiguous point of
> comparison.
>
> 4) Thanks for sending the flash.par, but could you also send along the
> setup_units and setup_call files? I'd like to see what units are being
> employed.
>
> I think that somewhere between these we could get to the bottom of this.
>
> Best,
>
> John
>
> On Sep 7, 2013, at 10:20 AM, St?ckl, Josef <Josef.Stoeckl at uibk.ac.at
> <mailto:Josef.Stoeckl at uibk.ac.at>> wrote:
>
> Dear FLASH users and devs,
>
> We have been conducting some comparisons of cosmological simulations
> conducted with FLASH 4.0.1, Gadget-2 and Enzo. We set up a cosmological
> volume of 60 Mpc/h box size with Planck cosmological parameters and evolve
> these initial conditions in all three codes. For the grid based codes
> (FLASH, Enzo) we used 256^3 resolution, and for Gadget-2 we used 256^3 DM
> particles and 256^3 gas particles.
>
> In doing so we noticed that the dark matter structure FLASH shows a
> significant difference to both Enzo and Gadget-2, while those two codes
> show practically perfect agreement. I have attached some projection plots
> at redshift z=0 to show you the differences. As you can see, it seems as if
> the DM structure is less evolved in FLASH. To see, if this is discrepancy
> is specific to FLASH 4.0.1, we also tried FLASH 3.1, but there is no change
> to be seen there.
>
> Since this simulation basically only includes cosmology, self-gravity,
> active particles and hydro, we can only make out three possible points,
> which could influence the DM evolution: the cosmology, the gravity or the
> LeapfrogCosmo unit.
>
> We tried to check the cosmology unit by comparing the dt/dz, z, t and tend
> values and the timesteps that the simulation takes, but everything is in
> very good agreement with our calculations and also Enzo. So at least in
> terms of general cosmology there doesn't seem to be a problem there.
>
> For checking the gravity unit, we switched from Multigrid to bhtree, but
> this did change absolutely nothing - the agreement between the two gravity
> units is very good. So we are inclined to exclude the gravity unit as the
> culprit.
>
> This leaves us basically with the LeapFrogCosmo unit as the only point
> left, where things could go wrong.
>
> I have checked everything that I can think of in terms of reading the ICs
> wrongly into FLASH, but all my checks turn out well (overall mass,
> min/max/avg density, particle masses, crit dens,...). To further exclude me
> as a source of error, a colleague also started with a FLASH 4.0.1 vanilla
> version from scratch and implemented the IC reading himself, but this also
> leads to the same results. The ICs have been filled in by assuming both
> distances and masses to be comoving cgs, but with the configured cosmology
> (aka not "over h").
>
> Since the structures seemed less evolved to us, we decided to try running
> the simulation into the future and set redshift z = -0.5 as the final z.
> The result is given as the sim1-future* attachment, and to our surprise, at
> this point the FLASH structures match the other two codes structures (with
> those at z=0).
>
> We would be thankful for any insights into this, since we are at our wits
> end.
>
> Cheers,
> Josef
>
> The attachments can also be found at
> http://astro-staff.uibk.ac.at/~josef/work/comparison/flash-users/together with the initial condition gadget-2 snapshot file ics_* and the
> resulting FLASH checkpoint 0000 file for simulation 1.
>
> Attachments:
> sim1_* ... the simulation 1 comparison projection plots
> sim2_* ... the simulation 2 comparison projection plots
> sim1-future_* ... simulation 1 evolved until z = -0.5
> flash.par ... simulation 1 flash.par with comments removed
>
>
> ----------------------------------------------
> Dr. Josef St?ckl
> Institute of Astro- and Particle Physics
> Technikerstrasse 25/8
> 6020 Innsbruck
> Austria
>
> Phone: +43 512 507 52020
> E-Mail: josef.stoeckl at uibk.ac.at<mailto:josef.stoeckl at uibk.ac.at>
>
> <sim1_enzo_Projection_z_Density.png><sim1_enzo_Projection_z_dm_density.png><sim1_flash4_Projection_z_Density.png><sim1_flash4_Projection_z_pden.png><sim1_gadget_Projection_z_Density.png><sim1_gadget_Projection_z_dm_density.png><sim1-future_flash4_Projection_z_Density.png><sim1-future_flash4_Projection_z_pden.png><sim2_enzo_Projection_z_Density.png><sim2_enzo_Projection_z_dm_density.png><sim2_flash4_Projection_z_Density.png><sim2_flash4_Projection_z_pden.png><sim2_gadget_Projection_z_Density.png><sim2_gadget_Projection_z_dm_density.png><flash.par>
>
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://flash.uchicago.edu/pipermail/flash-users/attachments/20130909/1b6c01ca/attachment.html
>
> ------------------------------
>
> _______________________________________________
> flash-users mailing list
> flash-users at flash.uchicago.edu
> http://flash.uchicago.edu/mailman/listinfo/flash-users
>
>
> End of flash-users Digest, Vol 71, Issue 3
> ******************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20130909/331b38dd/attachment.htm>
More information about the flash-users
mailing list