[FLASH-USERS] Filling Guard Cells in Burn

[James Guillochon]

I upgraded my Burn unit to 3.1 (and SimulationComposition), and things 
are running much faster now (10 times faster)! Considering 
fillGuardCells is still being called, I guess the issue must have been 
unrelated to my initial guess...

-- 
James Guillochon
Department of Astronomy & Astrophysics
University of California, Santa Cruz
jfg at ucolick.org


Klaus Weide wrote:
> On Tue, 28 Oct 2008, James Guillochon wrote:
>> So I'm using the nuclearBurn unit (FLASH 3.0), and the 
>> Grid_fillGuardCells routine at the top of Burn.F90 seems to be taking 
>> an inordinate amount of time to run. My simulation has sporadic 
>> burning and thus does not need the network to run at every time step, 
>> but the fillGuardCells routine is taking most of the processing time 
>> even with T << nuclearTempMin!
> 
> James,
> 
> First of all, I am surprised this takes an "inordinate amount of time".
> I would expect that the Grid_fillGuardCells calls in other places, 
> predominantly in Hydro, use more time, just because they would be
> called more often (at least when NDIM > 1).  I am thinking of a "normal"
> FLASH simulation that uses Hydro, and Burn, and probably some other 
> physics units.
> 
> If you just mean that the Grid_fillGuardCells in Burn dominates the 
> total time spent in Burn - that may well be, but see below.
> 
>> The question is: Do I have to fill the guard cells if the temperature 
>> criteria for burning isn't met? Is there some reason the 
>> fillGuardCells routine is taking so long in this context?
> 
> I am assuming you are referring to this source file:
>    source/physics/sourceTerms/Burn/BurnMain/nuclearBurn/Burn.F90 ,
> let me know if I'm wrong!
> 
> There has been a code change in this file from FLASH 3.0 to 3.1,
> associated with this comment:
> 
> ========================================================================================================= 
> 
> Filename: 
> trunk/source/physics/sourceTerms/Burn/BurnMain/nuclearBurn/Burn.F90
> Revision 8763
> Modified Fri Mar 21 13:28:56 2008 CDT (7 months, 1 week ago) by townsley
> File length: 9383 byte(s)
> Diff to previous 7876
> 
> [...]
> Also limited nuclear Burn calculations to only interior cells,
> no need to do the guard cells.
> ========================================================================================================= 
> 
> 
> Here is the diff:
> 
> ========================================================================================================= 
> 
> *** 
> trunk/source/physics/sourceTerms/Burn/BurnMain/nuclearBurn/Burn.F90    
> 2008/01/07 16:23:52    7876
> --- 
> trunk/source/physics/sourceTerms/Burn/BurnMain/nuclearBurn/Burn.F90    
> 2008/03/21 18:28:56    8763
> ***************
> *** 150,158 ****
> 
> 
>        ! now guaranteed that tmp, rho, etc. exist
> !      do k = blkLimitsGC(LOW,KAXIS), blkLimitsGC(HIGH,KAXIS)
> !         do j = blkLimitsGC(LOW,JAXIS), blkLimitsGC(HIGH,JAXIS)
> !            do i = blkLimitsGC(LOW,IAXIS), blkLimitsGC(HIGH,IAXIS)
>                 okBurnTemp = .FALSE.
>                 okBurnDens = .FALSE.
>                 okBurnShock = .FALSE.
> --- 150,158 ----
> 
> 
>        ! now guaranteed that tmp, rho, etc. exist
> !      do k = blkLimits(LOW,KAXIS), blkLimits(HIGH,KAXIS)
> !         do j = blkLimits(LOW,JAXIS), blkLimits(HIGH,JAXIS)
> !            do i = blkLimits(LOW,IAXIS), blkLimits(HIGH,IAXIS)
>                 okBurnTemp = .FALSE.
>                 okBurnDens = .FALSE.
>                 okBurnShock = .FALSE.
> ========================================================================================================= 
> 
> 
> (There were also changes in that update related to enabling 
> Burn_computeDt. If you use
> Burn_computeDt, it probably should have the same kind of blkLimitsGC -> 
> blkLimits change.)
> 
> I believe that with the FLASH 3.1 version, the only reason for having a 
> Grid_fillGuardCells call
> at the top of Burn.F90 is to provide valid data for Hydro_detectShock.  
> (That's why Grid_fillGuardCells
> is only called 'if (.NOT. bn_useShockBurn)').
> 
> If you look at the code of Hydro_detectShock.F90, you will find that it 
> requires only a limited number of guard cell layers (1 or 2?) for its 
> stencil, and it refers only to
> a small number of UNK variables (VEL{X,Y,Z}_VAR and PRES_VAR).  You 
> could therefore
> try to speed up the Grid_fillGuardCells calls somewhat by limiting the 
> number of
> cell layers and/or variables that are exchanged - see 
> Grid_fillGuardCells.F90 for
> the various OPTIONAL arguments that may apply.  To restrict the set of 
> variables
> that are exchanged, also make sure the runtime parameter
>   enableMaskedGcFill
> is TRUE.
> 
> 
> There is currently no way to limit guard cell exchanges only to some 
> blocks or processors that may need them - it's an all-or-nothing thing 
> in that way.  But if you know of some criteria to determine globally 
> that in a given invocation of Burn, there is no need to call 
> Hydro_detectShock on ANY block in the simulation (for example, because 
> you can determine positively that the Burn variables okBurnTemp .AND. 
> okBurnDens will be FALSE in all cells in all leaf blocks) - then you can 
> of course skip the call in that invocation of Burn.
> 
> 
> 
> Klaus
> 
> !DSPAM:10135,49073eed148013225631196!
>