[FLASH-USERS] Improving the load balancing in FLASH

[Brian O'Shea]

In all fairness, the statement that Anshu made depends a lot on the  
type of problem you are using.  For example, the Enzo AMR code is  
mainly used for astrophysical applications that are gravity-dominated,  
so most of the action takes place in a very small fraction of the  
simulation volume, and can use many levels of refinement.  The only  
way that this sort of problem is tractable is by using adaptive time- 
stepping.  One of the main complications of such an adaptive time- 
stepping scheme is keeping all of the mesh levels synched up.  It also  
makes load-balancing a significant challenge - one is forced to  
distribute grids on a level-by-level basis, and if only a small  
fraction of the volume is refined (as in our cosmology simulations...)  
you destroy grid locality, which increases communication and decreases  
scalability.

Brian

> This was considered several years ago. Then Mike Zingale and Jonathan
> Dursi proved that the computational savings from coarse blocks having
> a coarse time step are too insignificant to be worth pursuing. This is
> because the computational time in the finer blocks completely
> dominates that in coarser blocks for the same area of computational
> domain. I believe that study appeared in some proceedings, if you are
> interested I can dig up the reference.
>
> Anshu
>
> On Tue, Sep 8, 2009 at 4:17 AM, Ross Parkin<phy1erp at leeds.ac.uk>  
> wrote:
>> Hello,
>>
>> I've been thinking about the load balancing method used in FLASH. The
>> timestep used by the hydro units is the same on all refinement  
>> levels and so
>> blocks are essentially distributed by paramesh so that all  
>> processors have
>> the roughly the smae number of blocks. Has anyone tried modifying  
>> FLASH so
>> that each refinement level has its own timestep? To do this  
>> properly you
>> would need to modify how PARAMESH distributes blocks amongst  
>> processors,
>> taking account of the relative work done by each block on each  
>> refinement
>> level, i.e. a block on a finer refinement level may need to do  
>> twice as many
>> hydro loops before its simulation time equals that of a coarser  
>> block.
>>
>> Any ideas guys?
>>
>> Cheers,
>>
>> Ross Parkin
>>
>