[FLASH-USERS] Flash memory usage.

[Georgi Pavlovski]

maxblocks is the maximum amount of blocks allowed per cpu.

flash (paramesh module) creates a grid at the start of the simulation. 
Individual blocks are distributed among available cpus.  The goal is to 
make the load as equal as possible.  When refinement/derefinement 
happens the blocks are getting reshuffled among cpus.  The number of 
active (valid) blocks can grow/decrease during the run.

The storage (memory requirements) are constant for a run.  They are 
defined by the maximum allowed number of blocks per cpu during the 
compilation. At any one time every cpu has unk array of the quoted size. 
  Some of that space will never be used if local number of blocks per 
cpu (lnblocks changes after each refinement/derefinement event) is less 
than the maxblocks.

I hope that helps.

Brock Palen wrote:
> That does help allot,  but being a admin of a system ware a user is 
> running flash, I only compiled it and ran the test cases I do not 
> understand what its doing.
> 
> The max blocks, are blocks a physical property of the model?  maxblocks 
> is the maximum number of blocks on any one cpus any time during the 
> run,  but if block represent physical space in the model,  and block are 
> distributed among cpus, using more cpus (more nodes to get more ram)  
> and having fewer blocks per cpu would allow us to slower maxblocks thus 
> lowering the memory needed by each cpu right?
> 
> There is a 95% chance I am completely wrong.  The user is running a 3d 
> problem.
> 
> The formula given to me by Georgi Pavlovski was:
> 
> (nvar)x(2*nguard+nxb)x(2*nguard+nyb)x(2*nguard+nzb)x(maxblocks)
> 
> So the memory use is proportional to maxblocks.
> 
> Brock Palen
> Center for Advanced Computing
> brockp at umich.edu
> (734)936-1985
> 
> 
> On Nov 30, 2007, at 9:33 AM, Anshu Dubey wrote:
> 
>> The amount of static memory allocated is directly related to the 
>> number of
>> maxblocks you specify at compile time. Then there is local storage in
>> modules that goes on the stack and is usually dependent on the size
>> of the individual block. Hdf5_serial IO has relatively large dynamic
>> allocation, roughly the size of the local mesh on the processor, and
>> paramesh does its own dynamic allocation during communication. This
>> allocation depends upon the amount of data that needs to move between
>> processors. So the only way allocation is really influenced by spreading
>> work out on more nodes is by cutting down the maxblocks proportionately.
>> Maxblocks itself is a local quantity, it is the maximum number
>> of blocks you anticipate having on any one processor at any time during
>> your simulation.
>>
>> I hope that answers your question, if you need further clarification 
>> please
>> let me know.
>>
>> Anshu
>>
>>
>>> How does flash do memory allocation?  If I am running into memory
>>> problems but my only option for more memory is to run on more nodes
>>> does flash subdivide the amount of memory in use evenly?
>>> For example:
>>>
>>> a model when running on 4 cpus, each process takes 10GB of real memory.
>>>
>>> If I run that same model on 8 how close will each process get to 5 gb
>>> of use?
>>>
>>> Brock Palen
>>> Center for Advanced Computing
>>> brockp at umich.edu
>>> (734)936-1985
>>>
>>>
>>
>>
>>
> 
> 
> 
> 

-- 
===================================================================
   Dr Georgi B. Pavlovski        School of Physics and Astronomy
   Tel: +44-(0)-23-8059-2089     University of Southampton
   Fax: +44-(0)-23-8059-3910     Southampton SO17 1BJ, U.K.
   Email: gbp at phys.soton.ac.uk   Web: www.astro.soton.ac.uk/~gbp
===================================================================