[FLASH-USERS] More checkpoint files in parallel version

[Klaus Weide]

On Thu, 10 Mar 2016, Arne Westphal wrote:

> Dear all,
> 
> I'm pretty new to the FLASH Code and just got it running properly. Since
> there are multiple processes running at the same time which want to
> create their part of the checkpoint files, I wanted to ask, if there is
> a (simple) method to configure FLASH to create more checkpoint files. So
> maybe one checkpoint file per process or one per X processes.
> So if you would have 5 checkpoint files with the serial run and run it
> with 10 processes, you get 50 files.
> 
> I'm sorry if I just didn't find it in the documentation yet.

Hello Arne,

Excerpt from the Users Guide:


9.9.1.4 Split File IO

On machines with large numbers of processors, IO may perform better if, 
all processors write to a limited number of separate files rather than one 
single file. This technique can help mitigate IO bottlenecks and 
contention issues on these large machines better than even parallel-mode 
IO can. In addition this technique has the benefit of keeping the number 
of output files much lower than if every processor writes its own file. 
Split file IO can be enabled by setting the outputSplitNum parameter to 
the number of files desired (i.e. if outputSplitNum is set to 4, every 
checkpoint, plotfile and parfile will be broken into 4 files, by processor 
number). This feature is only available with the HDF5 parallel IO mode, 
and is still experimental. Users should use this at their own risk.


"Split file IO" is also mentioned in 9.8 Output Filenames.

Note the caveats; I don't know how well this actually works.

Klaus