9.1 IO Implementations

FLASH4 supports multiple IO implementations: direct, serial and parallel implementations as well as support for different parallel libraries. In addition, FLASH4 also supports multiple (Chp:Grid Unit) Grid implementations. As a consequence, there are many permutations of the IO API implementation, and the selected implementation must match not only the correct IO library, but also the correct grid. Although there are many IO options, the setup script in FLASH4 is quite `smart' and will not let the user setup a problem with incompatible IO and Grid unit implementations. Table 9.1 summarizes the different implementation of the FLASH IO unit in the current release.

Table 9.1: IO implementations available in FLASH. All implementations begin at the /source directory.
Implementation Path Description

IO/IOMain/HDF5/parallel/PM Hierarchical Data Format (HDF) 5 output. A single HDF5 file is created, with each processor writing its data to the same file simultaneously. This relies on the underlying MPIIO layer in HDF5. This particular implementation only works with the PARAMESH grid package.

IO/IOMain/hdf5/parallel/UG Hierarchical Data Format (HDF) 5 output. A single HDF5 file is created, with each processor writing its data to the same file simultaneously. This relies on the underlying MPIIO layer in HDF5. This particular implementation only works with the Uniform Grid.

IO/IOMain/hdf5/parallel/NoFbs Hierarchical Data Format (HDF) 5 output. A single HDF5 file is created, with each processor writing its data to the same file simultaneously. All data is written out as one block. This relies on the underlying MPIIO layer in HDF5. This particular implementation only works in non-fixedblocksize mode.

IO/IOMain/hdf5/serial/PM Hierarchical Data Format (HDF) 5 output. Each processor passes its data to processor 0 through explicit MPI sends and receives. Processor 0 does all of the writing. The resulting file format is identical to the parallel version; the only difference is how the data is moved during the writing. This particular implementation only works with the PARAMESH grid package.

IO/IOMain/hdf5/serial/UG Hierarchical Data Format (HDF) 5 output. Each processor passes its data to processor 0 through explicit MPI sends and receives. Processor 0 does all of the writing. The resulting file format is identical to the parallel version; the only difference is how the data is moved during the writing. This particular implementation only works with the Uniform Grid.

IO/IOMain/pnetcdf/PM ParallelNetCDF output. A single file is created with each processor writing its data to the same file simultaneously. This relies on the underlying MPI-IO layer in PNetCDF. This particular implementation only works with the PARAMESH grid package.

IO/IOMain/pnetcdf/UG ParallelNetCDF output. A single file is created with each processor writing its data to the same file simultaneously. This relies on the underlying MPI-IO layer in PNetCDF. This particular implementation only works with the Uniform Grid.

IO/IOMain/direct/UG Serial FORTRAN IO. Each processor writes its own data to a separate file. Warning! This choice can lead to many many files! Use only if neither HDF5 or Parallel-NetCDF is available. The FLASH tools are not compatible with the direct IO unit.

IO/IOMain/direct/PM Serial FORTRAN IO. Each processor writes its own data to a separate file. Warning! This choice can lead to many many files! Use only if neither HDF5 or Parallel-NetCDF is available. The FLASH tools are not compatible with the direct IO unit.

IO/IOMain/chombo Hierarchical Data Format (HDF) 5 output. The subroutines provide an interface to the Chombo I/O routines that are part of the standard Chombo release. This implementation can only be used in applications built with Chombo Grid. The Chombo file layout is incompatible with tools, such as fidlr and sfocu, that depend on FLASH file layout.

**Table 9.1:** IO implementations available in FLASH. All implementations begin at the /source directory.
Implementation Path	Description

`IO/IOMain/HDF5/parallel/PM`	Hierarchical Data Format (HDF) 5 output. A single HDF5 file is created, with each processor writing its data to the same file simultaneously. This relies on the underlying MPIIO layer in HDF5. This particular implementation only works with the `PARAMESH` grid package.
`IO/IOMain/hdf5/parallel/UG`	Hierarchical Data Format (HDF) 5 output. A single HDF5 file is created, with each processor writing its data to the same file simultaneously. This relies on the underlying MPIIO layer in HDF5. This particular implementation only works with the Uniform Grid.
`IO/IOMain/hdf5/parallel/NoFbs`	Hierarchical Data Format (HDF) 5 output. A single HDF5 file is created, with each processor writing its data to the same file simultaneously. All data is written out as one block. This relies on the underlying MPIIO layer in HDF5. This particular implementation only works in non-fixedblocksize mode.
`IO/IOMain/hdf5/serial/PM`	Hierarchical Data Format (HDF) 5 output. Each processor passes its data to processor 0 through explicit MPI sends and receives. Processor 0 does all of the writing. The resulting file format is identical to the parallel version; the only difference is how the data is moved during the writing. This particular implementation only works with the `PARAMESH` grid package.
`IO/IOMain/hdf5/serial/UG`	Hierarchical Data Format (HDF) 5 output. Each processor passes its data to processor 0 through explicit MPI sends and receives. Processor 0 does all of the writing. The resulting file format is identical to the parallel version; the only difference is how the data is moved during the writing. This particular implementation only works with the Uniform Grid.
`IO/IOMain/pnetcdf/PM`	ParallelNetCDF output. A single file is created with each processor writing its data to the same file simultaneously. This relies on the underlying MPI-IO layer in PNetCDF. This particular implementation only works with the `PARAMESH` grid package.
`IO/IOMain/pnetcdf/UG`	ParallelNetCDF output. A single file is created with each processor writing its data to the same file simultaneously. This relies on the underlying MPI-IO layer in PNetCDF. This particular implementation only works with the Uniform Grid.
`IO/IOMain/direct/UG`	Serial FORTRAN IO. Each processor writes its own data to a separate file. Warning! This choice can lead to many many files! Use only if neither HDF5 or Parallel-NetCDF is available. The FLASH tools are not compatible with the direct IO unit.
`IO/IOMain/direct/PM`	Serial FORTRAN IO. Each processor writes its own data to a separate file. Warning! This choice can lead to many many files! Use only if neither HDF5 or Parallel-NetCDF is available. The FLASH tools are not compatible with the direct IO unit.
`IO/IOMain/chombo`	Hierarchical Data Format (HDF) 5 output. The subroutines provide an interface to the `Chombo` I/O routines that are part of the standard `Chombo` release. This implementation can only be used in applications built with `Chombo` `Grid`. The `Chombo` file layout is incompatible with tools, such as `fidlr` and `sfocu`, that depend on FLASH file layout.

FLASH4 also comes with some predefined setup shortcuts which make choosing the correct IO significantly easier; see Chp:The FLASH configuration script for more details about shortcuts. In FLASH4 HDF5 serial IO is included by default. Since PARAMESH 4.0 is the default grid, the included IO implementations will be compatible with PARAMESH 4.0. For clarity, a number or examples are shown below.

An example of a basic setup with HDF5 serial IO and the PARAMESH grid, (both defaults) is:

./setup Sod -2d -auto

To include a parallel implementation of HDF5 for a PARAMESH grid the setup syntax is:

./setup Sod -2d -auto -unit=IO/IOMain/hdf5/parallel/PM

using the already defined shortcuts the setup line can be shortened to

./setup Sod -2d -auto +parallelio

To set up a problem with the Uniform Grid and HDF5 serial IO, the setup line is:

./setup Sod -2d -auto -unit=Grid/GridMain/UG -unit=IO/IOMain/hdf5/serial/UG

using the already defined shortcuts the setup line can be shortened to

./setup Sod -2d -auto +ug

To set up a problem with the Uniform Grid and HDF5 parallel IO, the complete setup line is:

./setup Sod -2d -auto -unit=Grid/GridMain/UG -unit=IO/IOMain/hdf5/parallel/UG

using the already defined shortcuts the setup line can be shortened to

./setup Sod -2d -auto +ug  +parallelio

If you do not want to use IO, you need to explicitly specify on the setup line that it should not be included, as in this example:

./setup Sod -2d -auto +noio

To setup a problem using the Parallel-NetCDF library the user should include either

-unit=IO/IOMain/pnetcdf/PM or -unit=IO/IOMain/pnetcdf/UG

to the setup line. The predefined shortcut for including the Parallel-NetCDF library is

+pnetcdf

Note that Parallel-NetCDF IO unit does not have a serial implementation.

If you are using non-fixedblocksize the shortcut

+nofbs

will bring in both Uniform Grid,set the mode to nonfixed blocksize, and choose the appropriate IO.

Note:: Presently, nonfixed blocksize is only supported by HDF5 parallel IO.

In keeping with the FLASH code architecture, the F90 module IO_data stores all the data with IO unit scope. The routine IO_init is called once by Driver_initFlash and initializes IO data and stores any runtime parameters. See Chp:Runtime Parameters Unit.