In FLASH4 we introduced an alternative I/O implementation for both HDF5 and Parallel-NetCDF which is a slight spin on the standard parallel I/O implementations. In this new implementation we select the data from the mesh data structures directly using HDF5 hyperslabs (HDF5) and MPI derived datatypes (Parallel-NetCDF) and then write the selected data to datasets in the file. This eliminates the need for manually copying data into a FLASH allocated temporary buffer and then writing the data from the temporary buffer to disk.
You can include derived data type I/O in your FLASH application by adding the setup shortcuts +hdf5TypeIO for HDF5 and +pnetTypeIO for Parallel-NetCDF to your setup line. If you are using the HDF5 implementation then you need a parallel installation of HDF5. All of the runtime parameters introduced in this chapter should be compatible with derived data type I/O.
A nice property of derived data type I/O is that it eliminates a lot of the I/O code duplication which has been spreading in the FLASH I/O unit over the last decade. The same code is used for UG, NoFBS and Paramesh FLASH applications and we have also shared code between the HDF5 and Parallel-NetCDF implementations. A technical reason for using the new I/O implementation is that we provide more information to the I/O libraries about the exact data we want to read from / write to disk. This allows us to take advantage of recent enhancements to I/O libraries such as the nonblocking APIs in the Parallel-NetCDF library. We discuss experimentation with this API and other ideas in the paper “A Case Study for Scientific I/O: Improving the FLASH Astrophysics Code” www.mcs.anl.gov/uploads/cels/papers/P1819.pdf
The new I/O code has been tested in our internal FLASH regression tests from before the FLASH4 release and there are no known issues, however, it will probably be in the release following FLASH4 when we will recommend using it as the default implementation. We have made the research ideas from our case study paper usable for all FLASH applications, however, the code still needs a clean up and exhaustive testing with all the FLASH runtime parameters introduced in this chapter.