Next: 42.5 Performance results Up: 42. Multithreaded FLASH Previous: 42.3 Running multithreaded FLASH Contents Index
The hybrid (MPI+OpenMP) tests using only the multithreaded Hydro and Eos units have the nice property that they can give identical results to the corresponding MPI only test. I emphasise can because it is possible for an optimized build of the MPI+OpenMP application to give a different answer to the MPI only application. This is because the OpenMP directives can impede the compiler from optimizing the floating point calculations. Therefore, to obtain an exact match between an unthreaded and threaded test you should add a compiler option which prevents the compiler from making any aggressive floating point transformations. For the Intel compiler we needed to add -fp-model precise to both the MPI-only and MPI+OpenMP compiler flags to get the same answer. It should be possible to get identical answers for tests like Sod 34.1.1 and Sedov 34.1.4.
This approach does not hold if you use the multithreaded Multipole or energy deposition unit because the floating point accumulation order is different to the order in the unthreaded versions. In the multithreading of the Multipole solver we give each thread its own private moments array and then accumululate the private moment arrays into a single moments array. Similarly, in the energy deposition unit we give each thread a private energy deposition array which is then accumulated into DEPO_VAR mesh variable. It is reasonable to expect a small discrepancy, e.g. a magnitude error of 1E-14 in DEPO_VAR, in a single time step, but be aware that over a large number of time steps the accumulation of small differences can become much larger.