[FLASH-USERS] bug in Driver_evolveFlash.F90 (any implementation); exit on elapsed wall clock time not correctly implemented

[Christoph Federrath]

Hi FLASH developers and users,

I want to report a bug in Driver_evolveFlash.F90 (any implementation). In case the code needs to be stopped when a maximum wall clock time is reached, I had code hang-ups in some cases (in particular when running with thousands of MPI tasks) and some cases it went through to the end. Looking into Driver_evolveFlash.F90, it seems that some MPI tasks can actually get past the exit on max wallclock time (if for them the max wall clock had not been reached yet), while others may already exit the evolution loop, producing inconsistent behavior and hang-ups when they try to communicate globally the next time. I fixed this by porting FLASH2.5 code from flash.F90. This needs to be replaced in Driver_evolveFlash.F90:


    call Driver_getElapsedWCTime(dr_elapsedWCTime)
    if (dr_elapsedWCTime >  dr_wallClockTimeLimit) then
       if(dr_globalMe == MASTER_PE) then
          print *, "exiting: reached max wall clock time"
       end if
       exit
    end if


with


    ! Christoph Federrath 2013 replaced below, because of hang-ups, if one process went beyond this point
    ! (wall clock time not yet reached), while another exits the loop here. Ported from FLASH2.5.
    call Driver_getElapsedWCTime(dr_elapsedWCTime)
    endtime = .false.
    if (dr_globalMe == MASTER_PE) then
        if (dr_elapsedWCTime >  dr_wallClockTimeLimit) endtime = .true.
        print *, "exiting: reached max wall clock time"
    endif
    call MPI_Bcast(endtime, 1, MPI_LOGICAL, MASTER_PE, MPI_COMM_WORLD, ierr)
    if (endtime) exit


and appropriate includes and definitions must be included for this to work:


 include "Flash_mpi.h"
 logical :: endtime
 integer :: ierr


I hope this helps. Please confirm that this is a bug in FLASH4.0.1 (latest public release version).

Thanks and kind regards,

Christoph Federrath


________________________________
Dr. Christoph Federrath
School of Mathematical Sciences,
Monash University,
Clayton, Vic 3800, Australia
+61 3 9905 9760
http://www.ita.uni-heidelberg.de/~chfeder/index.shtml?lang=en