[FLASH-BUGS] hangups on IBM regatta

[Andrew Siegel]

Hi Sebastian: Have you tried running with -O2? 

On Thu, 12 Sep 2002, Sebastian Heinz wrote:

> Dear FLASH team,
> 
> 
> I am having problems getting FLASH to run on the new IBM regatta system at
> the IPP in Garching, Germany. A description can be found at
> 
>   http://www.rzg.mpg.de/computing/IBM_P/
> 
> Here's a brief description of the problem:
> 
> Currently I am running a 2D hydro simulation in cylindrical coordinates at
> relatively high resolution (but the same problem occurs also in cartesian
> 3D simulations). 
> 
> After running for a while, the system simply hangs up without producing
> any errors. This only happens when I run the process on more than two
> processors, while it runs fine on one or two processors (i.e., this is not
> a maxblocks problem: I compiled using maxblocks=12000 and at the
> corresponding timestep the simulation has 4478 blocks). After some
> back-tracking, I found that the system hangs in the routine
> 
>   amr_redist_blk.F90
> 
> at the MPI_ALLREDUCE call in line 102. Some processors make it past this
> call, while the others don't.
> 
> I compiled using the flags
> 
>   ABI  = -q32 -qmaxmem=-1
>   OPT1 = -O3 -qstrict -bmaxdata:0x80000000
>   OPT2 = -qfloat=nomaf -bloadmap:makelog
>   OPT3 = -qcache=auto -qtune=auto -qarch=auto -qthreaded -qnosave
>   OPTS = $(OPT1) $(OPT2) $(OPT3)
> 
> Any ideas what this could be? (From what I read on the FLASH website, I
> don't seem to be the only one with problems on IBM systems).
> 
> I tried to compile the code with 64 bits, but I couldn't get that to work
> - any input on what compiler options to use for 64 bits (and which
> libraries I need to link) would also be highly welcome.
> 
> 
> Thanks for your help,
> 
> Best regards,
> 
> Sebastian
> 
> 
> P.S.: I also found a small bug in MarkRefLib.F90, which I will send you in
> a separate E-mail
>