[FLASH-BUGS] hangups on IBM regatta

[Mike Zingale]

I've never tried 32-bit ABI on an IBM.

The -O3 usually does not work -- runtime_parameters.F90 is the problem
(bug #66).  The amr_redist_blks hang may be a symptom of this.  As best as
I can tell, we do not have a bug in our code.  A pointer just magically
become dissociated at some point.  Things work fine for me if this routine
is compiled with -O2.

Mike

On Wed, 18 Sep 2002, Andrew Siegel wrote:

> ---------- Forwarded message ----------
> 
> 
> Date: Thu, 12 Sep 2002 10:30:02 -0500 (CDT)
> From: Andrew Siegel <siegela at sunspot.uchicago.edu>
> To: Sebastian Heinz <heinzs at mpa-garching.mpg.de>
> Cc: flash-bugs at flash.uchicago.edu
> Subject: Re: [FLASH-BUGS] hangups on IBM regatta
> 
> 
> Hi Sebastian: Have you tried running with -O2? 
> 
> On Thu, 12 Sep 2002, Sebastian Heinz wrote:
> 
> > Dear FLASH team,
> > 
> > 
> > I am having problems getting FLASH to run on the new IBM regatta system at
> > the IPP in Garching, Germany. A description can be found at
> > 
> >   http://www.rzg.mpg.de/computing/IBM_P/
> > 
> > Here's a brief description of the problem:
> > 
> > Currently I am running a 2D hydro simulation in cylindrical coordinates at
> > relatively high resolution (but the same problem occurs also in cartesian
> > 3D simulations). 
> > 
> > After running for a while, the system simply hangs up without producing
> > any errors. This only happens when I run the process on more than two
> > processors, while it runs fine on one or two processors (i.e., this is not
> > a maxblocks problem: I compiled using maxblocks=12000 and at the
> > corresponding timestep the simulation has 4478 blocks). After some
> > back-tracking, I found that the system hangs in the routine
> > 
> >   amr_redist_blk.F90
> > 
> > at the MPI_ALLREDUCE call in line 102. Some processors make it past this
> > call, while the others don't.
> > 
> > I compiled using the flags
> > 
> >   ABI  = -q32 -qmaxmem=-1
> >   OPT1 = -O3 -qstrict -bmaxdata:0x80000000
> >   OPT2 = -qfloat=nomaf -bloadmap:makelog
> >   OPT3 = -qcache=auto -qtune=auto -qarch=auto -qthreaded -qnosave
> >   OPTS = $(OPT1) $(OPT2) $(OPT3)
> > 
> > Any ideas what this could be? (From what I read on the FLASH website, I
> > don't seem to be the only one with problems on IBM systems).
> > 
> > I tried to compile the code with 64 bits, but I couldn't get that to work
> > - any input on what compiler options to use for 64 bits (and which
> > libraries I need to link) would also be highly welcome.
> > 
> > 
> > Thanks for your help,
> > 
> > Best regards,
> > 
> > Sebastian
> > 
> > 
> > P.S.: I also found a small bug in MarkRefLib.F90, which I will send you in
> > a separate E-mail
> > 
>