<div dir="ltr">Hello,<br><br>I am writing to describe an issue with running Flash4 on a Cray A64FX machine with the Cray compiler, version 10.0.1. We are running Flash 4.6.2. I thought I would describe what I am seeing so as to ask for any <br>advice before digging in further. I am not completely sure if this is a bug, hence my sending it here.  <br><br>First I should note that things work well with the Gnu compilers. We do not see this problem when compiling with the gnu compilers.<br><br>The shortest possible summary is that the code seems to enter an unphysical state and crash, and with the split hydro solver I see a warning<br><br>WARNING after gc filling: min. unk(EINT_VAR)=9.9999999735241242E-11      PE=4   block=6       <br>          type=1<br><br>We compile with -c -g -G 2 -s real64 -s integer32 and the code generates a few warnings, none of which seem relevant. We've tried a few problems, too, and are seeing the same issues. For this report I ran the Sedov problem. <br><br>With the default units, and set up with <br><br>./setup Sedov -auto -site=<a href="http://ookami.stonybrook.edu" target="_blank">ookami.stonybrook.edu</a><br><br>the code runs a while and then suddenly the solution seems to hit some bad state causing the time step to suddenly drop and then the code crashes.<br><br>[acalder@fj-debug1 object]$ !mpi<br>mpiexec -n 8 ./flash4<br> Grid_init: resolution based on runtime params:<br> lrefine      dx        dy<br>   1       0.125      0.125<br>   2       0.063      0.063<br>   3       0.031      0.031<br>   4       0.016      0.016<br>   5       0.008      0.008<br>   6       0.004      0.004<br> MaterialProperties initialized<br> Cosmology initialized<br> Source terms initialized<br> sim_rhoAmbient is 1.<br> iteration, no. not moved =  2*0<br> refined: total leaf blocks =  1<br> refined: total blocks =  1<br> [amr_morton_process]: Initializing surr_blks using standard orrery implementation<br> INFO: Grid_fillGuardCells is ignoring masking.<br> iteration, no. not moved =  2*0<br> refined: total leaf blocks =  4<br> refined: total blocks =  5<br> iteration, no. not moved =  0,  3<br> iteration, no. not moved =  2*1<br> iteration, no. not moved =  2,  0<br> refined: total leaf blocks =  16<br> refined: total blocks =  21<br> iteration, no. not moved =  0,  6<br> iteration, no. not moved =  1,  0<br> refined: total leaf blocks =  28<br> refined: total blocks =  37<br> iteration, no. not moved =  0,  9<br> iteration, no. not moved =  1,  2<br> iteration, no. not moved =  2,  0<br> refined: total leaf blocks =  40<br> refined: total blocks =  53<br> iteration, no. not moved =  0,  16<br> iteration, no. not moved =  1,  2<br> iteration, no. not moved =  2,  0<br> refined: total leaf blocks =  52<br> refined: total blocks =  69<br> Finished with Grid_initDomain, no restart<br> Ready to call Hydro_init<br> Hydro initialized<br> Gravity initialized<br> Initial dt verified<br> *** Wrote checkpoint file to sedov_hdf5_chk_0000 ****<br> *** Wrote plotfile to sedov_hdf5_plt_cnt_0000 ****<br> Initial plotfile written<br> Driver init all done<br>    n      t     dt  (     x,      y,      z) |  dt_hydro <br>    1 1.0000E-10 2.0000E-10  ( 0.498   ,  0.498   ,  0.00   ) |  1.012E-04<br>    2 3.0000E-10 4.0000E-10  ( 0.498   ,  0.494   ,  0.00   ) |  1.012E-04<br>    3 7.0000E-10 8.0000E-10  ( 0.498   ,  0.494   ,  0.00   ) |  1.012E-04<br>    4 1.5000E-09 1.6000E-09  ( 0.498   ,  0.494   ,  0.00   ) |  1.012E-04<br><div><br></div><div>snip</div><div><br></div>   26 4.4575E-04 4.7891E-05  ( 0.502   ,  0.510   ,  0.00   ) |  4.789E-05<br>   27 4.9364E-04 4.5144E-05  ( 0.498   ,  0.510   ,  0.00   ) |  4.514E-05<br>   28 5.3879E-04 4.1959E-05  ( 0.498   ,  0.510   ,  0.00   ) |  4.196E-05<br>   29 5.8075E-04 2.3721E-05  ( 0.506   ,  0.498   ,  0.00   ) |  2.372E-05<br> iteration, no. not moved =  0,  17<br> iteration, no. not moved =  1,  2<br> iteration, no. not moved =  2,  0<br> refined: total leaf blocks =  88<br> refined: total blocks =  117<br>   30 6.0447E-04 1.0000E-10  ( 0.506   ,  0.498   ,  0.00   ) |  4.269E-18<br>   31 6.0447E-04 1.0000E-10  ( 0.498   ,  0.498   ,  0.00   ) |  2.897E-56<br>   32 6.0447E-04 1.0000E-10  ( 0.510   ,  0.494   ,  0.00   ) |  1.206-156<br> dtCheck= 0.<br> DRIVER_ABORT: [Hydro]: Computed dt is not positive! Aborting!<br>--------------------------------------------------------------------------<br>MPI_ABORT was invoked on rank 1 in communicator MPI_COMM_WORLD<br>with errorcode 1.<br><br>NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.<br>You may or may not see output from other processes, depending on<br>exactly when Open MPI kills them.<br>--------------------------------------------------------------------------<br><br><br>In the above case, the glitch happened after a refinement, but it does not always happen after a refinement. Here I changed the max level of refinement. After step 69 the glitch happens.<br><br>[acalder@fj-debug1 object]$ vi flash.par<br>[acalder@fj-debug1 object]$ !mpi<br>mpiexec -n 8 ./flash4<br> Grid_init: resolution based on runtime params:<br> lrefine      dx        dy<br>   1       0.125      0.125<br>   2       0.063      0.063<br>   3       0.031      0.031<br>   4       0.016      0.016<br>   5       0.008      0.008<br>   6       0.004      0.004<br>   7       0.002      0.002<br> MaterialProperties initialized<br> Cosmology initialized<br> Source terms initialized<br> sim_rhoAmbient is 1.<br> iteration, no. not moved =  2*0<br> refined: total leaf blocks =  1<br> refined: total blocks =  1<br> [amr_morton_process]: Initializing surr_blks using standard orrery implementation<br> INFO: Grid_fillGuardCells is ignoring masking.<br> iteration, no. not moved =  2*0<br> refined: total leaf blocks =  4<br> refined: total blocks =  5<br> iteration, no. not moved =  0,  3<br> iteration, no. not moved =  2*1<br> iteration, no. not moved =  2,  0<br> refined: total leaf blocks =  16<br> refined: total blocks =  21<br> iteration, no. not moved =  0,  6<br> iteration, no. not moved =  1,  0<br> refined: total leaf blocks =  28<br> refined: total blocks =  37<br> iteration, no. not moved =  0,  9<br> iteration, no. not moved =  1,  2<br> iteration, no. not moved =  2,  0<br> refined: total leaf blocks =  40<br> refined: total blocks =  53<br> iteration, no. not moved =  0,  16<br> iteration, no. not moved =  1,  2<br> iteration, no. not moved =  2,  0<br> refined: total leaf blocks =  52<br> refined: total blocks =  69<br> iteration, no. not moved =  0,  20<br> iteration, no. not moved =  2*1<br> iteration, no. not moved =  2,  0<br> refined: total leaf blocks =  64<br> refined: total blocks =  85<br> Finished with Grid_initDomain, no restart<br> Ready to call Hydro_init<br> Hydro initialized<br> Gravity initialized<br> Initial dt verified<br> *** Wrote checkpoint file to sedov_hdf5_chk_0000 ****<br> *** Wrote plotfile to sedov_hdf5_plt_cnt_0000 ****<br> Initial plotfile written<br> Driver init all done<br>    n      t     dt  (     x,      y,      z) |  dt_hydro <br>    1 1.0000E-10 2.0000E-10  ( 0.499   ,  0.491   ,  0.00   ) |  5.060E-05<br>    2 3.0000E-10 4.0000E-10  ( 0.499   ,  0.493   ,  0.00   ) |  5.060E-05<br>    3 7.0000E-10 8.0000E-10  ( 0.499   ,  0.493   ,  0.00   ) |  5.060E-05<br>    4 1.5000E-09 1.6000E-09  ( 0.499   ,  0.493   ,  0.00   ) |  5.060E-05<br>    5 3.1000E-09 3.2000E-09  ( 0.499   ,  0.493   ,  0.00   ) |  5.060E-05<br>    6 6.3000E-09 6.4000E-09  ( 0.499   ,  0.493   ,  0.00   ) |  5.059E-05<br>    7 1.2700E-08 1.2800E-08  ( 0.499   ,  0.493   ,  0.00   ) |  5.059E-05<br>    8 2.5500E-08 2.5600E-08  ( 0.499   ,  0.493   ,  0.00   ) |  5.058E-05<br>    9 5.1100E-08 5.1200E-08  ( 0.499   ,  0.495   ,  0.00   ) |  5.057E-05<br>   10 1.0230E-07 1.0240E-07  ( 0.499   ,  0.493   ,  0.00   ) |  5.054E-05<br>   11 2.0470E-07 2.0480E-07  ( 0.499   ,  0.493   ,  0.00   ) |  5.048E-05<br>   12 4.0950E-07 4.0960E-07  ( 0.499   ,  0.493   ,  0.00   ) |  5.036E-05<br>   13 8.1910E-07 8.1920E-07  ( 0.499   ,  0.493   ,  0.00   ) |  5.014E-05<br> iteration, no. not moved =  0,  19<br> iteration, no. not moved =  2*1<br> iteration, no. not moved =  2,  0<br> refined: total leaf blocks =  112<br> refined: total blocks =  149<br>   14 1.6383E-06 1.6384E-06  ( 0.499   ,  0.493   ,  0.00   ) |  4.968E-05<br>   15 3.2767E-06 3.2768E-06  ( 0.499   ,  0.493   ,  0.00   ) |  4.881E-05<br>   16 6.5535E-06 6.5536E-06  ( 0.499   ,  0.493   ,  0.00   ) |  4.719E-05<br>   17 1.3107E-05 1.3107E-05  ( 0.499   ,  0.487   ,  0.00   ) |  4.400E-05<br>   18 2.6214E-05 2.6214E-05  ( 0.499   ,  0.487   ,  0.00   ) |  3.861E-05<br>   19 5.2429E-05 3.1822E-05  ( 0.499   ,  0.487   ,  0.00   ) |  3.182E-05<br> iteration, no. not moved =  0,  56<br> iteration, no. not moved =  1,  2<br> iteration, no. not moved =  2,  0<br> refined: total leaf blocks =  160<br> refined: total blocks =  213<br>   20 8.4250E-05 2.8211E-05  ( 0.499   ,  0.487   ,  0.00   ) |  2.821E-05<br>   21 1.1246E-04 2.6569E-05  ( 0.499   ,  0.487   ,  0.00   ) |  2.657E-05<br>   22 1.3903E-04 2.5651E-05  ( 0.499   ,  0.487   ,  0.00   ) |  2.565E-05<br><div><br></div><div>snip</div><div><br></div>   64 5.0247E-04 1.7003E-06  ( 0.501   ,  0.501   ,  0.00   ) |  1.700E-06<br>   65 5.0417E-04 1.6235E-06  ( 0.501   ,  0.501   ,  0.00   ) |  1.624E-06<br>   66 5.0580E-04 1.5568E-06  ( 0.501   ,  0.501   ,  0.00   ) |  1.557E-06<br>   67 5.0735E-04 1.4925E-06  ( 0.501   ,  0.501   ,  0.00   ) |  1.493E-06<br>   68 5.0885E-04 1.4258E-06  ( 0.501   ,  0.501   ,  0.00   ) |  1.426E-06<br>   69 5.1027E-04 1.3543E-06  ( 0.501   ,  0.501   ,  0.00   ) |  1.354E-06<br>   70 5.1163E-04 1.0000E-10  ( 0.513   ,  0.489   ,  0.00   ) |  1.888E-14<br>   71 5.1163E-04 1.0000E-10  ( 0.487   ,  0.489   ,  0.00   ) |  3.189E-44<br> dtCheck= 0.<br> DRIVER_ABORT: [Hydro]: Computed dt is not positive! Aborting!<br>--------------------------------------------------------------------------<br>MPI_ABORT was invoked on rank 1 in communicator MPI_COMM_WORLD<br>with errorcode 1.<br><br>NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.<br>You may or may not see output from other processes, depending on<br>exactly when Open MPI kills them.<br>--------------------------------------------------------------------------<br>[acalder@fj-debug1 object]$ <br><br><br>Just to play with it, I tried the split hydro solver. In Units, I see among others<br><br>INCLUDE Driver/DriverMain/Split<br>INCLUDE physics/Hydro/HydroMain/split/PPM/PPMKernel<br><br>And when I run it, I see an intriguing warning<br><br>WARNING after gc filling: min. unk(EINT_VAR)=9.9999999735241242E-11      PE=4   block=6       <br>          type=1<br><br>Here's the output of a run:<br><br>[acalder@fj-debug1 object]$ !mpi<br>mpiexec -n 8 ./flash4<br> RuntimeParameters_read:  ignoring unknown parameter "order"...<br> RuntimeParameters_read:  ignoring unknown parameter "slopeLimiter"...<br> RuntimeParameters_read:  ignoring unknown parameter "LimitedSlopeBeta"...<br> RuntimeParameters_read:  ignoring unknown parameter "use_avisc"...<br> RuntimeParameters_read:  ignoring unknown parameter "use_flattening"...<br> RuntimeParameters_read:  ignoring unknown parameter "use_upwindTVD"...<br> RuntimeParameters_read:  ignoring unknown parameter "RiemannSolver"...<br> RuntimeParameters_read:  ignoring unknown parameter "entropy"...<br> RuntimeParameters_read:  ignoring unknown parameter "shockDetect"...<br> Grid_init: resolution based on runtime params:<br> lrefine      dx        dy<br>   1       0.125      0.125<br>   2       0.063      0.063<br>   3       0.031      0.031<br>   4       0.016      0.016<br>   5       0.008      0.008<br>   6       0.004      0.004<br> MaterialProperties initialized<br> Cosmology initialized<br> Source terms initialized<br> sim_rhoAmbient is 1.<br> iteration, no. not moved =  2*0<br> refined: total leaf blocks =  1<br> refined: total blocks =  1<br> [amr_morton_process]: Initializing surr_blks using standard orrery implementation<br> INFO: Grid_fillGuardCells is ignoring masking.<br> iteration, no. not moved =  2*0<br> refined: total leaf blocks =  4<br> refined: total blocks =  5<br> iteration, no. not moved =  0,  3<br> iteration, no. not moved =  2*1<br> iteration, no. not moved =  2,  0<br> refined: total leaf blocks =  16<br> refined: total blocks =  21<br> iteration, no. not moved =  0,  6<br> iteration, no. not moved =  1,  0<br> refined: total leaf blocks =  28<br> refined: total blocks =  37<br> iteration, no. not moved =  0,  9<br> iteration, no. not moved =  1,  2<br> iteration, no. not moved =  2,  0<br> refined: total leaf blocks =  40<br> refined: total blocks =  53<br> iteration, no. not moved =  0,  16<br> iteration, no. not moved =  1,  2<br> iteration, no. not moved =  2,  0<br> refined: total leaf blocks =  52<br> refined: total blocks =  69<br> Finished with Grid_initDomain, no restart<br> Ready to call Hydro_init<br> Info: Hydro_init has set hy_specialFluxVars to  3<br> Hydro initialized<br> Gravity initialized<br> Initial dt verified<br> *** Wrote checkpoint file to sedov_hdf5_chk_0000 ****<br> *** Wrote plotfile to sedov_hdf5_plt_cnt_0000 ****<br> Initial plotfile written<br> Driver init all done<br>    n      t     dt  (     x,      y,      z) |  dt_hydro <br>    1 2.0000E-10 2.0000E-10  ( 0.498   ,  0.502   ,  0.00   ) |  7.969E-05<br>WARNING after gc filling: min. unk(EINT_VAR)=9.9999999735241242E-11      PE=4   block=6       <br>          type=1                <br> 16  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 15  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 14  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 13  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 12  0.25E-04 0.25E-04 0.10E-09 0.10E-09 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 11  0.25E-04 0.25E-04 0.10E-09 0.10E-09 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 10  0.25E-04 0.25E-04 0.10E-09 0.10E-09 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 9  0.25E-04 0.25E-04 0.10E-09 0.10E-09 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 8  0.25E-04 0.25E-04 0.10E-09 0.10E-09 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>10E-09 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 7  0.25E-04 0.25E-04 0.10E-09 0.10E-09 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.26E-04 0.<br>10E-09 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 6  0.25E-04 0.25E-04 0.10E-09 0.10E-09 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.26E-04 0.<br>10E-09 0.25E-04 0.25E-04 0.25E-04 0.18E-04<br> 5  0.25E-04 0.25E-04 0.10E-09 0.10E-09 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.26E-04 0.<br>10E-09 0.25E-04 0.25E-04 0.25E-04 0.22E-04<br> 4  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>15E-04 0.25E-04 0.25E-04 0.25E-04 0.13E-04<br> 3  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>15E-04 0.25E-04 0.25E-04 0.25E-04 0.17E-04<br> 2  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>15E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 1  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>23E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br>WARNING after gc filling: min. unk(EINT_VAR)=9.9999999735241242E-11      PE=5   block=3       <br>          type=1                <br> 16  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 15  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 14  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 13  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 12  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 11  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 10  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 9  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 8  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 7  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 6  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 5  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 4  0.25E-04 0.25E-04 0.10E-09 0.10E-09 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 3  0.25E-04 0.25E-04 0.10E-09 0.10E-09 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 2  0.25E-04 0.25E-04 0.10E-09 0.10E-09 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 1  0.25E-04 0.25E-04 0.10E-09 0.10E-09 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br>WARNING after gc filling: min. unk(EINT_VAR)=9.9999999384891323E-11      PE=4   block=6       <br>          type=1                <br> 16  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 15  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 14  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 13  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 12  0.25E-04 0.25E-04 0.10E-09 0.10E-09 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 11  0.25E-04 0.25E-04 0.10E-09 0.10E-09 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 10  0.25E-04 0.25E-04 0.10E-09 0.10E-09 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 9  0.25E-04 0.25E-04 0.10E-09 0.10E-09 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 8  0.25E-04 0.25E-04 0.10E-09 0.10E-09 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.26E-04 0.<br>10E-09 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 7  0.25E-04 0.25E-04 0.10E-09 0.10E-09 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.34E-04 0.<br>10E-09 0.25E-04 0.25E-04 0.25E-04 0.25E-04<br> 6  0.25E-04 0.25E-04 0.10E-09 0.10E-09 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.34E-04 0.<br>10E-09 0.25E-04 0.25E-04 0.25E-04 0.50E-04<br> 5  0.25E-04 0.25E-04 0.10E-09 0.10E-09 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.34E-04 0.<br>10E-09 0.25E-04 0.25E-04 0.25E-04 0.81E-04<br> 4  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>28E-05 0.25E-04 0.25E-04 0.25E-04 0.13E-04<br> 3  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>28E-05 0.25E-04 0.25E-04 0.25E-04 0.15E-04<br> 2  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>28E-05 0.24E-04 0.25E-04 0.25E-04 0.25E-04<br> 1  0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.25E-04 0.<br>21E-04 0.24E-04 0.25E-04 0.25E-04 0.25E-04<br>WARNING after gc filling: min. unk(EINT_VAR)=9.9999999384891323E-11      PE=5   block=3       <br>          type=1                <br><br><br>Finally, here are warnings.<br><br><br>Several things like this:<br><br>USE_16_API -O0 -c -DMAXBLOCKS=1000 -DNXB=8 -DNYB=8 -DNZB=1 -DN_DIM=2 io_attribute.c<br>io_attribute.c: In function ‘io_attribute_create_’:<br>io_attribute.c:66:5: warning: implicit declaration of function ‘Driver_abortFlashC’ [-Wimplicit-function-d<br>eclaration]<br>   Driver_abortFlashC("[io_attribute_create]: Unknown I/O");<br>   ^~~~~~~~~~~~~~~~~~<br><br><br>And this one:<br><br>/lustre/projects/global/software/a64fx/openmpi/4.0.5/bin/mpicc -I/cm/shared/apps/hdf5/1.10.1/include -DH5_<br>USE_16_API -O0 -c -DMAXBLOCKS=1000 -DNXB=8 -DNYB=8 -DNZB=1 -DN_DIM=2 io_h5write_generic_real_arr.c<br>io_h5write_generic_int_arr.c: In function ‘io_h5write_generic_int_arr_’:<br>io_h5write_generic_int_arr.c:82:31: warning: implicit declaration of function ‘malloc’ [-Wimplicit-functio<br>n-declaration]<br>  dataset_name_new = (char *) malloc((*name_len) + 1 * sizeof(char));<br>                ^~~~~~<br>io_h5write_generic_int_arr.c:82:31: warning: incompatible implicit declaration of built-in function ‘mallo<br>c’<br>io_h5write_generic_int_arr.c:82:31: note: include ‘<stdlib.h>’ or provide a declaration of ‘malloc’<br>io_h5write_generic_int_arr.c:6:1:<br>+#include <stdlib.h><br> <br>io_h5write_generic_int_arr.c:82:31:<br>  dataset_name_new = (char *) malloc((*name_len) + 1 * sizeof(char));<br>                ^~~~~~<br>io_h5write_generic_int_arr.c:87:3: warning: implicit declaration of function ‘strncpy’ [-Wimplicit-functio<br>n-declaration]<br>  strncpy(dataset_name_new, dataset_name, *name_len);<br>  ^~~~~~~<br>io_h5write_generic_int_arr.c:87:3: warning: incompatible implicit declaration of built-in function ‘strncp<br>y’<br>io_h5write_generic_int_arr.c:87:3: note: include ‘<string.h>’ or provide a declaration of ‘strncpy’<br>io_h5write_generic_int_arr.c:6:1:<br>+#include <string.h><br> <br>io_h5write_generic_int_arr.c:87:3:<br>  strncpy(dataset_name_new, dataset_name, *name_len);<br>  ^~~~~~~<br>io_h5write_generic_int_arr.c:187:3: warning: implicit declaration of function ‘free’ [-Wimplicit-function-<br>declaration]<br>  free(dataset_name_new);<br>  ^~~~<br>io_h5write_generic_int_arr.c:187:3: warning: incompatible implicit declaration of built-in function ‘free’<br>io_h5write_generic_int_arr.c:187:3: note: include ‘<stdlib.h>’ or provide a declaration of ‘free’<br>/lustre/projects/global/software/a64fx/openmpi/4.0.5/bin/mpicc -I/cm/shared/apps/hdf5/1.10.1/include -DH5_<br>USE_16_API -O0 -c -DMAXBLOCKS=1000 -DNXB=8 -DNYB=8 -DNZB=1 -DN_DIM=2 io_h5write_generic_real_arr.c<br>io_h5write_generic_real_arr.c: In function ‘io_h5write_generic_real_arr_’:<br>io_h5write_generic_real_arr.c:33:31: warning: implicit declaration of function ‘malloc’ [-Wimplicit-functi<br>on-declaration]<br>  dataset_name_new = (char *) malloc((*name_len) + 1 * sizeof(char));<br>                ^~~~~~<br>io_h5write_generic_real_arr.c:33:31: warning: incompatible implicit declaration of built-in function ‘mall<br>oc’<br>io_h5write_generic_real_arr.c:33:31: note: include ‘<stdlib.h>’ or provide a declaration of ‘malloc’<br>io_h5write_generic_real_arr.c:6:1:<br>+#include <stdlib.h><br> <br>io_h5write_generic_real_arr.c:33:31:<br>  dataset_name_new = (char *) malloc((*name_len) + 1 * sizeof(char));<br>                ^~~~~~<br>io_h5write_generic_real_arr.c:38:3: warning: implicit declaration of function ‘strncpy’ [-Wimplicit-functi<br>on-declaration]<br>  strncpy(dataset_name_new, dataset_name, *name_len);<br>  ^~~~~~~<br>io_h5write_generic_real_arr.c:38:3: warning: incompatible implicit declaration of built-in function ‘strnc<br>py’<br>io_h5write_generic_real_arr.c:38:3: note: include ‘<string.h>’ or provide a declaration of ‘strncpy’<br>io_h5write_generic_real_arr.c:6:1:<br>+#include <string.h><br> <br>io_h5write_generic_real_arr.c:38:3:<br>  strncpy(dataset_name_new, dataset_name, *name_len);<br>  ^~~~~~~<br>io_h5write_generic_real_arr.c:139:3: warning: implicit declaration of function ‘free’ [-Wimplicit-function<br>-declaration]<br>  free(dataset_name_new);<br>  ^~~~<br>io_h5write_generic_real_arr.c:139:3: warning: incompatible implicit declaration of built-in function ‘free<br>’<br>io_h5write_generic_real_arr.c:139:3: note: include ‘<stdlib.h>’ or provide a declaration of ‘free’<br><br><br><br>Thanks for any advice or suggestions!<br><br>Alan Calder<br><br clear="all"><br>-- <br><div dir="ltr" data-smartmail="gmail_signature"><div dir="ltr">Alan C. Calder<br>Department of Physics and Astronomy<br>State University of New York at Stony Brook<br>Stony Brook, NY 11794-3800<br><br>office: ESS 438<br>phone: (631) 632-1176<br>fax: (631) 632-1745<br>web: <a href="http://www.astro.sunysb.edu/acalder" target="_blank">http://www.astro.sunysb.edu/acalder</a><br></div></div></div>