[FLASH-USERS] BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES , EXIT CODE 9
g.granda at irya.unam.mx
g.granda at irya.unam.mx
Fri Sep 28 17:27:17 EDT 2018
Hello,
I have been using Flash 4.5 to run simulations of thermal instability
for a while. Everything was fine until
I tried to increase the resolution of the uniform grid used for previous
simulation. I don't know what is the problem since the error message is
not that informative, but the simulation crash right after enetering the
evolution loop.
Could you please help me?
The printed output is :
MaterialProperties initialized
Cosmology initialized
Source terms initialized
Finished with Grid_initDomain, no restart
Ready to call Hydro_init
Hydro initialized
Gravity initialized
Initial dt verified
*** Wrote checkpoint file to lr_hdf5_chk_0000 ****
*** Wrote plotfile to lr_hdf5_plt_cnt_0000 ****
Initial plotfile written
Driver init all done
===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= PID 29030 RUNNING AT compute-0-1
= EXIT CODE: 9
= CLEANING UP REMAINING PROCESSES
= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
The error file generated after running was :
[proxy:0:0 at compute-0-2.local] HYD_pmcd_pmip_control_cmd_cb
(pm/pmiserv/pmip_cb.c:887): assert (!closed) failed
[proxy:0:0 at compute-0-2.local] HYDT_dmxu_poll_wait_for_event
(tools/demux/demux_poll.c:76): callback returned error status
[proxy:0:0 at compute-0-2.local] main (pm/pmiserv/pmip.c:202): demux engine
error waiting for event
[mpiexec at compute-0-2.local] HYDT_bscu_wait_for_completion
(tools/bootstrap/utils/bscu_wait.c:76): one of the processes terminated
badly; aborting
[mpiexec at compute-0-2.local] HYDT_bsci_wait_for_completion
(tools/bootstrap/src/bsci_wait.c:23): launcher returned error waiting
for completion
[mpiexec at compute-0-2.local] HYD_pmci_wait_for_completion
(pm/pmiserv/pmiserv_pmci.c:218): launcher returned error waiting for
completion
[mpiexec at compute-0-2.local] main (ui/mpich/mpiexec.c:340): process
manager error waiting for completion
And the log file of the run is:
usehydro = T
useincompns = F
useionize = F
uselegacylabels = T
usemagneticresistivity = F
usemassdiffusivity = F
useopacity = F
useparticles = F
useplasmastate = F
usepolytrope = F
useprimordialchemistry = F
useprotonemission = F
useprotonimaging = F
useradtrans = F
useraytrace = F
usests = F
usestsfordiffusion = F
usestir = F
usethomsonscattering = F
usetreeray = F
useturb = T
useviscosity = F
usexrayimaging = F
use_3dfullctu = T
use_auxeinteqn = T
use_avisc = F
use_cma_advection = F
use_cma_flattening = F
use_flattening = F
use_gravhalfupdate = T
use_hybridorder = F
use_steepening = F
use_upwindtvd = F
writestatsummary = T
==============================================================================
WARNING: Ignored Parameters :
These parameters were found in the flash.par file, but they were
not declared in any Config file for the simulation!
sim_dens_num
==============================================================================
Known units of measurement:
Unit CGS Value
Base Unit
1 cm 1.0000
cm
2 s 1.0000
s
3 g 1.0000
g
4 K 1.0000
K
5 esu 1.0000
esu
6 mol 1.0000
mol
7 m 100.00
cm
8 km 1.00000E+05
cm
9 pc 3.08568E+18
cm
10 kpc 3.08568E+21
cm
11 Mpc 3.08568E+24
cm
12 Gpc 3.08568E+27
cm
13 Rsun 6.96000E+10
cm
14 AU 1.49598E+13
cm
15 yr 3.15569E+07
s
16 Myr 3.15569E+13
s
17 Gyr 3.15569E+16
s
18 kg 1000.0
g
19 Msun 1.98892E+33
g
20 amu 1.66054E-24
g
21 eV 11605.
K
22 C 2.99792E+09
esu
23 LFLY 3.08568E+24
cm
24 TFLY 2.05759E+17
s
25 MFLY 9.88470E+45
g
26 clLength 3.08568E+24
cm
27 clTime 3.15569E+16
s
28 clMass 1.98892E+48
g
29 clTemp 1.16045E+07
K
-----------End of Units--------------------
Known physical constants:
Constant Name Constant Value cm s g K
esu mol
1 Newton 6.67408E-08 3.0 -2.0 -1.0 0.0
0.0 0.0
2 speed of light 2.99792E+10 1.0 -1.0 0.0 0.0
0.0 0.0
3 Planck 6.62607E-27 2.0 -1.0 1.0 0.0
0.0 0.0
4 electron charge 4.80320E-10 0.0 0.0 0.0 0.0
1.0 0.0
5 electron mass 9.10938E-28 0.0 0.0 1.0 0.0
0.0 0.0
6 proton mass 1.67262E-24 0.0 0.0 1.0 0.0
0.0 0.0
7 fine-structure 7.29735E-03 0.0 0.0 0.0 0.0
0.0 0.0
8 Avogadro 6.02214E+23 0.0 0.0 0.0 0.0
0.0 -1.0
9 Boltzmann 1.38065E-16 2.0 -2.0 1.0 -1.0
0.0 0.0
10 ideal gas constant 8.31446E+07 2.0 -2.0 1.0 -1.0
0.0 -1.0
11 Wien 0.28978 1.0 0.0 0.0 1.0
0.0 0.0
12 Stefan-Boltzmann 5.67037E-05 0.0 -3.0 1.0 -4.0
0.0 0.0
13 Radiation Constant 7.56572E-15 -1.0 -2.0 1.0 -4.0
0.0 0.0
14 pi 3.1416 0.0 0.0 0.0 0.0
0.0 0.0
15 e 2.7183 0.0 0.0 0.0 0.0
0.0 0.0
16 Euler 0.57722 0.0 0.0 0.0 0.0
0.0 0.0
==============================================================================
Multifluid database: not configured in
==============================================================================
[ 09-21-2018 17:20:18.790 ] [gr_initGeometry] checking BCs for idir: 1
[ 09-21-2018 17:20:18.790 ] [gr_initGeometry] checking BCs for idir: 2
[ 09-21-2018 17:20:18.791 ] [gr_initGeometry] checking BCs for idir: 3
[ 09-21-2018 17:20:27.954 ] memory: /proc vsize (MiB): 1164.65
(min) 1164.71 (max) 1164.71 (avg)
[ 09-21-2018 17:20:27.954 ] memory: /proc rss (MiB): 224.20
(min) 224.47 (max) 224.27 (avg)
[ 09-21-2018 17:20:27.984 ] memory: /proc vsize (MiB): 1164.65
(min) 1164.71 (max) 1164.71 (avg)
[ 09-21-2018 17:20:27.984 ] memory: /proc rss (MiB): 224.23
(min) 224.52 (max) 224.30 (avg)
[ 09-21-2018 17:20:30.478 ] memory: /proc vsize (MiB): 1164.65
(min) 1164.71 (max) 1164.71 (avg)
[ 09-21-2018 17:20:30.479 ] memory: /proc rss (MiB): 224.26
(min) 224.54 (max) 224.32 (avg)
[ 09-21-2018 17:20:32.397 ] [IO_writeCheckpoint] open: type=checkpoint
name=lr_hdf5_chk_0000
[ 09-21-2018 17:27:09.537 ] [IO_writeCheckpoint] close:
type=checkpoint name=lr_hdf5_chk_0000
[ 09-21-2018 17:27:10.066 ] [IO_writePlotfile] open: type=plotfile
name=lr_hdf5_plt_cnt_0000
[ 09-21-2018 17:29:48.712 ] [IO_writePlotfile] close: type=plotfile
name=lr_hdf5_plt_cnt_0000
[ 09-21-2018 17:29:48.713 ] memory: /proc vsize (MiB): 1180.71
(min) 1204.71 (max) 1181.19 (avg)
[ 09-21-2018 17:29:48.714 ] memory: /proc rss (MiB): 242.32
(min) 277.39 (max) 243.06 (avg)
[ 09-21-2018 17:29:48.715 ] [Driver_evolveFlash]: Entering evolution
loop
[ 09-21-2018 17:29:48.717 ] step: n=1 t=0.000000E+00 dt=1.370702E+08
FLASH log file: 09-28-2018 15:46:36.193 Run number: 1
==============================================================================
Number of MPI tasks: 64
MPI version: 3
MPI subversion: 1
Dimensionality: 3
Max Number of Blocks/Proc: 1
Number x zones: 128
Number y zones: 128
Number z zones: 128
Setup stamp: Fri Sep 21 12:33:44 2018
Build stamp: vie sep 21 12:34:17 2018
System info: Linux mouruka.crya.privado 2.6.32-504.16.2.el6.x86_64
#1 SMP Wed Apr 22 06:48:29
Version: FLASH 4.5_release
Build directory: /home/guido/FLASH4.5/obj_lr_2res
Setup syntax: /home/guido/FLASH4.5/bin/setup.py LinearRegime_mz
-auto -3d -objdir=obj_lr_2res +ug -site=irya.guido -nxb=128 -nyb=128
-nzb=128
f compiler flags:
/home/guido/libraries/compiled_with_gcc-7.3.0/mpich-3.2.1/bin/mpif90 -g
-c -O2 -fdefault-real-8 -fdefault-double-8 -ffree-line-length-none
-Wuninitialized -ggdb -c -O2 -fdefault-real-8 -fdefault-double-8
-ffree-line-length-none -Wuninitialized -DMAXBLOCKS=1 -DNXB=128
-DNYB=128 -DNZB=128 -DN_DIM=3
c compiler flags:
/home/guido/libraries/compiled_with_gcc-7.3.0/mpich-3.2.1/bin/mpicc
-I/home/guido/libraries/compiled_with_gcc-7.3.0/hdf5-1.8.20/include
-DH5_USE_16_API -O2 -c -DMAXBLOCKS=1 -DNXB=128 -DNYB=128 -DNZB=128
-DN_DIM=3
==============================================================================
Comment: Linear Regime
==============================================================================
FLASH Units used:
Driver/DriverMain/Unsplit
Driver/localAPI
Grid/GridBoundaryConditions
Grid/GridMain/UG
Grid/localAPI
IO/IOMain/hdf5/serial/UG
IO/localAPI
PhysicalConstants/PhysicalConstantsMain
RuntimeParameters/RuntimeParametersMain
Simulation/SimulationMain/LinearRegime_mz
flashUtilities/contiguousConversion
flashUtilities/general
flashUtilities/interpolation/oneDim
flashUtilities/nameValueLL
flashUtilities/system/memoryUsage/legacy
monitors/Logfile/LogfileMain
monitors/Timers/TimersMain/MPINative
physics/Eos/EosMain/Gamma
physics/Eos/localAPI
physics/Hydro/HydroMain/unsplit/Hydro_Unsplit
physics/Hydro/localAPI
physics/sourceTerms/Cool/CoolMain/equilibrium_cooling
==============================================================================
RuntimeParameters:
==============================================================================
bndpriorityone = 1
bndprioritythree = 3
bndprioritytwo = 2
checkpointfileintervalstep = 100000000 [CHANGED]
checkpointfilenumber = 0
dr_abortpause = 2
dr_dtminbelowaction = 1
dr_numposdefvars = 4
drift_break_inst = 0
drift_trunc_mantissa = 2
drift_verbose_inst = 0
eos_entrelescalechoice = 6
eos_loglevel = 700
fileformatversion = 9
forcedplotfilenumber = 0
hy_3torder = -1
hydrocomputedtoption = -1
igridsize = 1
iprocs = 4 [CHANGED]
iguard = 4
irenorm = 0
jgridsize = 1
jprocs = 4 [CHANGED]
jguard = 4
kgridsize = 1
kprocs = 4 [CHANGED]
kguard = 4
memory_stat_freq = 100000
meshcopycount = 1
nbegin = 1
nblockx = 1
nblocky = 1
nblockz = 1
nend = 1000000 [CHANGED]
nsteptotalsts = 5
order = 2
outputsplitnum = 1
plotfileintervalstep = 1000000 [CHANGED]
plotfilenumber = 0
rolling_checkpoint = 10000
sim_nk = 1 [CHANGED]
sweeporder = 123
transorder = 1
wr_integrals_freq = 1
limitedslopebeta = 0.100E+01
t_cool_min = 0.100E+02 [CHANGED]
cfl = 0.500E+00 [CHANGED]
checkpointfileintervaltime = 0.343E+14 [CHANGED]
checkpointfileintervalz = 0.180+309
cvisc = 0.100E+00
dr_dtmincontinue = 0.000E+00
dr_posdefdtfactor = 0.100E+01
dr_tstepslowstartfactor = 0.100E+00
dtinit = 0.137E+09 [CHANGED]
dtmax = 0.137E+11 [CHANGED]
dtmin = 0.137E+08 [CHANGED]
eintswitch = 0.000E+00
eos_singlespeciesa = 0.100E+01
eos_singlespeciesz = 0.100E+01
gamma = 0.167E+01 [CHANGED]
hy_cflfallbackfactor = 0.900E+00
hy_fpresinmomflux = 0.100E+01
hybridorderkappa = 0.000E+00
mu_mol = 0.127E+01
nd_cool_max = 0.100E+21
nd_cool_min = 0.100E-01
nusts = 0.100E+00
plotfileintervaltime = 0.137E+13 [CHANGED]
plotfileintervalz = 0.180+309
radiusgp = 0.200E+01
rss_limit = -0.100E+01
sigmagp = 0.300E+01
sim_num_dens = 0.300E+01
sim_rho_amp = 0.100E-02
sim_temp = 0.730E+03
small = 0.100E-39 [CHANGED]
smalle = 0.100E-09
smallp = 0.100E-21 [CHANGED]
smallt = 0.100E+01 [CHANGED]
smallu = 0.100E-39 [CHANGED]
smallx = 0.100E-09
smlrho = 0.100E-29 [CHANGED]
tinitial = 0.000E+00
tiny = 0.100E-15
tmax = 0.137E+15 [CHANGED]
tstep_change_factor = 0.200E+01
wall_clock_checkpoint = 0.108E+05 [CHANGED]
wall_clock_time_limit = 0.605E+06
xmax = 0.154E+20 [CHANGED]
xmin = -0.154E+20 [CHANGED]
ymax = 0.154E+20 [CHANGED]
ymin = -0.154E+20 [CHANGED]
zfinal = 0.000E+00
zinitial = -0.100E+01
zmax = 0.154E+20 [CHANGED]
zmin = -0.154E+20 [CHANGED]
riemannsolver = HLLC
unitsystem = CGS [CHANGED]
basenm = lr_ [CHANGED]
dr_posdefvar_1 = none
dr_posdefvar_2 = none
dr_posdefvar_3 = none
dr_posdefvar_4 = none
entropyfixmethod = HARTENHYMAN
eosmode = dens_pres [CHANGED]
eosmodeinit = dens_ie
geometry = cartesian
grav_boundary_type = isolated
hy_eosmodegc = see eosMode
log_file = lr.log [CHANGED]
output_directory =
pc_unitsbase = CGS
plot_grid_var_1 = none
plot_grid_var_10 = none
plot_grid_var_11 = none
plot_grid_var_12 = none
plot_grid_var_2 = none
plot_grid_var_3 = none
plot_grid_var_4 = none
plot_grid_var_5 = none
plot_grid_var_6 = none
plot_grid_var_7 = none
plot_grid_var_8 = none
plot_grid_var_9 = none
plot_var_1 = dens [CHANGED]
plot_var_10 = none
plot_var_11 = none
plot_var_12 = none
plot_var_2 = pres [CHANGED]
plot_var_3 = temp [CHANGED]
plot_var_4 = eint [CHANGED]
plot_var_5 = velx [CHANGED]
plot_var_6 = vely [CHANGED]
plot_var_7 = velz [CHANGED]
plot_var_8 = none
plot_var_9 = none
prof_file = profile.dat
refine_var_thresh = none
run_comment = Linear Regime [CHANGED]
run_number = 1
slopelimiter = vanLeer
stats_file = flash.dat
wenomethod = WENO5
xl_boundary_type = periodic
xr_boundary_type = periodic
yl_boundary_type = periodic
yr_boundary_type = periodic
zl_boundary_type = periodic
zr_boundary_type = periodic
eosforriemann = F
addthermalflux = T
allowdtstsdominate = F
alwayscomputeuservars = T
alwaysrestrictcheckpoint = T
charlimiting = T
chkguardcellsinput = F
chkguardcellsoutput = F
compute_grid_size = T
conserveangmom = F
converttoconsvdformeshcalls = F
corners = F
dr_printtsteploc = T
dr_shortenlaststepbeforetmax = F
dr_useposdefcomputedt = F
drift_tuples = F
eachprocwritesownabortlog = F
eachprocwritessummary = F
entropy = F
flux_correct = F
geometryoverride = F
gr_bcenableapplymixedgds = T
hy_fallbacklowercfl = F
hy_fullspecmsfluxhandling = T
ignoreforcedplot = F
io_writemscalarintegrals = F
plotfilegridquantitydp = F
plotfilemetadatadp = F
reducegcellfills = F
restart = F
shockdetect = F
shocklowercfl = F
summaryoutputonly = F
threadblocklistbuild = F
threaddriverblocklist = F
threaddriverwithinblock = F
threadeoswithinblock = F
threadhydroblocklist = F
threadhydrowithinblock = F
threadraytracebuild = F
threadwithinblockbuild = F
typematchedxfer = T
unbiased_geometry = F
updatehydrofluxes = T
useburn = F
usecollectivehdf5 = T
useconductivity = F
usecool = F
usecosmology = F
usedeleptonize = F
usediffuse = F
usediffusecomputedtspecies = F
usediffusecomputedttherm = F
usediffusecomputedtvisc = F
usediffusecomputedtmagnetic = F
useenergydeposition = F
useflame = F
usegravity = F
useheat = F
useheatexchange = F
usehydro = T
useincompns = F
useionize = F
uselegacylabels = T
usemagneticresistivity = F
usemassdiffusivity = F
useopacity = F
useparticles = F
useplasmastate = F
usepolytrope = F
useprimordialchemistry = F
useprotonemission = F
useprotonimaging = F
useradtrans = F
useraytrace = F
usests = F
usestsfordiffusion = F
usestir = F
usethomsonscattering = F
usetreeray = F
useturb = T
useviscosity = F
usexrayimaging = F
use_3dfullctu = T
use_auxeinteqn = T
use_avisc = F
use_cma_advection = F
use_cma_flattening = F
use_flattening = F
use_gravhalfupdate = T
use_hybridorder = F
use_steepening = F
use_upwindtvd = F
writestatsummary = T
==============================================================================
WARNING: Ignored Parameters :
These parameters were found in the flash.par file, but they were
not declared in any Config file for the simulation!
sim_dens_num
==============================================================================
Known units of measurement:
Unit CGS Value
Base Unit
1 cm 1.0000
cm
2 s 1.0000
s
3 g 1.0000
g
4 K 1.0000
K
5 esu 1.0000
esu
6 mol 1.0000
mol
7 m 100.00
cm
8 km 1.00000E+05
cm
9 pc 3.08568E+18
cm
10 kpc 3.08568E+21
cm
11 Mpc 3.08568E+24
cm
12 Gpc 3.08568E+27
cm
13 Rsun 6.96000E+10
cm
14 AU 1.49598E+13
cm
15 yr 3.15569E+07
s
16 Myr 3.15569E+13
s
17 Gyr 3.15569E+16
s
18 kg 1000.0
g
19 Msun 1.98892E+33
g
20 amu 1.66054E-24
g
21 eV 11605.
K
22 C 2.99792E+09
esu
23 LFLY 3.08568E+24
cm
24 TFLY 2.05759E+17
s
25 MFLY 9.88470E+45
g
26 clLength 3.08568E+24
cm
27 clTime 3.15569E+16
s
28 clMass 1.98892E+48
g
29 clTemp 1.16045E+07
K
-----------End of Units--------------------
Known physical constants:
Constant Name Constant Value cm s g K
esu mol
1 Newton 6.67408E-08 3.0 -2.0 -1.0 0.0
0.0 0.0
2 speed of light 2.99792E+10 1.0 -1.0 0.0 0.0
0.0 0.0
3 Planck 6.62607E-27 2.0 -1.0 1.0 0.0
0.0 0.0
4 electron charge 4.80320E-10 0.0 0.0 0.0 0.0
1.0 0.0
5 electron mass 9.10938E-28 0.0 0.0 1.0 0.0
0.0 0.0
6 proton mass 1.67262E-24 0.0 0.0 1.0 0.0
0.0 0.0
7 fine-structure 7.29735E-03 0.0 0.0 0.0 0.0
0.0 0.0
8 Avogadro 6.02214E+23 0.0 0.0 0.0 0.0
0.0 -1.0
9 Boltzmann 1.38065E-16 2.0 -2.0 1.0 -1.0
0.0 0.0
10 ideal gas constant 8.31446E+07 2.0 -2.0 1.0 -1.0
0.0 -1.0
11 Wien 0.28978 1.0 0.0 0.0 1.0
0.0 0.0
12 Stefan-Boltzmann 5.67037E-05 0.0 -3.0 1.0 -4.0
0.0 0.0
13 Radiation Constant 7.56572E-15 -1.0 -2.0 1.0 -4.0
0.0 0.0
14 pi 3.1416 0.0 0.0 0.0 0.0
0.0 0.0
15 e 2.7183 0.0 0.0 0.0 0.0
0.0 0.0
16 Euler 0.57722 0.0 0.0 0.0 0.0
0.0 0.0
==============================================================================
Multifluid database: not configured in
==============================================================================
[ 09-28-2018 15:46:36.267 ] [gr_initGeometry] checking BCs for idir: 1
[ 09-28-2018 15:46:36.272 ] [gr_initGeometry] checking BCs for idir: 2
[ 09-28-2018 15:46:36.277 ] [gr_initGeometry] checking BCs for idir: 3
[ 09-28-2018 15:46:43.593 ] memory: /proc vsize (MiB): 1164.65
(min) 1164.71 (max) 1164.71 (avg)
[ 09-28-2018 15:46:43.598 ] memory: /proc rss (MiB): 224.21
(min) 224.50 (max) 224.29 (avg)
[ 09-28-2018 15:46:43.604 ] memory: /proc vsize (MiB): 1164.65
(min) 1164.71 (max) 1164.71 (avg)
[ 09-28-2018 15:46:43.610 ] memory: /proc rss (MiB): 224.23
(min) 224.54 (max) 224.32 (avg)
[ 09-28-2018 15:46:45.714 ] memory: /proc vsize (MiB): 1164.65
(min) 1164.71 (max) 1164.71 (avg)
[ 09-28-2018 15:46:45.719 ] memory: /proc rss (MiB): 224.26
(min) 224.56 (max) 224.34 (avg)
[ 09-28-2018 15:46:45.903 ] [IO_writeCheckpoint] open: type=checkpoint
name=lr_hdf5_chk_0000
[ 09-28-2018 15:53:51.458 ] [IO_writeCheckpoint] close:
type=checkpoint name=lr_hdf5_chk_0000
[ 09-28-2018 15:53:51.460 ] [IO_writePlotfile] open: type=plotfile
name=lr_hdf5_plt_cnt_0000
[ 09-28-2018 15:55:57.701 ] [IO_writePlotfile] close: type=plotfile
name=lr_hdf5_plt_cnt_0000
[ 09-28-2018 15:55:57.702 ] memory: /proc vsize (MiB): 1180.71
(min) 1204.71 (max) 1181.19 (avg)
[ 09-28-2018 15:55:57.702 ] memory: /proc rss (MiB): 242.32
(min) 277.42 (max) 243.07 (avg)
[ 09-28-2018 15:55:57.704 ] [Driver_evolveFlash]: Entering evolution
loop
[ 09-28-2018 15:55:57.705 ] step: n=1 t=0.000000E+00 dt=1.370702E+08
Cheers,
~
More information about the flash-users
mailing list