<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">Hi Gabriel & Ryan,<div><br></div><div>Does the stalling happen only for parallel runs (mpirun -np > 1)? It seems likely that it could be stalling during some mpi communication. In my experience making sure all the dependencies are built consistently with the same compilers & mpi has helped.</div><div><br></div><div>As for pinpointing where exactly the stall is happening there are a couple of things you can try:</div><div><br></div><div>* Setup with "-defines=DEBUG ALL”, this will turn on a lot of debugging messages in all the FLASH units.</div><div><br></div><div>* you can add a line like </div><div><blockquote type="cite">print *, “timers start”, name</blockquote></div><div> to “Timers_start/stop.F90</div><div><br></div><div>Both of these should print plenty of messages to help narrow down where exactly the code is stalling.</div><div><ul class="MailOutline"><span><div><br></div><div><br></div></span><span></span></ul></div><div><div><span>*********************************************<br>Adam Reyes</span><br class="Apple-interchange-newline"><span><img alt="FLASH.jpg" src="cid:EA5A7E59-94B9-4162-879C-B52E774AEA24"></span><meta charset="UTF-8"><br style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline !important; float: none;">Code Group Leader, Flash Center for </span><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline !important; float: none;">Computational Science</span><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline !important; float: none;"> </span><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline !important; float: none;"> </span><br style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline !important; float: none;">Research Scientist, Dept. of Physics and </span><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline !important; float: none;">Astronomy</span><br style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline !important; float: none;">University of Rochester</span><br style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline !important; float: none;">River Campus: Bausch and Lomb Hall, </span><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline !important; float: none;">369</span><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline !important; float: none;"> </span><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline !important; float: none;"> </span><br style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline !important; float: none;">500 Wilson Blvd. PO Box 270171, </span><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline !important; float: none;">Rochester, NY 14627</span><br style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline !important; float: none;">Email</span><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline !important; float: none;"> </span><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline !important; float: none;">adam.reyes@rochester.edu</span><br style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline !important; float: none;">Web</span><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline !important; float: none;"> </span><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline !important; float: none;">https://flash.rochester.edu</span><br style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline !important; float: none;"> </span><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline !important; float: none;">(he / him / his)</span><br style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span><img alt="FLASH-pride-sml.png" src="cid:CE45460D-2261-4A69-865C-5CF7BEACF620"></span><div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><br>*********************************************</div><div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><br></div><br class="Apple-interchange-newline">
</div>
<div><br><blockquote type="cite"><div>On Oct 15, 2024, at 12:18 PM, Gabriel Pérez Callejo <gabriel.perez.callejo@uva.es> wrote:</div><br class="Apple-interchange-newline"><div>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<div><p>Hi Ryan,</p><p>Thanks for the quick response. I am attaching to this email the
STDOUT, STDERR and log files.</p><p>To answer your questions, the simulation does stall. The ps
command shows the parallel processes as active, as well as the
mpirun, but no progress is done, nothing is printed in the log,
STDOUT or STDERR files, and if I run a *top* command, the machine
is not working on FLASH.</p><p>I have retried including +noio and -debug in my setup command,
but it works identically, same problem.</p><p>Best,<br>
</p>
<div class="moz-signature"><b>Gabriel Pérez-Callejo</b> <br>
Profesor Ayudante Doctor (Assistant Professor) <br>
Departamento de Física Teórica, Atómica y Óptica <br>
Universidad de Valladolid <br>
Valladolid, Spain <br>
+34 983 18 6513 <br>
<br>
<br>
</div>
<div class="moz-cite-prefix">El 15/10/24 a las 12:00, Ryan Farber
escribió:<br>
</div>
<blockquote type="cite" cite="mid:CAF9W8=3JLRQe379WXrPGN1woCE=fo+301GyT2QinQfjdbEmgsg@mail.gmail.com">
<div dir="ltr">Hi Gabriel,
<div><br>
</div>
<div>I have encountered (and am to some extent still trying to
understand) a similar, possibly the same, issue (also with
FLASH 4.8). I think the usual issue I encounter is caused due
to running out of memory, but it may also be related to
HDF5...</div>
<div><br>
</div>
<div>Regarding your issue, does the run just stall? Such that ps
aux | grep flash shows the process is running but the
simulation makes no progress in outputting to your log file or
STDOUT/STDERR file(s)?<br>
<br>
Or does the run die? [Some error is encountered / ps no longer
shows the process or it's in a completing, i/o, or zombie
state.]<br>
<br>
It would be helpful if you can attach your log file and your
STDOUT/STDERR file(s). It would also be useful if you try
using +noio to determine if you have an HDF5 issue, and -debug
to provide a traceback if an exception is raised.<br>
<br>
It's interesting this happened for you just changing
distributions. I'm hoping you re-installed openmpi, hdf5, etc.
on the new OS rather than copying your installations from your
old OS(?)<br>
<br>
Best wishes,<br clear="all">
<div>
<div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div dir="ltr">--------
<div>Ryan</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, Oct 15, 2024 at
2:51 AM Gabriel Pérez Callejo <<a href="mailto:gabriel.perez.callejo@uva.es" moz-do-not-send="true" class="moz-txt-link-freetext">gabriel.perez.callejo@uva.es</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div><p>Dear all,</p><p>I have been using FLASH for a while in Ubuntu 18, and am
moving now to use the linux distribution OpenSUSE.
However, when running flash in parallel mode, I am
encountering the following problem.</p><p>I am testing the LaserSlab example, with FLASH4.6.2,
using hdf5-1.10.7, hypre-2.11.2 and openmpi-4.0.5 (same as
I used in Ubuntu 18). <br>
</p><p>I am launching the simulation by using <i>"./setup -auto
LaserSlab -2d +cylindrical -nxb=16 -nyb=16 +hdf5typeio
species=cha</i><i><br>
</i><i>m,targ +mtmmmt +laser +uhd3t +mgd mgd_meshgroups=6
-parfile=example.par" </i>then moving to the <i>object</i>
directory, using <i>"make -j"</i> and after SUCCESS
running "<i>mpirun -np 3 flash4"</i>. <br>
</p><p>Now, this is what I used to do in Ubuntu, but what I am
finding in this case is that the calculation is
initialized, but after printing <i>"Initial dt verified"
</i>nothing else happens. The code does not move forward.
I can see that the chk_0000 file has been generated, but
not the plt_0000.</p><p>Has anyone encountered this problem before? Does anyone
have any suggestions on how to fix it?</p><p>Best,<br>
</p>
<div>-- <br>
<b>Gabriel Pérez-Callejo</b> <br>
Profesor Ayudante Doctor (Assistant Professor) <br>
Departamento de Física Teórica, Atómica y Óptica <br>
Universidad de Valladolid <br>
Valladolid, Spain <br>
+34 983 18 6513 <br>
<br>
<br>
</div>
</div>
_______________________________________________<br>
flash-users mailing list<br>
<a href="mailto:flash-users@flash.rochester.edu" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">flash-users@flash.rochester.edu</a><br>
<br>
For list info, including unsubscribe:<br>
<a href="https://flash.rochester.edu/mailman/listinfo/flash-users" rel="noreferrer" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">https://flash.rochester.edu/mailman/listinfo/flash-users</a><br>
</blockquote>
</div>
</blockquote>
</div>
<span id="cid:F1587759-672D-4571-B16D-58436AC90EA1"><lasslab.log></span><span id="cid:822F86A6-25C0-4EC4-8773-CBFDBE295191"><STDERR></span><span id="cid:2F8F9989-548F-4AA5-A5BE-BEFDC4643712"><STDOUT.txt></span>_______________________________________________<br>flash-users mailing list<br>flash-users@flash.rochester.edu<br><br>For list info, including unsubscribe:<br>https://flash.rochester.edu/mailman/listinfo/flash-users<br></div></blockquote></div><br></div></body></html>