It looks like pointer memory allocation is failing for OpenMPI. I would try a different version of OpenMPI.<br>Can you tell me what instruction it is attempting to execute at opal_memory_ptmalloc2_int_malloc()+(~0x260-0x2aa) and opal_memory_ptmalloc2_malloc+(~0x10-0x59)?<br>
You can enter the offset in hte, or use gdb.<br><br><a href="Http://hte">Http://hte</a>(dot)sourceforge(dot)net<br><br>Thanks,<br><br>Michael Sabino<br><br><div class="gmail_quote">On Fri, May 24, 2013 at 11:05 AM, Asif ud-Doula <span dir="ltr"><<a href="mailto:auu4@psu.edu" target="_blank">auu4@psu.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Everyone,<br>
I am having some trouble running the latest version of Flash on linux cluster with OpenMPI 1.6.1 version along with Intel (composer_xe_2011_sp1.8.27). There seems to be some memory allocation issues. Note that it compiles just fine.<br>
<br>
<br>
I have tried the standard ./setup Sedov -auto -2d and tried to run it with e.g. 4 processes with flash.par virtually unchanged. Here is the error I receive when I run it in an interactive mode:<br>
<br>
RuntimeParameters_read: ignoring unknown parameter "order"...<br>
RuntimeParameters_read: ignoring unknown parameter "slopeLimiter"...<br>
RuntimeParameters_read: ignoring unknown parameter "LimitedSlopeBeta"...<br>
RuntimeParameters_read: ignoring unknown parameter "use_avisc"...<br>
RuntimeParameters_read: ignoring unknown parameter "use_flattening"...<br>
RuntimeParameters_read: ignoring unknown parameter "use_upwindTVD"...<br>
RuntimeParameters_read: ignoring unknown parameter "RiemannSolver"...<br>
RuntimeParameters_read: ignoring unknown parameter "entropy"...<br>
RuntimeParameters_read: ignoring unknown parameter "shockDetect"...<br>
<br>
flash4:27361 terminated with signal 11 at PC=2b66b15f539a SP=7fff34566480. Backtrace:<br>
<br>
flash4:27362 terminated with signal 11 at PC=2b506c4cd39a SP=7fffd1113480. Backtrace:<br>
<br>
flash4:27363 terminated with signal 11 at PC=2b812071539a SP=7fffd2f31780. Backtrace:<br>
/opt/shared/openmpi/1.6.1-<u></u>intel64/lib/libmpi.so.1(opal_<u></u>memory_ptmalloc2_int_malloc+<u></u>0x2aa)[0x2b506c4cd39a]<br>
/opt/shared/openmpi/1.6.1-<u></u>intel64/lib/libmpi.so.1(opal_<u></u>memory_ptmalloc2_int_malloc+<u></u>0x2aa)[0x2b812071539a]<br>
/opt/shared/openmpi/1.6.1-<u></u>intel64/lib/libmpi.so.1(opal_<u></u>memory_ptmalloc2_malloc+0x59)[<u></u>0x2b506c4d0659]<br>
/opt/shared/openmpi/1.6.1-<u></u>intel64/lib/libmpi.so.1(opal_<u></u>memory_ptmalloc2_malloc+0x59)[<u></u>0x2b8120718659]<br>
/opt/shared/openmpi/1.6.1-<u></u>intel64/lib/libmpi.so.1(+<u></u>0x110346)[0x2b8120718346]<br>
/opt/shared/openmpi/1.6.1-<u></u>intel64/lib/libmpi.so.1(+<u></u>0x110346)[0x2b506c4d0346]<br>
/home/1232/FLASH4/object/<u></u>flash4(for_allocate+0x164)[<u></u>0x6a4544]<br>
/home/1232/FLASH4/object/<u></u>flash4(for_allocate+0x164)[<u></u>0x6a4544]<br>
/home/1232/FLASH4/object/<u></u>flash4(namevaluell_data_mp_<u></u>namevaluell_addreal_+0x35c)[<u></u>0x632bbc]<br>
/home/1232/FLASH4/object/<u></u>flash4(namevaluell_data_mp_<u></u>namevaluell_addreal_+0x35c)[<u></u>0x632bbc]<br>
/home/1232/FLASH4/object/<u></u>flash4(namevaluell_bcast_+<u></u>0x1e6f)[0x631f5f]<br>
/home/1232/FLASH4/object/<u></u>flash4(namevaluell_bcast_+<u></u>0x1e6f)[0x631f5f]<br>
/home/1232/FLASH4/object/<u></u>flash4(runtimeparameters_<u></u>bcast_+0x12)[0x46edb2]<br>
/home/1232/FLASH4/object/<u></u>flash4(runtimeparameters_<u></u>bcast_+0x12)[0x46edb2]<br>
/home/1232/FLASH4/object/<u></u>flash4(runtimeparameters_init_<u></u>+0x265)[0x46f3c5]<br>
/home/1232/FLASH4/object/<u></u>flash4(runtimeparameters_init_<u></u>+0x265)[0x46f3c5]<br>
/home/1232/FLASH4/object/<u></u>flash4(driver_initflash_+0x35)<u></u>[0x430245]<br>
/home/1232/FLASH4/object/<u></u>flash4(driver_initflash_+0x35)<u></u>[0x430245]<br>
/home/1232/FLASH4/object/<u></u>flash4(MAIN__+0x42)[0x435802]<br>
/home/1232/FLASH4/object/<u></u>flash4(MAIN__+0x42)[0x435802]<br>
/home/1232/FLASH4/object/<u></u>flash4(main+0x3c)[0x42b1dc]<br>
/home/1232/FLASH4/object/<u></u>flash4(main+0x3c)[0x42b1dc]<br>
/lib64/libc.so.6(__libc_start_<u></u>main+0xfd)[0x2b506db26cdd]<br>
/lib64/libc.so.6(__libc_start_<u></u>main+0xfd)[0x2b8121d6ecdd]<br>
/home/1232/FLASH4/object/<u></u>flash4[0x42b0d9]<br>
/home/1232/FLASH4/object/<u></u>flash4[0x42b0d9]<br>
/opt/shared/openmpi/1.6.1-<u></u>intel64/lib/libmpi.so.1(opal_<u></u>memory_ptmalloc2_int_malloc+<u></u>0x2aa)[0x2b66b15f539a]<br>
/opt/shared/openmpi/1.6.1-<u></u>intel64/lib/libmpi.so.1(opal_<u></u>memory_ptmalloc2_malloc+0x59)[<u></u>0x2b66b15f8659]<br>
/opt/shared/openmpi/1.6.1-<u></u>intel64/lib/libmpi.so.1(+<u></u>0x110346)[0x2b66b15f8346]<br>
/home/1232/FLASH4/object/<u></u>flash4(for_allocate+0x164)[<u></u>0x6a4544]<br>
/home/1232/FLASH4/object/<u></u>flash4(namevaluell_data_mp_<u></u>namevaluell_addreal_+0x35c)[<u></u>0x632bbc]<br>
/home/1232/FLASH4/object/<u></u>flash4(namevaluell_bcast_+<u></u>0x1e6f)[0x631f5f]<br>
/home/1232/FLASH4/object/<u></u>flash4(runtimeparameters_<u></u>bcast_+0x12)[0x46edb2]<br>
/home/1232/FLASH4/object/<u></u>flash4(runtimeparameters_init_<u></u>+0x265)[0x46f3c5]<br>
/home/1232/FLASH4/object/<u></u>flash4(driver_initflash_+0x35)<u></u>[0x430245]<br>
/home/1232/FLASH4/object/<u></u>flash4(MAIN__+0x42)[0x435802]<br>
/home/1232/FLASH4/object/<u></u>flash4(main+0x3c)[0x42b1dc]<br>
/lib64/libc.so.6(__libc_start_<u></u>main+0xfd)[0x2b66b2c4ecdd]<br>
/home/1232/FLASH4/object/<u></u>flash4[0x42b0d9]<br>
<br>
<br>
Just to provide you with more information about the system:<br>
* uname -a<br>
Linux n106 2.6.32-279.19.1.el6.x86_64 #1 SMP Tue Dec 18 17:22:54 CST 2012 x86_64 x86_64 x86_64 GNU/Linux<br>
<br>
* ldd flash4<br>
linux-vdso.so.1 => (0x00007fff8d5d0000)<br>
libhdf5.so.7 => /opt/shared/hdf5/1.8.8-<u></u>intel64/lib/libhdf5.so.7 (0x00002ab4a4f58000)<br>
libmpi.so.1 => /opt/shared/openmpi/1.6.1-<u></u>intel64/lib/libmpi.so.1 (0x00002ab4a54f8000)<br>
libm.so.6 => /lib64/libm.so.6 (0x00002ab4a5908000)<br>
libpthread.so.0 => /lib64/libpthread.so.0 (0x00002ab4a5b90000)<br>
libz.so.1 => /lib64/libz.so.1 (0x00002ab4a5db0000)<br>
libmpi_f90.so.1 => /opt/shared/openmpi/1.6.1-<u></u>intel64/lib/libmpi_f90.so.1 (0x00002ab4a5fc8000)<br>
libmpi_f77.so.1 => /opt/shared/openmpi/1.6.1-<u></u>intel64/lib/libmpi_f77.so.1 (0x00002ab4a61d0000)<br>
libdl.so.2 => /lib64/libdl.so.2 (0x00002ab4a6408000)<br>
librt.so.1 => /lib64/librt.so.1 (0x00002ab4a6610000)<br>
libnsl.so.1 => /lib64/libnsl.so.1 (0x00002ab4a6818000)<br>
libutil.so.1 => /lib64/libutil.so.1 (0x00002ab4a6a38000)<br>
libc.so.6 => /lib64/libc.so.6 (0x00002ab4a6c40000)<br>
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002ab4a6fd8000)<br>
libimf.so => /home/software/intel/composer_<u></u>xe_2011_sp1.8.273/compiler/<u></u>lib/intel64/libimf.so (0x00002ab4a71f0000)<br>
libsvml.so => /home/software/intel/composer_<u></u>xe_2011_sp1.8.273/compiler/<u></u>lib/intel64/libsvml.so (0x00002ab4a75c0000)<br>
libintlc.so.5 => /home/software/intel/composer_<u></u>xe_2011_sp1.8.273/compiler/<u></u>lib/intel64/libintlc.so.5 (0x00002ab4a7d38000)<br>
/lib64/ld-linux-x86-64.so.2 (0x00002ab4a4d30000)<br>
libifport.so.5 => /home/software/intel/composer_<u></u>xe_2011_sp1.8.273/compiler/<u></u>lib/intel64/libifport.so.5 (0x00002ab4a7e88000)<br>
libifcore.so.5 => /home/software/intel/composer_<u></u>xe_2011_sp1.8.273/compiler/<u></u>lib/intel64/libifcore.so.5 (0x00002ab4a7fc0000)<br>
libifcoremt.so.5 => /home/software/intel/composer_<u></u>xe_2011_sp1.8.273/compiler/<u></u>lib/intel64/libifcoremt.so.5 (0x00002ab4a8208000)<br>
<br>
<br>
I wonder if you have encountered a similar issue, or perhaps have a suggestion for me. Thank you in advance for your time and help,<br>
<br>
Asif<span class="HOEnZb"><font color="#888888"><br>
<br>
<br>
-- <br>
______________________________<u></u>__<br>
Asif ud-Doula<br>
Assistant Professor of Physics<br>
Penn State Worthington Scranton<br>
120 Ridge View Drive<br>
Dunmore, PA 18512<br>
USA<br>
tel. <a href="tel:%2B1-570-963-2582" value="+15709632582" target="_blank">+1-570-963-2582</a><br>
______________________________<u></u>__<br>
</font></span></blockquote></div><br>