<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div>Hello,</div><div><br></div><div>I am trying to use paramesh on a custom simulation using particles. </div><div><br></div><div>The codes run well with this setup:</div><div><div><div style="margin: 0px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"> </span>setup.py PlasmaObstacleIonosphere -auto -3d +ug +parallelIO -nxb=18 -nyb=16 -nzb=16</div></div></div><div>combined with </div><div><div style="margin: 0px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space: pre;"> </span>iProcs = 2 </div><div style="margin: 0px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"> </span>jProcs = 4</div><div style="margin: 0px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"> </span>kProcs = 4</div></div><div><br></div><div><br></div><div>To use paramesh, the setup command is :</div><div><div style="margin: 0px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space: pre;"> </span>setup.py PlasmaObstacleIonosphere -auto -3d +pm4dev +parallelIO -nxb=18 -nyb=16 -nzb=16 -maxblocks=512</div></div><div style="margin: 0px; font-family: Menlo;"><br></div><div style="margin: 0px;">The relevant running time parameters are, in the flash.par file:</div><div style="margin: 0px;"><br></div><div style="margin: 0px;"><div style="margin: 0px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"> </span>iProcs = 2 </div><div style="margin: 0px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"> </span>jProcs = 4</div><div style="margin: 0px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"> </span>kProcs = 4</div><div style="margin: 0px; font-family: Menlo; min-height: 16px;"><br></div><div style="margin: 0px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"> </span>lrefine_max = 3 </div><div style="margin: 0px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"> </span>lrefine_min = 1 </div><div style="margin: 0px; font-family: Menlo; min-height: 16px;"><br></div><div style="margin: 0px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"> </span>refine_var_1 = "cde2"</div><div style="margin: 0px; font-family: Menlo; min-height: 16px;"><br></div><div style="margin: 0px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"> </span>nblockx = 2 </div><div style="margin: 0px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"> </span>nblocky = 4 </div><div style="margin: 0px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"> </span>nblockz = 4 </div><div style="margin: 0px; font-family: Menlo; min-height: 16px;"><br></div><div style="margin: 0px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"> </span>nrefs = 4 </div><div style="margin: 0px; font-family: Menlo;"><br></div><div style="margin: 0px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"> </span>refine_on_particle_count = .false.<span class="Apple-tab-span" style="white-space:pre"> </span># The ultimate goal is to refine grids by the number of particles per cell. But refinement by field value also gives the error.</div><div style="margin: 0px; font-family: Menlo; min-height: 16px;"><span class="Apple-tab-span" style="white-space:pre"> </span>max_particles_per_blk = 100000</div><div style="margin: 0px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"> </span>min_particles_per_blk = 10<span class="Apple-tab-span" style="white-space:pre"> </span># Now these 3 lines should not matter since refine_on_particle_count = .false., refinement should only depend on “cde2"</div><div><br></div><div>However, error always appears after the first step:</div><div><br></div><div><div style="margin: 0px; font-family: Menlo;">==============================================================================</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:07:21.180 ] [gr_initGeometry] checking BCs for idir: 1</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:07:21.182 ] [gr_initGeometry] checking BCs for idir: 2</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:07:21.183 ] [gr_initGeometry] checking BCs for idir: 3</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:07:51.760 ] [GRID amr_refine_derefine]: initiating refinement</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:07:51.761 ] [GRID amr_refine_derefine]: redist. phase. tot blks requested: 32</div><div style="margin: 0px; font-family: Menlo;"> [GRID amr_refine_derefine] min blks 1 max blks 1 tot blks 32</div><div style="margin: 0px; font-family: Menlo;"> [GRID amr_refine_derefine] min leaf blks 1 max leaf blks 1 tot leaf blks 32</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:07:52.524 ] [GRID amr_refine_derefine]: refinement complete</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:07:52.664 ] [GRID gr_expandDomain]: iteration=1, create level=2</div><div style="margin: 0px; font-family: Menlo;"> INFO: Grid_fillGuardCells is ignoring masking.</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:07:52.685 ] [mpi_amr_comm_setup]: buffer_dim_send=711877, buffer_dim_recv=711877</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:07:53.736 ] [GRID gr_expandDomain]: iteration=2, create level=3</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:07:54.408 ] [GRID gr_expandDomain]: iteration=3, create level=3</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:07:55.239 ] [GRID gr_expandDomain]: iteration=4, create level=3</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:07:55.472 ] [GRID gr_expandDomain]: iteration=5, create level=3</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:07:55.491 ] memory: /proc vsize (MB): 4974.98 (min) 4975.39 (max) 4975.15 (avg)</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:07:55.492 ] memory: /proc rss (MB): 3307.12 (min) 3307.66 (max) 3307.46 (avg)</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:07:55.493 ] memory: /proc vsize (MB): 4974.98 (min) 4975.39 (max) 4975.15 (avg)</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:07:55.494 ] memory: /proc rss (MB): 3307.36 (min) 3307.71 (max) 3307.50 (avg)</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:07:56.261 ] memory: /proc vsize (MB): 4974.98 (min) 4975.39 (max) 4975.15 (avg)</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:07:56.263 ] memory: /proc rss (MB): 3310.65 (min) 3312.32 (max) 3310.90 (avg)</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:07:56.273 ] [Particles_getGlobalNum]: Number of particles now: 442368</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:07:56.396 ] [IO_writePlotfile] open: type=plotfile name=flash_hdf5_plt_cnt_0000</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:07:58.051 ] [io_writeData]: wrote 32 blocks</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:07:58.096 ] [IO_writePlotfile] close: type=plotfile name=flash_hdf5_plt_cnt_0000</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:07:58.160 ] [IO_writeParticles] open: type=particles name=flash_hdf5_part_0000</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:07:58.161 ] [IO_writeParticles]: done called Particles_updateAttributes()</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:08:00.940 ] [IO_writeParticles] close: type=particles name=flash_hdf5_part_0000</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:08:00.955 ] memory: /proc vsize (MB): 4977.71 (min) 4978.27 (max) 4977.82 (avg)</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:08:00.956 ] memory: /proc rss (MB): 3312.65 (min) 3314.48 (max) 3312.93 (avg)</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:08:00.957 ] [Driver_evolveFlash]: Entering evolution loop</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:08:00.957 ] step: n=1 t=0.000000E+00 dt=2.500000E-02</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:08:02.667 ] [DRIVER_ABORT]: Driver_abort() called by PE 0</div><div style="margin: 0px; font-family: Menlo;"> [ 05-02-2016 16:08:02.668 ] abort_message: src == gr_meshMe, and we still have unmatched neData, bad</div></div><div><br></div><div>The final abort message comes from </div><div><flash home>/source/Grid/GridParticles/GridParticlesMove/Sieve/BlockMatch/gr_ptNextProcPair.F90</div></div><div>But except for that no further information is available.</div><div><br></div><div>I have tried different values for lrefine_min/_max, nblock[xyz], n[xyz]b, as well as +pm40 instead of +pm4dev for setup, but this problem persists.</div><div><br></div><div>Could anyone give a hint on the problem? Thank you very much!</div><br><div apple-content-edited="true">
<span class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px;"><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>Best regards,</div><div>Xiao-Dong Wang<br><br></div><div>Swedish Institute of Space Physics (IRF)</div><div>Box 812, SE-981 28 Kiruna, Sweden<br>E-mail: <a href="mailto:wang@irf.se">wang@irf.se</a><br>Phone: +46 980 79008<br></div></div></span></div></span></span>
</div>
<br></body></html>