[FLASH-USERS] Regarding slow computation on multi-node

[McFarland, Jacob A.]

Mohammad,

It might be helpful to send some information on the computer system you are using. What kind of mpi command are you using to run flash and what kind of job scheduler is your computer using. Maybe you could copy your run command into the email as well.

Thanks,
Jacob

From: flash-users-bounces at flash.uchicago.edu <flash-users-bounces at flash.uchicago.edu> On Behalf Of Mohaghar, Mohammad
Sent: Friday, September 27, 2019 5:29 PM
To: flash-users at flash.uchicago.edu
Subject: [FLASH-USERS] Regarding slow computation on multi-node


Dear FLASH users,

I figured out that the 3 dimensional computation on 1 node with N processors runs much faster than >1 node where each node has N processors. Hence, the computational time drops on one node as I increase the number of processors (The maximum available processors for me at the moment is 24), however, as soon as I run on multi-node processors (even if I double or triple the number of processors) the computational time increases. The computational time can increase even to 8 times the computational time for one node (so it's way too slow). Do you have any idea why this happens? I understand that it takes more time for communication between the nodes, but never thought it'd be this slow. With this problem, there is no point point for me to run on multi-node since it takes way too long time for the 3D simulation to run. On one node, I can run the 3D simulation with lower resolution, but if I increase the resolution I get an "out of memory" error.

I'd appreciate any thoughts that you may have to help me out.

Thank you.

Best,
Mohammad

----------------------------------------------------------------------
----------------------------------------------------------------------
Mohammad Mohaghar, Ph.D.
Postdoctoral Fellow
School of Civil and Environmental Engineering
Georgia Institute of Technology
Phone: (678) 900-5453


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20190927/2191a99b/attachment-0001.htm>