[FLASH-USERS] Fwd: how to use parallel VisIt on Ubuntu?

Fri Sep 23 00:45:37 EDT 2016

Thank you dear Scott Feister.

I use my PARAMESH program, run two cases to do output. one with one processor, the other with four processors.
The HDF5 files from these two cases have exactly the same file size.

Then I use VisIt (I think I install it just as you mentioned: serial version 2.10.2,  run install script, and start "visit" command) to visual these two files.
The one from one processor is opened and the display is nice. As to the one from four processors, I can open it, but visibly only a small part of meshs are displayed.
I am really confused.

As to the mpi and HDF5 libraries, I test the below commands and am told that the computer has already have these installed.
# MPI, HDF5 (parallel)
sudo apt-get install mpi-default-dev
sudo apt-get install openmpi-bin
sudo apt-get install libhdf5-openmpi-dev # NOTE: I originally used "libhdf5-dev", but changed later to libhdf5-openmpi-dev for parallel I/O.   

simondonxq

From: Scott Feister
Date: 2016-09-22 12:05
To: Simon Don
CC: flash-users
Subject: Re: [FLASH-USERS] Fwd: how to use parallel VisIt on Ubuntu?
Hi Simon,

If you're still stuck, I may be able to help a bit. In the past, I have successfully used the serial version of VisIt in Ubuntu 14.04.3 to read HDF5 files created by FLASH's parallel I/O. I don't think the parallel VisIt installation is necessary.

One idea is that maybe it's not VisIt, but it's your HDF5 file. E.g., maybe the outputs are getting corrupted when you run in parallel. Were the HDF5 files made on Ubuntu, too?

Last year, I did a full setup on FLASH and VisIt in Ubuntu. I organized some notes for you in case they are of help. Especially relevant may be the choice of hdf5 libraries: I used sudo apt-get install libhdf5-openmpi-dev (parallel hdf5 library) rather than libhdf5-dev (serial hdf5 library).

Scott's notes on installing FLASH 4.3 in Ubuntu 14.04.3

My installation followed the FLASH documentation.

1. Installed FLASH dependencies, including optional parallel HDF5, using the package manager.

# Going down the list at: http://flash.uchicago.edu/site/flashcode/user_support/flash4_ug_4p3/node5.html
sudo apt-get update
sudo apt-get upgrade

# GCC and GNU Make
sudo apt-get install build-essential

# Fortran90
sudo apt-get install gfortran

# MPI, HDF5 (parallel)
sudo apt-get install mpi-default-dev
sudo apt-get install openmpi-bin
sudo apt-get install libhdf5-openmpi-dev # NOTE: I originally used "libhdf5-dev", but changed later to libhdf5-openmpi-dev for parallel I/O.

2. Installed HYPRE dependency from source, into its own folder.

# Download HYPRE 2.8 source code (2.10 does not work with FLASH 4.3), and unzip.
tar -xvf hypre-2.8.0b.tar.gz
# Navigate into the unzipped folder, and then make the library from source:
cd src
./configure
make
make install

3. Download and unzip FLASH 4.3, and give your computer a Makefile.h in the "sites" directory.

tar -xvf FLASH4.3.tar.gz
cd FLASH4.3/sites
mkdir virtuoso # Make a new folder named after your computer. Here, my computer's name is "virtuoso".
cp Prototypes/Linux/Makefile.h virtuoso/Makefile.h # Copy the Linux makefile prototype
# Now, change the Makefile.h to point to appropriate locations.
nano virtuoso/Makefile.h

The beginning few lines of the copied Makefile.h should be modified to be similar to what's shown below.
Each of those folders (no ending slash) must contain the subfolders called “lib” and/or “bin”.
The HYPRE_PATH is where you just installed it.
Find path of MPI_PATH by examining the output of: “type mpicc”
Find path of HDF5_PATH by examining the output of: “echo $HDF5_C_LIBS”

After modification, first few lines of …/sites/virtuoso/Makefile.h look like:

MPI_PATH   = /usr
HDF4_PATH  =
HDF5_PATH  = /usr
HYPRE_PATH = /home/joeblow/hypre-2.8.0b/src/hypre

ZLIB_PATH  =
PAPI_PATH  =
PAPI_FLAGS =
NCMPI_PATH =
MPE_PATH   =

4. Test the installation with WindTunnel (does not use Hypre) or LaserSlab (does use Hypre)

./setup WindTunnel -auto
# Ends with “SUCCESS”
cd object
make -j 4 (runs on all four cores)
# Ends with “SUCCESS”
# Before running the simulation, edit flash.par to include some plotting variables (better test I/O); e.g. add the line:
plot_var_1 = “dens”

# Run the simulation and create outputs

mpirun -np 4 flash4

Note: To test LaserSlab, you could start instead with something like:

./setup -auto LaserSlab -2d +uhd3t +pm4dev +hdf5typeio +mgd mgd_meshgroups=30 -geometry=cylindrical -nxb=10 -nyb=10 +mtmmmt -maxblocks=8192 -without-unit=Particles species=cham,targ

Scott's notes on installing VisIt 2.9.2 in Ubuntu 14.04.3
1. Downloaded the VisIt 2.9.2 tarball for Ubuntu 14.04 64-bit Linux: “visit2_9_2.linux-x86_64-ubuntu14.tar.gz” from: https://wci.llnl.gov/simulation/computer-codes/visit/executables
Downloaded and read the “VisIt install notes” for VisIt 2.9.2 at the same website
Downloaded the “VisIt install script” for VisIt 2.9.2 from the same website, named it “visit-install” and put it in the same folder as the tarball
2. Made the install script executable: “sudo chmod +x visit-install”
3. Ran the install script:

“sudo ./visit-install 2.9.2 linux-x86_64-ubuntu14 /usr/local/visit”
Note that the syntax is “sudo ./visit-install version platform directory”
When prompted, selected “No affiliation” or “no site” (choice 1)

4. Added the newly-created binaries folder to my user path:

Edited (home)/.bashrc and added the following line at the bottom:
“export PATH=$PATH:/usr/local/visit/bin”
Then, restarted terminal
5. Can now open VisIt by typing “visit” at the terminal window

Best,

Scott

Scott Feister, Ph.D.
Postdoctoral Researcher, Flash Center for Computational Science

University of Chicago, Department of Astronomy and Astrophysics

On Wed, Sep 21, 2016 at 9:29 AM, Simon Don <simondonxq at gmail.com> wrote:

Thank you very much. In fact my code is now just using PARAMESH amr package and the HDF5 files is generated by papamesh 's parallel HDF5 I/O. We have not finished the shift from paramesh to flash.

Now I want to open these hdf5 files by visit. But I can only do this when the hdf5 file is written by only one processor. When the  MPI is used to write the hdf5 file, my present visit on ubuntu can not open it rightly.

That the situation and why I am asking parallel visit things.

Regards.

2016年9月21日星期三，Scott Feister <sfeister at gmail.com> 写道：

Hi Simon,

Just to clarify, are you having trouble when opening an HDF5 made by using FLASH's parallel HDF5 I/O, or trying to run a parallel installation VisIt itself? Or both?

Best,

Scott

Scott Feister, Ph.D.
Postdoctoral Researcher, Flash Center for Computational Science

University of Chicago, Department of Astronomy and Astrophysics

On Wed, Sep 21, 2016 at 8:25 AM, Simon Don <simondonxq at gmail.com> wrote:

Dear all:

I now get some result data of AMR. It is written by Fujitsu MPI in hdf5 format. I want to use VisIt to visualize it. 

I downloaded the built executables from the visit homepage and installed it on my ubuntu computer. When I use it to open hdf5 generated by one processor, it works well. We I open file generated by MPI, the display is not right or sometimes I can't open the file.  

So my question is, now the executables of VisIt for ubuntu are without mpi and can't support MPI? In order to get a parallel VisIt on ubuntu, I need to install from the source code and build it by myself with the certain MPI library options?

As I said, the MPI used for calculations is Fujitsu MPI, so I have to build a parallel VisIt with Fujitsu MPi to open such a data file? Can i build a parallel visit using other mpi like openmpi and open a data file generated by Fujitsu mpi?

Anyone who is familiar with this? Thank you very much!!

Shimon 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20160923/e24642f0/attachment.htm>