[FLASH-USERS] [EXT] Trying to use viscosity
Reyes, Adam
adam.reyes at rochester.edu
Sun Oct 8 07:04:48 EDT 2023
Hi Max,
As far as I know there is no known bug with the HYPRE solvers. That section is referring to the nature of Krylov subspace methods giving different results depending on the number of mpi ranks used to find the solution. Both solutions would be correct up to some specified error tolerance, just slightly different from each other.
*********************************************
Adam Reyes

Code Group Leader, Flash Center for Computational Science
Research Scientist, Dept. of Physics and Astronomy
University of Rochester
River Campus: Bausch and Lomb Hall, 369
500 Wilson Blvd. PO Box 270171, Rochester, NY 14627
Email adam.reyes at rochester.edu
Web https://flash.rochester.edu
(he / him / his)

*********************************************
> On Oct 8, 2023, at 7:19 AM, Maximus Allfree <mall7710 at uni.sydney.edu.au> wrote:
>
> Thank you both for your help, and the paper. I really appreciate it.
>
> I have tried to implement the unsplit hydro and unsplit diffusion as suggested. However I keep running into issues with adding the unsplit diffusion requirement as it needs HYPRE which clashed with MPICH. I have tried to fix this by installing Open-MPI and HYPRE instead of MPICH. I then had to change the default darwin makefile to add the HYPRE paths and the HYPRE include directory for FFLAGS and CFLAGS as it couldn't find a bunch of files:
> MPI_PATH = /opt/homebrew/Cellar/open-mpi/4.1.5
> FCOMP = $(MPI_PATH)/bin/mpifort
> CCOMP = $(MPI_PATH)/bin/mpicc
> CPPCOMP = $(MPI_PATH)/bin/mpiCC
> LINK = $(MPI_PATH)/bin/mpifort
> LIB_MPI =
>
> HYPRE_PATH = /opt/homebrew/Cellar/hypre/2.29.0
> FFLAGS_HYPRE = -I${HYPRE_PATH}/include
> CFLAGS_HYPRE = -I${HYPRE_PATH}/include
> LIB_HYPRE = -L${HYPRE_PATH}/lib -lHYPRE
>
>
> Currently, my config file has this:
> REQUIRES Driver
> REQUIRES physics/Hydro/HydroMain/unsplit
> REQUIRES physics/Diffuse/DiffuseMain/Unsplit
> REQUIRES Multispecies/MultispeciesMain
> REQUIRES physics/Eos/EosMain/Multigamma
> # REQUIRES Grid/GridSolvers/HYPRE
> REQUIRES physics/materialProperties/Conductivity
> REQUIRES physics/materialProperties/Viscosity
>
>
>
> My flash.par file has this:
> #--------------------------------------------------------------------
> # Diffusive parameters:
> # useDiffuse flags whether the Diffuse unit is being used at all
> useDiffuse = .true.
> dt_diff_factor = 0.8
>
> #--------------------------------------------------------------------
> # Viscous parameters:
> # useViscosity flags whether the viscosity material property is being used
> useViscosity = .true.
> diff_visc_mu = 0.1560365234
> visc_whichCoefficientIsConst = 1
>
> #--------------------------------------------------------------------
> # Conductive parameters:
> useConductivity = .true.
> diff_constant = 1.0
>
>
> With these parameters the code:
> > generates the makefile successfully,
> > runs the makefile process successfully (I had to use this flag to downgrade the implicit errors to warnings):
> -Wno-implicit-function-declaration
>
> But it then has an issue before it starts the simulation. Any clue what this error may mean? When I run without the diffuse/unsplit line in config it works fine, so seems to be a problem with HYPRE and how I have set it up but I could be wrong.
> Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
>
> Backtrace for this error:
>
> Could not print backtrace: executable file is not an executable
>
> Also the FLASH documentation 8.10.4.1 claims there is a bug with HYPRE and using multiple procs when using HYPRE based solvers, should I be concerned about this or is that fine?
>
> Again, really stuck here if anyone has any advice I would love to hear it. I have been through the documentation all day and haven't been able to fix this, but I don't really understand the FLASH architecture very well so I imagine there is some oversight here.
>
>
> Kind regards,
> Max
>
>
> From: Hansen, Eddie <ehansen at pas.rochester.edu>
> Sent: Saturday, 7 October 2023 10:29 PM
> To: Reyes, Adam <adam.reyes at rochester.edu>; Maximus Allfree <mall7710 at uni.sydney.edu.au>
> Cc: flash-users at flash.rochester.edu <flash-users at flash.rochester.edu>
> Subject: Re: [FLASH-USERS] Trying to use viscosity
>
> Max,
>
> I would also add that you probably want this line in your Config if you need heat conduction:
> REQUESTS physics/Diffuse/DiffuseMain/Unsplit
>
> This will let you use the unsplit implicit diffusion solvers, one of which is for thermal conduction. See user guide Sec. 18.1.4 on how to use isotropic thermal diffusion. Also check out Sec. 22.1 for different conductivity coefficient implementations.
> --
> Eddie Hansen
> Applications Group Leader
> Flash Center for Computational Science
>
>
> From: flash-users <flash-users-bounces at flash.rochester.edu> on behalf of Reyes, Adam <adam.reyes at rochester.edu>
> Date: Saturday, October 7, 2023 at 6:58 AM
> To: Maximus Allfree <mall7710 at uni.sydney.edu.au>
> Cc: flash-users at flash.rochester.edu <flash-users at flash.rochester.edu>
> Subject: Re: [FLASH-USERS] Trying to use viscosity
>
> Hi Max,
>
> Yes you’re right those should be commas.
>
> I would recommend against using those combinations of solvers. The unspilt hydro solver, which has its' own implementation of flux-based diffusion solvers is in general better for multidimensional flows. It is also better maintained than the split solvers. You could have in you Config this instead:
>
> REQUIRES Driver
> REQUIRES physics/Hydro/HydroMain/unsplit
> REQUIRES Multispecies/MultispeciesMain
> REQUIRES physics/Eos/EosMain/Multigamma
>
> this <https://urldefense.com/v3/__https://protect-au.mimecast.com/s/iAD9CBNqjlCDXgAOJSz3klR?domain=doi.org__;!!CGUSO5OYRnA7CQ!dmar34EhH_oAGyarZTmZYq4ajKmNTovcSY-Cy1IuxHrKXVVGWTZ7YcquZP9PKRiyptgTUy5S3BMZbAoyUZrNsVeas1xZ5qo$> paper on the CASTRO code has some pretty striking figures showing the advantages of using an unspilt hydro solver.
>
> You shouldn’t need to add those parameters to your own simulation Config. They exist elsewhere in the source code and make them available to be set from you flash.par at runtime.
>
> Thanks for reporting the bug!
>
>
> *********************************************
> Adam Reyes
>
> <FLASH.jpg>
> Code Group Leader, Flash Center for Computational Science
> Research Scientist, Dept. of Physics and Astronomy
> University of Rochester
> River Campus: Bausch and Lomb Hall, 369
> 500 Wilson Blvd. PO Box 270171, Rochester, NY 14627
> Email adam.reyes at rochester.edu
> Web https://flash.rochester.edu <https://urldefense.com/v3/__https://protect-au.mimecast.com/s/CMSDCD1vlpToLRMkrF5pP8X?domain=flash.rochester.edu__;!!CGUSO5OYRnA7CQ!dmar34EhH_oAGyarZTmZYq4ajKmNTovcSY-Cy1IuxHrKXVVGWTZ7YcquZP9PKRiyptgTUy5S3BMZbAoyUZrNsVeaC6AGPME$>
> (he / him / his)
> <FLASH-pride-sml.png>
>
> *********************************************
>
>
>
>
> On Oct 7, 2023, at 8:21 AM, Maximus Allfree <mall7710 at uni.sydney.edu.au> wrote:
>
> Hey FLASH Users,
>
>
> I am having some problems with diffusion and viscosity. I am trying to run RMI simulations and have required these in the config file:
>
> REQUIRES Driver
> REQUIRES physics/Hydro/HydroMain/split/PPM
> REQUIRES Multispecies/MultispeciesMain
> REQUIRES physics/Eos/EosMain/Multigamma
> REQUIRES physics/Diffuse/DiffuseFluxBased
>
> In the flash user documentation it mentioned that HYDRO PPM was needed for diffusive effects, so I am using that. Before I was just using physics/Hydro. I also am using DiffuseFluxBased so I can use heat conduction and viscosity.
>
> I have the following parameters in the config file to set the required runtime variables. I have also set these in the flash.par file, but they don't seem to do anything? They aren't overriding the config file default values. Not sure if I have done something wrong here, or if they're not meant to be used in the flash.par file:
>
> # Diffusive parameters:
> D useDiffuse flags whether the Diffuse unit is being used at all
> PARAMETER useDiffuse BOOLEAN true
>
> D useDiffuseTherm flags whether Diffuse_therm should contribute to fluxes
> PARAMETER useDiffuseTherm BOOLEAN true
>
> D useDiffuseVisc flags whether Diffuse_visc should contribute to fluxes
> PARAMETER useDiffuseVisc BOOLEAN true
> D useViscosity flags whether the viscosity material property is being used
> PARAMETER useViscosity BOOLEAN true
>
> D diff_visc_mu constant dynamic viscosity
> PARAMETER diff_visc_mu REAL 0.1 [ to ]
>
> D diff_visc_nu constant kinematic viscosity
> PARAMETER diff_visc_nu REAL 0.1 [ to ]
>
> D visc_whichCoefficientIsConst which is constant: (1 for dynamic, 2 for kinematic)
> PARAMETER visc_whichCoefficientIsConst INTEGER 1 [1 to 2]
>
>
> Anyway, when I try to make the code, I am getting this error:
>
> /opt/homebrew/Cellar/mpich/4.1.2/bin/mpif90 -ggdb -c -O2 -fdefault-real-8 -fdefault-double-8 -Wuninitialized -fallow-argument-mismatch -ffree-line-length-none -DMAXBLOCKS=1 -DNXB=45 -DNYB=32 -DNZB=1 -DN_DIM=2 Diffuse_visc.F90
> Diffuse_visc.F90:225:59:
>
> 225 | call Viscosity(U(:,i,j,k),viscDynamic(i,j,k).viscUnusedHere(i,j,k), component=1)
> | 1
> Error: Syntax error in argument list at (1)
> Diffuse_visc.F90:327:52:
>
> 327 | call Viscosity(U(:,i,j,k),viscDynamic(i,j,k).viscUnusedHere(i,j,k), component=1)
> | 1
> Error: Syntax error in argument list at (1)
> Diffuse_visc.F90:411:52:
>
> 411 | call Viscosity(U(:,i,j,k),viscDynamic(i,j,k).viscUnusedHere(i,j,k), component=1)
> | 1
> Error: Syntax error in argument list at (1)
> make: *** [Diffuse_visc.o] Error 1
>
> I looked into the Diffuse_visc.F90 file, and I was wondering whether the full stops are meant to be commas? It feels unlikely that this would be a FLASH error and far more likely I have done something wrong with implementing diffusion / viscosity as I am new to FLASH but I figured I would ask if anyone else has had problems with this.
>
> Cheers for reading. Any advice would be greatly appreciated.
>
>
> Kind regards,
> Max
>
>
> _______________________________________________
> flash-users mailing list
> flash-users at flash.rochester.edu <mailto:flash-users at flash.rochester.edu>
>
> For list info, including unsubscribe:
> https://flash.rochester.edu/mailman/listinfo/flash-users <https://urldefense.com/v3/__https://protect-au.mimecast.com/s/JnnWCE8wmrtl8QnNosphxJE?domain=flash.rochester.edu__;!!CGUSO5OYRnA7CQ!dmar34EhH_oAGyarZTmZYq4ajKmNTovcSY-Cy1IuxHrKXVVGWTZ7YcquZP9PKRiyptgTUy5S3BMZbAoyUZrNsVea1kTmSOw$>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20231008/937fd7f6/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: FLASH.jpg
Type: image/jpeg
Size: 23876 bytes
Desc: not available
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20231008/937fd7f6/attachment-0001.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: FLASH-pride-sml.png
Type: image/png
Size: 12732 bytes
Desc: not available
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20231008/937fd7f6/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1391 bytes
Desc: not available
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20231008/937fd7f6/attachment-0001.p7s>
More information about the flash-users
mailing list