[FLASH-USERS] [EXT] Re: Retrieving values in HYPRE matrix

Fri Feb 26 10:34:25 EST 2021

Hi Akash, Eddie,

Thanks for the feedback. I tried adding HYPRE_SStructMatrixAssemble before trying to retrieve the values of gr_hypreMatA, and it did have an effect, but the values are still wrong. Here is what I am seeing for a 2D matrix:
   1      0.250226772E-02      0.250226772E-02
   2      0.000000000E+00      -.249827048E-02
   3      -.249827048E-02      -.399723277E-05
   4      0.000000000E+00      -.399723277E-05
   5      -.399723277E-05      -.399723277E-05
   6      0.250626495E-02      0.250626495E-02
   7      0.000000000E+00      -.249827048E-02
   8      -.249827048E-02      0.000000000E+00
   9      -.399723277E-05      0.250626495E-02
  10      -.399723277E-05      0.000000000E+00
  …
  476      0.500053820E-02      0.000000000E+00
 477      -.249827048E-02      0.000000000E+00
 478      -.249827048E-02      0.000000000E+00
 479      -.399723277E-05      0.000000000E+00
 480      0.000000000E+00      0.000000000E+00
 481      0.500053820E-02      0.000000000E+00
 482      -.249827048E-02      0.000000000E+00
…

Maybe a bit more context into what I’m interested in doing will be helpful. I’m currently writing a new module within the general implicit diffusion solver to do reduced-order modeling (via projection-based methods, e.g. POD). Since HYPRE is a bit of a mystery to me as well, I decided to use PETSc in my module (due to familiarity and more built-in functionality – at least to my knowledge). Doing so requires me to transform the HYPRE matrix, gr_hypreMatA, into a PETSc matrix. Conceptually it is easy to do, I just need to be able to extract the values from gr_hypreMatA into an array of reals so I can insert them into a PETSc matrix. I was able to do it in 1D, but my method does not work for 2D – thus me reaching out to the mailing list. Any suggestions would be welcomed.

Cheers,
Tony

From: Eddie Hansen <ehansen at pas.rochester.edu>
Date: Friday, February 26, 2021 at 6:27 AM
To: Akash Dhruv <akashdhruv at email.gwu.edu>
Cc: "Alberti, Anthony L" <anthony.alberti at oregonstate.edu>, "flash-users at flash.uchicago.edu" <flash-users at flash.uchicago.edu>
Subject: Re: [EXT] Re: [FLASH-USERS] Retrieving values in HYPRE matrix

[This email originated from outside of OSU. Use caution with links and attachments.]
Hi Tony,

I'm not sure if you can retrieve all the box values in a stencil-wise manner from gr_hypreMatA the way you're thinking. If my understanding is correct, the SetBoxValues routine uses the stencil and the offsets like a guide so that it knows which entries in gr_hypreMatA to set or add to. So when you use GetBoxValues this way, you're assigning "combined" matrix entries from gr_hypreMatA to test and you'll have repeated values based on the stencil and offsets.

In other words, let's say you're in 2D and your stencil is 5 points in this order: (i,j), (i-1,j), (i+1,j), (i,j-1), (i,j+1). The GetBoxValues routine will set test from matrix A as follows:
test(1) = A(i,j)
test(2) = A(i-1,j)
test(3) = A(i+1,j)
test(4) = A(i,j-1)
test(5) = A(i,j+1)
test(6) = A(i,j+1)
test(7) = A(i-1,j+1)
test(8) = A(i+1,j+1)
test(9) = A(i,j)
test(10) = A(i,j+2)
etc., etc.

I've dealt with the HYPRE setup routines in FLASH a lot since I'm the one primarily in charge of implementing new implicit diffusion solvers. However, much of HYPRE itself is still a mystery to me, so I could still be wrong about how the SetBoxValues and GetBoxValues routines are operating.

Alternatively, if you're just interested in looking at gr_hypreMatA, you can use the subroutine HYPRE_SStructMatrixPrint(). There should be some lines in Grid_advanceDiffusion.F90 that you can uncomment to do this. Then you can use matlab or whatever program you prefer to look at the matrix.

On Thu, Feb 25, 2021 at 9:09 PM Akash Dhruv <akashdhruv at email.gwu.edu<mailto:akashdhruv at email.gwu.edu>> wrote:
Hi Tony,

Can you share a snippet of the output? Also, can you try getting the BoxValues after HYPRE_SStructMatrixAssemble is called?

Akash

On Feb 25, 2021, at 9:48 PM, Alberti, Anthony L <anthony.alberti at oregonstate.edu<mailto:anthony.alberti at oregonstate.edu>> wrote:

Hi folks,

I’m interested in retrieving the values written to the matrix for the system of equations built in source/Grid/GridSolvers/HYPRE/UG/gr_hypreCreateMatrix.F90. After BoxVal is filled and then assigned to gr_hypreMatA(via the call to HYPRE_SStructMatrixSetBoxValues) I’ve added a snippet of code that attempts to retrieve the values just written to gr_hypreMatA and writes to the screen a comparison of BoxVal and an array of reals called test – see the snippet below. However, when I run this the values for BoxVal and test are not even close. Does anyone have an idea on how to alter this snippet so test and BoxVal are the same?

allocate(test(nentries*product(datasize(1:NDIM)))) !nentries * total number of grid cells per block )
call HYPRE_SStructMatrixGetBoxValues(gr_hypreMatA, mypart, gr_hypreLower(lb,1:NDIM), &
     gr_hypreUpper(lb,1:NDIM), var, nentries, stencil_indices(1:nentries), test(:), ierr)
do i=1,nentries*product(datasize(1:NDIM))
   WRITE(*,fmt='(I4.0, 6X, E15.9, 6X, E15.9)') i, BoxVal(i), test(i)
end do
call exit(0)

Thanks in advance,
Tony

-----------------------------------------------------------
Tony Alberti
Postdoctoral Scholar
School of Nuclear Science and Engineering
https://ne.oregonstate.edu/tony-alberti<https://urldefense.proofpoint.com/v2/url?u=https-3A__ne.oregonstate.edu_tony-2Dalberti&d=DwMFaQ&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=6F-TMED_bsUqKfX7XKE7kRK41M2Ogti8WJdx1GwNjkQ&m=viEEdaFV01zo9zrxAe3CDWtm6Kozcf5kb-Yoh2znXIA&s=aObgG-RLEKrNZ5hGr5VKNmbiTxeqGRkOBnGgU1QNERw&e=>

--
Eddie Hansen, PhD
Postdoctoral Associate
University of Rochester
607-341-6126 | Flash Center
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://flash.rochester.edu/pipermail/flash-users/attachments/20210226/0620149c/attachment.htm>