<meta http-equiv="Content-Type" content="text/html; charset=utf-8"><div dir="ltr">You might've already tried this, but it's possible that your method for what you're trying to do is fine and you just need to flip the i and j indices. The ordering in FLASH (i,j) is opposite that of HYPRE (j,i).</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Feb 26, 2021 at 9:34 AM Alberti, Anthony L <<a href="mailto:anthony.alberti@oregonstate.edu">anthony.alberti@oregonstate.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div lang="EN-US" style="overflow-wrap: break-word;">
<div class="gmail-m_2541460198776133665WordSection1">
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Arial,sans-serif">Hi Akash, Eddie,
<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Arial,sans-serif"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Arial,sans-serif">Thanks for the feedback. I tried adding
</span>HYPRE_SStructMatrixAssemble before trying to retrieve the values of gr_hypreMatA, and it did have an effect, but the values are still wrong. Here is what I am seeing for a 2D matrix:<u></u><u></u></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10.5pt;font-family:Arial,sans-serif"> 1 0.250226772E-02 0.250226772E-02<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10.5pt;font-family:Arial,sans-serif"> 2 0.000000000E+00 -.249827048E-02<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10.5pt;font-family:Arial,sans-serif"> 3 -.249827048E-02 -.399723277E-05<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10.5pt;font-family:Arial,sans-serif"> 4 0.000000000E+00 -.399723277E-05<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10.5pt;font-family:Arial,sans-serif"> 5 -.399723277E-05 -.399723277E-05<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10.5pt;font-family:Arial,sans-serif"> 6 0.250626495E-02 0.250626495E-02<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10.5pt;font-family:Arial,sans-serif"> 7 0.000000000E+00 -.249827048E-02<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10.5pt;font-family:Arial,sans-serif"> 8 -.249827048E-02 0.000000000E+00<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10.5pt;font-family:Arial,sans-serif"> 9 -.399723277E-05 0.250626495E-02<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10.5pt;font-family:Arial,sans-serif"> 10 -.399723277E-05 0.000000000E+00<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10.5pt;font-family:Arial,sans-serif"> …<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10.5pt;font-family:Arial,sans-serif"> 476 0.500053820E-02 0.000000000E+00<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10.5pt;font-family:Arial,sans-serif"> 477 -.249827048E-02 0.000000000E+00<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10.5pt;font-family:Arial,sans-serif"> 478 -.249827048E-02 0.000000000E+00<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10.5pt;font-family:Arial,sans-serif"> 479 -.399723277E-05 0.000000000E+00<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10.5pt;font-family:Arial,sans-serif"> 480 0.000000000E+00 0.000000000E+00<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10.5pt;font-family:Arial,sans-serif"> 481 0.500053820E-02 0.000000000E+00<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10.5pt;font-family:Arial,sans-serif"> 482 -.249827048E-02 0.000000000E+00<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10.5pt;font-family:Arial,sans-serif">…<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Arial,sans-serif"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Arial,sans-serif">Maybe a bit more context into what I’m interested in doing will be helpful. I’m currently writing a new module within the general implicit diffusion solver to do reduced-order
modeling (via projection-based methods, e.g. POD). Since HYPRE is a bit of a mystery to me as well, I decided to use PETSc in my module (due to familiarity and more built-in functionality – at least to my knowledge). Doing so requires me to transform the HYPRE
matrix, gr_hypreMatA, into a PETSc matrix. Conceptually it is easy to do, I just need to be able to extract the values from gr_hypreMatA into an array of reals so I can insert them into a PETSc matrix. I was able to do it in 1D, but my method does not work
for 2D – thus me reaching out to the mailing list. Any suggestions would be welcomed.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Arial,sans-serif"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Arial,sans-serif">Cheers,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Arial,sans-serif">Tony
<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Arial,sans-serif"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Arial,sans-serif"><u></u> <u></u></span></p>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(181,196,223);padding:3pt 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12pt;color:black">From: </span></b><span style="font-size:12pt;color:black">Eddie Hansen <<a href="mailto:ehansen@pas.rochester.edu" target="_blank">ehansen@pas.rochester.edu</a>><br>
<b>Date: </b>Friday, February 26, 2021 at 6:27 AM<br>
<b>To: </b>Akash Dhruv <<a href="mailto:akashdhruv@email.gwu.edu" target="_blank">akashdhruv@email.gwu.edu</a>><br>
<b>Cc: </b>"Alberti, Anthony L" <<a href="mailto:anthony.alberti@oregonstate.edu" target="_blank">anthony.alberti@oregonstate.edu</a>>, "<a href="mailto:flash-users@flash.uchicago.edu" target="_blank">flash-users@flash.uchicago.edu</a>" <<a href="mailto:flash-users@flash.uchicago.edu" target="_blank">flash-users@flash.uchicago.edu</a>><br>
<b>Subject: </b>Re: [EXT] Re: [FLASH-USERS] Retrieving values in HYPRE matrix<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p><span style="color:rgb(215,63,9)">[This email originated from outside of OSU. Use caution with links and attachments.]</span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal">Hi Tony, <u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I'm not sure if you can retrieve all the box values in a stencil-wise manner from gr_hypreMatA the way you're thinking. If my understanding is correct, the SetBoxValues routine uses the stencil and the offsets like a guide so that it knows
which entries in gr_hypreMatA to set or add to. So when you use GetBoxValues this way, you're assigning "combined" matrix entries from gr_hypreMatA to test and you'll have repeated values based on the stencil and offsets.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">In other words, let's say you're in 2D and your stencil is 5 points in this order: (i,j), (i-1,j), (i+1,j), (i,j-1), (i,j+1). The GetBoxValues routine will set test from matrix A as follows:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">test(1) = A(i,j)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">test(2) = A(i-1,j)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">test(3) = A(i+1,j)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">test(4) = A(i,j-1)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">test(5) = A(i,j+1)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">test(6) = A(i,j+1)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">test(7) = A(i-1,j+1)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">test(8) = A(i+1,j+1)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">test(9) = A(i,j)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">test(10) = A(i,j+2)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">etc., etc.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I've dealt with the HYPRE setup routines in FLASH a lot since I'm the one primarily in charge of implementing new implicit diffusion solvers. However, much of HYPRE itself is still a mystery to me, so I could still be wrong about how the
SetBoxValues and GetBoxValues routines are operating.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Alternatively, if you're just interested in looking at gr_hypreMatA, you can use the subroutine HYPRE_SStructMatrixPrint(). There should be some lines in Grid_advanceDiffusion.F90 that you can uncomment to do this. Then you can use matlab
or whatever program you prefer to look at the matrix.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">On Thu, Feb 25, 2021 at 9:09 PM Akash Dhruv <<a href="mailto:akashdhruv@email.gwu.edu" target="_blank">akashdhruv@email.gwu.edu</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin-left:4.8pt;margin-right:0in">
<div>
<p class="MsoNormal">Hi Tony, <u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Can you share a snippet of the output? Also, can you try getting the BoxValues after HYPRE_SStructMatrixAssemble is called?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">Akash<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><br>
<br>
<u></u><u></u></p>
<blockquote style="margin-top:5pt;margin-bottom:5pt">
<div>
<p class="MsoNormal">On Feb 25, 2021, at 9:48 PM, Alberti, Anthony L <<a href="mailto:anthony.alberti@oregonstate.edu" target="_blank">anthony.alberti@oregonstate.edu</a>> wrote:<u></u><u></u></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Arial,sans-serif">Hi folks,</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Arial,sans-serif"> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Arial,sans-serif">I’m interested in retrieving the values written to the matrix for the system of equations built in source/Grid/GridSolvers/HYPRE/UG/gr_hypreCreateMatrix.F90. After </span><span style="font-size:10.5pt;font-family:Menlo">BoxVal</span><span style="font-size:10.5pt;font-family:Arial,sans-serif"> is
filled and then assigned to </span><span style="font-size:10.5pt;font-family:Menlo">gr_hypreMatA</span><span style="font-size:10.5pt;font-family:Arial,sans-serif">(via the call to </span><span style="font-size:10.5pt;font-family:Menlo">HYPRE_SStructMatrixSetBoxValues</span><span style="font-size:10.5pt;font-family:Arial,sans-serif">)
I’ve added a snippet of code that attempts to retrieve the values just written to </span><span style="font-size:10.5pt;font-family:Menlo">gr_hypreMatA</span><span style="font-size:10.5pt;font-family:Arial,sans-serif"> and writes to the screen a comparison
of </span><span style="font-size:10.5pt;font-family:Menlo">BoxVal</span><span style="font-size:10.5pt;font-family:Arial,sans-serif"> and an array of reals called </span><span style="font-size:10.5pt;font-family:Menlo">test</span><span style="font-size:10.5pt;font-family:Arial,sans-serif"> –
see the snippet below. However, when I run this the values for </span><span style="font-size:10.5pt;font-family:Menlo">BoxVal</span><span style="font-size:10.5pt;font-family:Arial,sans-serif"> and </span><span style="font-size:10.5pt;font-family:Menlo">test</span><span style="font-size:10.5pt;font-family:Arial,sans-serif"> are
not even close. Does anyone have an idea on how to alter this snippet so </span><span style="font-size:10.5pt;font-family:Menlo">test</span><span style="font-size:10.5pt;font-family:Arial,sans-serif"> and </span><span style="font-size:10.5pt;font-family:Menlo">BoxVal</span><span style="font-size:10.5pt;font-family:Arial,sans-serif"> are
the same?</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Arial,sans-serif"> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Menlo">allocate(test(nentries*product(datasize(1:NDIM)))) !nentries * total number of grid cells per block )</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Menlo">call HYPRE_SStructMatrixGetBoxValues(gr_hypreMatA, mypart, gr_hypreLower(lb,1:NDIM), &</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Menlo"> gr_hypreUpper(lb,1:NDIM), var, nentries, stencil_indices(1:nentries), test(:), ierr)</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Menlo">do i=1,nentries*product(datasize(1:NDIM))</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Menlo"> WRITE(*,fmt='(I4.0, 6X, E15.9, 6X, E15.9)') i, BoxVal(i), test(i)</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Menlo">end do</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Menlo">call exit(0)</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Arial,sans-serif"> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Arial,sans-serif">Thanks in advance,</span><u></u><u></u></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal">Tony<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Arial,sans-serif">-----------------------------------------------------------</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Arial,sans-serif;color:black;background:white">Tony Alberti</span><span style="font-size:10.5pt;font-family:Arial,sans-serif"><br>
<span style="color:black;background:white">Postdoctoral Scholar</span><br>
<span style="color:black;background:white">School of Nuclear Science and Engineering</span><br>
<a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__ne.oregonstate.edu_tony-2Dalberti&d=DwMFaQ&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=6F-TMED_bsUqKfX7XKE7kRK41M2Ogti8WJdx1GwNjkQ&m=viEEdaFV01zo9zrxAe3CDWtm6Kozcf5kb-Yoh2znXIA&s=aObgG-RLEKrNZ5hGr5VKNmbiTxeqGRkOBnGgU1QNERw&e=" target="_blank"><span style="color:rgb(17,85,204);background:white">https://ne.oregonstate.edu/tony-alberti</span></a></span><u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><br clear="all">
<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal">-- <u></u><u></u></p>
<div>
<div>
<div>
<p class="MsoNormal">Eddie Hansen, PhD<br>
Postdoctoral Associate<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">University of Rochester<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">607-341-6126 | Flash Center<u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div>Eddie Hansen, PhD<br>Postdoctoral Associate</div><div>University of Rochester</div><div>607-341-6126 | Flash Center</div></div></div>