[FLASH-USERS] hangup with laser2Din3D
Norbert Flocke
flocke at flash.uchicago.edu
Fri Apr 22 15:00:43 EDT 2016
Hi Mario,
>From the comments right next to your ed_lens[X,Y,Z] runtime parameters
(the X referring to radial position and the Y to axial position) I
can see that you set up the laser beams in the original cylindrical 2D
coordinates, which is what you should not do. The ed_lens[X,Y,Z] and the
ed_target[X,Y,Z] must be set up using the expanded 3D space. The
cylindrical domain is placed inside the 3D coordinate system such that
the 3D z axis corresponds now to the cylinder axis and the 3D (x,y) plane
contains the circular cross sections of the cylinder. I would suggest you
read section 16.4.7 '3D Laser Ray Tracing in 2D Cylindrical Symmetry' in
the user's guide to get an idea and a feeling for the 3D in 2D laser
simulations.
Other than that I can only tell you that it seems to be somewhat unlikely
that your crashes are due to the laser, especially since they occur
suddenly without any message. If something is wrong with the laser setup
(for example placing targets outside of the domain or lenses inside the
domain) the laser package will tell you that. Those crashes you get seem
to be more like memory issues.
Best,
Norbert
On Thu, 21 Apr 2016, Mario Manuel wrote:
> Hi Norbert,
>
> I am not sure. I found it quite confusing, so to check things I thought I'd look
> at it in visit. The code that doesn't run defines the lasers using:
>
> --------------------------------------------------------------------------------------------------------
> --------------------------------------------------------------------------------------------------------
> useEnergyDeposition = .true.
> ed_maxRayCount = 100000
> ed_gradOrder = 2
>
> # Activate 3D-in-2D ray trace:
> ed_laser3Din2D = .true.
> ed_laser3Din2DwedgeAngle = 0.5
>
> ### LASER IO OPTIONS ###
> ed_useLaserIO = .true.
> ed_laserIOMaxNumberOfPositions = 10000
> ed_laserIOMaxNumberOfRays = 128
>
> ed_numberOfPulses = 3
> ed_numberOfBeams = 3
>
> # Define Beam 1: 2/7 of the power 2.84e12*(2/7)
> # approximate lens location for OMEGA (R_cham=3.33m) define angles relative to target normal
> #---------- CONE 1 at 32 degrees, y=R*cos(theta), x=R*sin(theta)
> ed_numberOfSections_1 = 4
> ed_time_1_1 = 0.0
> ed_time_1_2 = 0.1e-09
> ed_time_1_3 = 1.1e-09
> ed_time_1_4 = 1.2e-09
> # power can be adjusted to test cone dependence in the future
> ed_power_1_1 = 0.0
> ed_power_1_2 = 0.81e+12
> ed_power_1_3 = 0.81e+12
> ed_power_1_4 = 0.0
> # set up the beam profile
> ed_lensX_1 = 176.5 # radial position
> ed_lensY_1 = 282.4 # axial position
> ed_lensZ_1 = 0.0
> ed_lensSemiAxisMajor_1 = 14.925
> ed_targetX_1 = 0.0e-04
> ed_targetZ_1 = 0.0e-04
> ed_targetY_1 = 0.0e-04
> # define major and minor radii at the target
> ed_targetSemiAxisMajor_1 = 500.0e-04
> ed_targetSemiAxisMinor_1 = 500.0e-04
> ed_pulseNumber_1 = 1
> ed_wavelength_1 = 0.351
> ed_crossSectionFunctionType_1 = "gaussian2D"
> ed_gaussianExponent_1 = 4.3
> ed_gaussianRadiusMajor_1 = 250.0e-04
> ed_gaussianRadiusMinor_1 = 250.0e-04
> ed_numberOfRays_1 = 5000
> ed_gridType_1 = "radial2D"
> ed_semiAxisMajorTorsionAngle_1= 0.0
> ed_semiAxisMajorTorsionAxis_1 = "x"
>
>
> # Define Beam 2: 3/7 of the power 2.84e12*(3/7)
> # approximate lens location for OMEGA (R_cham=3.33m) define angles relative to target normal
> #-------------- CONE 2 at 42 degrees, y=R*cos(theta), x=R*sin(theta)
> ed_numberOfSections_2 = 4
> ed_time_2_1 = 0.0
> ed_time_2_2 = 0.1e-09
> ed_time_2_3 = 1.1e-09
> ed_time_2_4 = 1.2e-09
> # power can be adjusted to test cone dependence in the future
> ed_power_2_1 = 0.0
> ed_power_2_2 = 1.22e+12
> ed_power_2_3 = 1.22e+12
> ed_power_2_4 = 0.0
> # set up the beam profile
> ed_lensX_2 = 222.8 # radial position
> ed_lensY_2 = 247.5 # axial position
> ed_lensZ_2 = 0.0
> ed_lensSemiAxisMajor_2 = 14.925
> ed_targetX_2 = 0.0e-04
> ed_targetZ_2 = 0.0e-04
> ed_targetY_2 = 0.0e-04
> # define major and minor radii at the target
> ed_targetSemiAxisMajor_2 = 500.0e-04
> ed_targetSemiAxisMinor_2 = 500.0e-04
> # laser profile from with SG4 profile
> ed_pulseNumber_2 = 2
> ed_wavelength_2 = 0.351
> ed_crossSectionFunctionType_2 = "gaussian2D"
> ed_gaussianExponent_2 = 4.3
> ed_gaussianRadiusMajor_2 = 250.0e-04
> ed_gaussianRadiusMinor_2 = 250.0e-04
> ed_numberOfRays_2 = 5000
> ed_gridType_2 = "radial2D"
> ed_semiAxisMajorTorsionAngle_2= 0.0
> ed_semiAxisMajorTorsionAxis_2 = "x"
>
>
> # Define Beam 3: 2/7 of the power 2.84e12*(2/7)
> # approximate lens location for OMEGA (R_cham=3.33m) define angles relative to target normal
> #------------- CONE 3 at 50 degrees, y=R*cos(theta), x=R*sin(theta)
> ed_numberOfSections_3 = 4
> ed_time_3_1 = 0.0
> ed_time_3_2 = 0.1e-09
> ed_time_3_3 = 1.1e-09
> ed_time_3_4 = 1.2e-09
> # power can be adjusted to test cone dependence in the future
> ed_power_3_1 = 0.0
> ed_power_3_2 = 0.81e+12
> ed_power_3_3 = 0.81e+12
> ed_power_3_4 = 0.0
> # set up the beam profile
> ed_lensX_3 = 255.1 # radial position
> ed_lensY_3 = 214.0 # axial position
> ed_lensZ_3 = 0.0
> ed_lensSemiAxisMajor_3 = 14.925
> ed_targetX_3 = 0.0e-04
> ed_targetZ_3 = 0.0e-04
> ed_targetY_3 = 0.0e-04
> # define major and minor radii at the target
> ed_targetSemiAxisMajor_3 = 500.0e-04
> ed_targetSemiAxisMinor_3 = 500.0e-04
> ed_pulseNumber_3 = 3
> ed_wavelength_3 = 0.351
> ed_crossSectionFunctionType_3 = "gaussian2D"
> ed_gaussianExponent_3 = 4.3
> ed_gaussianRadiusMajor_3 = 250.0e-04
> ed_gaussianRadiusMinor_3 = 250.0e-04
> ed_numberOfRays_3 = 5000
> ed_gridType_3 = "radial2D"
> ed_semiAxisMajorTorsionAngle_3= 0.0
> ed_semiAxisMajorTorsionAxis_3 = "x"
>
> --------------------------------------------------------------------------------------------------------
> --------------------------------------------------------------------------------------------------------
>
> however when I change the lines:
>
> ed_laser3Din2D = .false.
> ed_crossSectionFunctionType_X = "gaussian1D"
> ed_gridType_X = "regular1D"
>
> The code runs and the attached image shows a sample output early in the simulation. The geometry at least looks correct in that there are 3 beams coming in at different angles onto the target. I don't quite understand why they don't seem to be centered on 0,0, but perhaps I am not saving enough rays for this to look correct.
>
> For what its worth, I have also compiled the LaserSlab simulation on the system I'm using with the standard setup script in the 4.2.2 manual. When I run LaserSlab with the example.par setup, it reproduces the behavior by hanging up right after the output line 'Initial dt verified'. That to me suggests something more fundamental is going wrong on my system.
>
> Any input is very much welcomed.
>
> Cheers,
> Mario
>
> ______________________________________
>
> Dr. Mario Manuel
> Einstein Postdoctoral Fellow, Smithsonian Astrophysical Observatory
> mmanuel at umich.edu
> 617-899-8381
>
> University of Michigan
> Space Research Building
> 2455 Hayward Street
> Ann Arbor, MI 48109
> Office: 734-764-4585
> ______________________________________
>
>
> \
>
> On Apr 21, 2016, at 4:15 PM, Norbert Flocke <flocke at flash.uchicago.edu> wrote:
>
>> Hi Manuel,
>>
>> Are you sure you are setting up the beams correctly for this 3D in 2D simulation? The reason I am asking is because it is easy to get confused
>> between the cylindrical FLASH coordinates (x,y) -> (R,z) and the 3D beam coordinates you need to use. In your application (cylindrical) the FLASH y coordinate corresponds to the beam 3D z coordinate.
>>
>> Best,
>> Norbert
>>
>>
>> On Thu, 21 Apr 2016, Mario Manuel wrote:
>>
>>> Hello FLASH users,
>>>
>>> I am running a FLASH4.2.2 simulation that is a combination of the LaserSlab and FieldLoop
>>> example simulations. It is a laser-matter interaction (multi-material, 3T, tabulated EOSs) with MHD as well. I recently implemented a more complex laser setup with multiple beams and wanted to use the ed_laser3Din2D = .true. for a more accurate ray tracing in the R-Z geometry that I am using. After fixing all the initialization errors, I got it to run, but it never gets in to the first step. The simulation hangs up, but doesn't throw an error. The output looks like:
>>>
>>>
>>> -------------------------------------------------------------------------------------------------------------------------------
>>> -------------------------------------------------------------------------------------------------------------------------------
>>> Wed Apr 20 20:57:51 PDT 2016
>>> NODES= 40
>>> NTASKS= 1280
>>> [Driver_initParallel]: Called MPI_Init_thread - requested level 2, given level 2
>>> [Driver_initParallel]: Number of OpenMP threads in each parallel region 2
>>> NOTE: Enabling curvilinear, cartesian_pm/cylindrical_pm/spherical_pm/polar_pm so far was F T F F
>>> MaterialProperties initialized
>>> Cosmology initialized
>>> [eos_tabBrowseIonmix4Tables] IONMIX4 file found: he-imx-005.cn4
>>> in eos_inittabulated, tableName = he-imx-005.cn4
>>> in eos_inittabulated, tableName = he-imx-005.cn4
>>> in eos_inittabulated, groupName = -none-
>>> in eos_inittabulated, groupName = -none-
>>> in eos_inittabulated, tableName = he-imx-005.cn4
>>> in eos_inittabulated, groupName = -none-
>>> [eos_tabBrowseIonmix4Tables] IONMIX4 file found: polystyrene-imx-002.cn4
>>> in eos_inittabulated, tableName = he-imx-005.cn4
>>> in eos_inittabulated, groupName = -none-
>>> in eos_inittabulated, tableName = polystyrene-imx-002.cn4
>>> in eos_inittabulated, tableName = polystyrene-imx-002.cn4
>>> in eos_inittabulated, groupName = -none-
>>> in eos_inittabulated, groupName = -none-
>>> in eos_inittabulated, tableName = polystyrene-imx-002.cn4
>>> in eos_inittabulated, groupName = -none-
>>> in eos_inittabulated, tableName = polystyrene-imx-002.cn4
>>> in eos_inittabulated, groupName = -none-
>>> Source terms initialized
>>> iteration, no. not moved = 0 0
>>> refined: total leaf blocks = 3
>>> refined: total blocks = 3
>>> INFO: Grid_fillGuardCells is ignoring masking.
>>> iteration, no. not moved = 0 2
>>> iteration, no. not moved = 1 0
>>> refined: total leaf blocks = 12
>>> refined: total blocks = 15
>>> iteration, no. not moved = 0 13
>>> iteration, no. not moved = 1 2
>>> iteration, no. not moved = 2 0
>>> refined: total leaf blocks = 48
>>> refined: total blocks = 63
>>> iteration, no. not moved = 0 60
>>> iteration, no. not moved = 1 13
>>> iteration, no. not moved = 2 0
>>> refined: total leaf blocks = 192
>>> refined: total blocks = 255
>>> iteration, no. not moved = 0 251
>>> iteration, no. not moved = 1 60
>>> iteration, no. not moved = 2 0
>>> refined: total leaf blocks = 768
>>> refined: total blocks = 1023
>>> iteration, no. not moved = 0 283
>>> iteration, no. not moved = 1 101
>>> iteration, no. not moved = 2 0
>>> refined: total leaf blocks = 216
>>> refined: total blocks = 287
>>> iteration, no. not moved = 0 50
>>> iteration, no. not moved = 1 12
>>> iteration, no. not moved = 2 0
>>> refined: total leaf blocks = 96
>>> refined: total blocks = 127
>>> Finished with Grid_initDomain, no restart
>>> Ready to call Hydro_init
>>> [Hydro_init] The upwind electric field construction is NOT allowed for diffusion.
>>> Hydro initialized
>>> Gravity initialized
>>> Initial dt verified
>>> *** Wrote checkpoint file to /p/lscratchv/manuel3/ACSEL15B_CH/ACSEL15B_hdf5_chk_0000 ****
>>> *** Wrote plotfile to /p/lscratchv/manuel3/ACSEL15B_CH/ACSEL15B_hdf5_plt_cnt_0000 ****
>>> Initial plotfile written
>>> Driver init all done
>>> srun: got SIGCONT
>>> srun: forcing job termination
>>> -------------------------------------------------------------------------------------------------------------------------------
>>> -------------------------------------------------------------------------------------------------------------------------------
>>>
>>> and the last lines of the log file look like:
>>>
>>> -------------------------------------------------------------------------------------------------------------------------------
>>> -------------------------------------------------------------------------------------------------------------------------------
>>> [ 04-20-2016 21:18:56.632 ] [GRID amr_refine_derefine]: redist. phase. tot blks requested: 287
>>> [GRID amr_refine_derefine] min blks 0 max blks 1 tot blks 287
>>> [GRID amr_refine_derefine] min leaf blks 0 max leaf blks 1 tot leaf blks 216
>>> [ 04-20-2016 21:18:56.654 ] [GRID amr_refine_derefine]: refinement complete
>>> [ 04-20-2016 21:18:56.847 ] [GRID gr_expandDomain]: iteration=6, create level=5
>>> [ 04-20-2016 21:18:56.891 ] [GRID amr_refine_derefine]: initiating refinement
>>> [ 04-20-2016 21:18:56.896 ] [GRID amr_refine_derefine]: redist. phase. tot blks requested: 127
>>> [GRID amr_refine_derefine] min blks 0 max blks 1 tot blks 127
>>> [GRID amr_refine_derefine] min leaf blks 0 max leaf blks 1 tot leaf blks 96
>>> [ 04-20-2016 21:18:56.919 ] [GRID amr_refine_derefine]: refinement complete
>>> [ 04-20-2016 21:18:57.110 ] [GRID gr_expandDomain]: iteration=7, create level=5
>>> [ 04-20-2016 21:18:57.116 ] [gr_ensureValidNeighborInfo] found mpi_pattern_id: -10
>>> [ 04-20-2016 21:18:57.124 ] [mpi_amr_comm_setup]: buffer_dim_send=157, buffer_dim_recv=1
>>> [ 04-20-2016 21:18:57.130 ] [gr_hypreInit]: HYPRE_RELEASE_NAME=hypre, HYPRE_RELEASE_VERSION=2.8.0b, HYPRE_RELEASE_DATE=2011/11/14
>>> [ 04-20-2016 21:18:57.136 ] memory: bg heap use (MB): 126.46 (min) 135.21 (max) 131.85 (avg)
>>> [ 04-20-2016 21:18:57.141 ] memory: bg heap peak (MB): 126.46 (min) 135.21 (max) 131.85 (avg)
>>> [ 04-20-2016 21:18:57.145 ] memory: bg heap free (MB): 324.20 (min) 332.95 (max) 327.56 (avg)
>>> [ 04-20-2016 21:18:57.155 ] memory: bg heap use (MB): 126.46 (min) 135.21 (max) 131.85 (avg)
>>> [ 04-20-2016 21:18:57.169 ] memory: bg heap peak (MB): 126.46 (min) 135.21 (max) 131.85 (avg)
>>> [ 04-20-2016 21:18:57.263 ] memory: bg heap free (MB): 324.20 (min) 332.95 (max) 327.56 (avg)
>>> [ 04-20-2016 21:18:58.199 ] memory: bg heap use (MB): 126.46 (min) 135.21 (max) 131.85 (avg)
>>> [ 04-20-2016 21:18:58.205 ] memory: bg heap peak (MB): 126.46 (min) 135.21 (max) 131.85 (avg)
>>> [ 04-20-2016 21:18:58.209 ] memory: bg heap free (MB): 324.26 (min) 333.01 (max) 327.63 (avg)
>>> [ 04-20-2016 21:18:58.407 ] [IO_writeCheckpoint] open: type=checkpoint name=/p/lscratchv/manuel3/ACSEL15B_CH/ACSEL15B_hdf5_chk_0000
>>> [ 04-20-2016 21:18:59.943 ] [io_writeData]: wrote 127 blocks
>>> [ 04-20-2016 21:19:00.596 ] [IO_writeCheckpoint] close: type=checkpoint name=/p/lscratchv/manuel3/ACSEL15B_CH/ACSEL15B_hdf5_chk_0000
>>> [ 04-20-2016 21:19:00.682 ] [IO_writePlotfile] open: type=plotfile name=/p/lscratchv/manuel3/ACSEL15B_CH/ACSEL15B_hdf5_plt_cnt_0000
>>> [ 04-20-2016 21:19:01.269 ] [io_writeData]: wrote 127 blocks
>>> [ 04-20-2016 21:19:01.695 ] [IO_writePlotfile] close: type=plotfile name=/p/lscratchv/manuel3/ACSEL15B_CH/ACSEL15B_hdf5_plt_cnt_0000
>>> [ 04-20-2016 21:19:01.706 ] memory: bg heap use (MB): 134.27 (min) 135.21 (max) 134.74 (avg)
>>> [ 04-20-2016 21:19:01.711 ] memory: bg heap peak (MB): 134.27 (min) 151.21 (max) 134.77 (avg)
>>> [ 04-20-2016 21:19:01.716 ] memory: bg heap free (MB): 324.26 (min) 325.20 (max) 324.73 (avg)
>>> [ 04-20-2016 21:19:01.722 ] [Driver_evolveFlash]: Entering evolution loop
>>> [ 04-20-2016 21:19:01.727 ] step: n=1 t=0.000000E+00 dt=1.000000E-15
>>> [ 04-20-2016 21:19:01.746 ] [mpi_amr_comm_setup]: buffer_dim_send=27933, buffer_dim_recv=1
>>> [ 04-20-2016 21:19:01.783 ] [hy_uhd_unsplit]: gcNeed(MAGI_FACE_VAR,MAG_FACE_VAR) - FACES
>>> [ 04-20-2016 21:19:01.792 ] [mpi_amr_comm_setup]: buffer_dim_send=2333, buffer_dim_recv=1
>>> [ 04-20-2016 21:19:01.886 ] [mpi_amr_comm_setup]: buffer_dim_send=27933, buffer_dim_recv=1
>>> [ 04-20-2016 21:19:01.913 ] [mpi_amr_comm_setup]: buffer_dim_send=25757, buffer_dim_recv=1
>>> -------------------------------------------------------------------------------------------------------------------------------
>>> -------------------------------------------------------------------------------------------------------------------------------
>>>
>>> Everything runs fine when I switch the flag to ed_laser2Din3D = .false.
>>>
>>> Has anyone come across this type of thing before? Are there issues with the 2Din3D package and some other packages I may have implemented? Any thoughts are much welcomed.
>>>
>>> Cheers,
>>> Mario
>>>
>>> ______________________________________
>>>
>>> Dr. Mario Manuel
>>> Einstein Postdoctoral Fellow, Smithsonian Astrophysical Observatory
>>> mmanuel at umich.edu
>>> 617-899-8381
>>>
>>> University of Michigan
>>> Space Research Building
>>> 2455 Hayward Street
>>> Ann Arbor, MI 48109
>>> Office: 734-764-4585
>>> ______________________________________
>>>
>>>
>
>
More information about the flash-users
mailing list