FLASH's New Fully Implicit Solver

As computational resources have grown more powerful, the ability to utilize them effectively has grown proportionately more challenging. The availability of efficient computational tools and mathematical algorithms has become increasingly more important for researchers in the computational physics community. Among the more challenging research problems are ''stiff'' systems, involving nonlinear physics with wide ranges of both length and time scales spread over several orders of magnitude.


Typical examples of stiff systems arise when the small scale variations are much more rapid than the overall dynamical time scale of the system. Some examples of stiff systems include (i) chemical reaction systems, where chemical reactions occur on much faster time scales than the fastest wave speeds in the gas, (ii) advection-diffusion systems, for which the diffusion time scale starts to dominate the flow dynamics, (iii) nonlinear stiff gravity wave systems, where the fast gravity wave exceeds the fluid velocity by several orders of magnitude, (iv) transonic compressible flows containing shocks, where in low Mach number (subsonic) region time steps are limited by the fast sound speed, while in supersonic regions time steps are limited by the slow fluid advection velocity, (v) plasma physics, for which the fast electron time scale becomes more important than the slow ion time scale (the two-fluid Hall MHD), and (vi) radiation diffusion dominated systems, where the radiative diffusion time scale is limited by a radiation Courant limit, approximately the inverse of the Boltzmann number, typically larger than the fluid advection time step limit by a factor of 100.


All of the above examples readily demonstrate the difficulty of solving stiff problems with explicit time-stepping methods. The timesteps in explicit methods are dictated by the smallest scales; the system would need to evolve over a very large number of timesteps to achieve any meaningful solution. The small scales are by themselves not very informative, but they need to be resolved to reach the correct overall solution. Such computations are therefore exceedingly wasteful of computing resources.


Our goal in this proposal is to develop a high-order, fully implicit numerical algorithm based on a Jacobian-Free Newton-Krylov method with a choice of a Schwarz preconditioner on a parallel AMR grid for equations of magnetohydrodynamics (MHD).


The proposed work will also extend the flexibility of the available solution options by judiciously combining elements of the explicit and implicit methods through careful studies for balancing accuracy and performance. The basic idea is to be able to select the most suitable method for a region of the domain based upon the associated stiffness in that region. In this hybrid approach, the explicit unsplit solvers (gas dynamics or MHD) will be used in regions of shocks to better resolve discontinuities (without suffering stiffness in time scales), while the implicit solver will be used in regions where explicit time scales become too restricted and stiff. The overall hybrid solution can therefore progress at a reasonably large time-scale. However, the JFNK-based implicit solver is very likely to be substantially more expensive than the explicit solver per time advancement calculation. This disparity will also necessitate research into more suitable load balancing techniques for the hybrid methodology on massively parallel computer architectures.


Research Team

Dongwook Lee
Shravan Gopal

References
  1. Lee D., Xia G., Daley, C., Dubey, A., Gopal, S., Graziani, C., Lamb, D., Weide, K., 2011, Progress Toward HEDP in FLASH's Unsplit Staggered Mesh MHD Solver , An International Journal of Astronomy, Astrophysics and Space Science,
  2. Toth, G. and De Zeeuw, D.L. and Gombosi, T.I. and Powell, K.G., 2006, A parallel explicit/implicit time stepping scheme on block-adaptive grids, Journal of Computational Physics, 217
  3. Knoll, DA and Keyes, DE, 2004, Jacobian-free Newton-Krylov methods: a survey of approaches and applications, Journal of Computational Physics, 193
  4. Saad, Y., 2003, Iterative Methods for Sparse Linear Systems , SIAM, Second Edition