<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<meta content="text/html; charset=utf-8">
</head>
<body>
<div style="">
<div dir="auto">
<div dir="auto">
<div id="signature-div-E5DFA236-53BD-4135-BCD1-22B3214E45A4" dir="auto">For the record, most centers have a mechanism in place for new users to generate such scaling tests. For example, this is one of the reasons why director’s discretionary time is available
at many centers.</div>
<div id="signature-div-E5DFA236-53BD-4135-BCD1-22B3214E45A4" dir="auto"><br>
</div>
<div id="signature-div-E5DFA236-53BD-4135-BCD1-22B3214E45A4" dir="auto">Raph<br>
<br>
</div>
</div>
</div>
</div>
<hr>
<div dir="auto" style=""><br>
<b>From:</b> Sean M. Couch <couch@pa.msu.edu><br>
<b>Date:</b> May 7, 2018 at 11:34:30 AM EDT<br>
<b>To:</b> Jason Galyardt <jason.galyardt@gmail.com>, flash-users <flash-users@flash.uchicago.edu><br>
<b>Subject:</b> Re: [FLASH-USERS] FLASH Scaling Data<br>
</div>
<br>
<div>
<div name="messageBodySection" style="font-size:14px">Hi Jason et al.,
<div><br>
</div>
<div>To throw more fuel on the fire, I agree that major allocation programs will want performance data for your specific application on their specific resource. And I think of FLASH as a simulation _framework_, not a single code. I.e., there is an near-infinite
number of applications that can be built from it with all the possible permutations of physics, solvers, grids, etc. </div>
<div><br>
</div>
<div>That said, as an example (and at the risk of revealing all of our top secrets. Gasp!) I’ve made our most recent successful INCITE proposal using FLASH public: <a href="https://github.com/smcouch/INCITE_2018">https://github.com/smcouch/INCITE_2018</a>.
Of particular interest to you will be the computational readiness section where we show scaling on both Mira and Theta at ALCF to the entire platforms. Mira weak scaling plot attached. Now, these will not be directly applicable for you because they are for
our specific neutrino transport+MHD application, and both of those solvers I wrote from scratch in FLASH and have not made public...yet :) </div>
<div><br>
</div>
<div>Hope this helps (you, and others)!</div>
<div><br>
</div>
<div>Sean</div>
</div>
<div name="messageSignatureSection" style="font-size:14px"><br>
<div><br>
</div>
<div>
<p style="margin:0px; line-height:normal; font-family:'Helvetica Neue'; color:rgb(51,51,51)">
--------------------------------------------------------------------------------------------</p>
<p style="margin:0px; line-height:normal; font-family:'Helvetica Neue'; color:rgb(51,51,51)">
Sean M. Couch</p>
<p style="margin:0px; line-height:normal; font-family:'Helvetica Neue'; color:rgb(51,51,51)">
Assistant Professor</p>
<p style="margin:0px; line-height:normal; font-family:'Helvetica Neue'; color:rgb(51,51,51)">
Department of Physics and Astronomy</p>
<p style="margin:0px; line-height:normal; font-family:'Helvetica Neue'; color:rgb(51,51,51)">
Department of Computational Mathematics, Science, and Engineering</p>
<p style="margin:0px; line-height:normal; font-family:'Helvetica Neue'; color:rgb(51,51,51)">
National Superconducting Cyclotron Laboratory/Facility for Rare Isotope Beams</p>
<p style="margin:0px; line-height:normal; font-family:'Helvetica Neue'; color:rgb(51,51,51)">
Michigan State University</p>
<p style="margin:0px; line-height:normal; font-family:'Helvetica Neue'; color:rgb(51,51,51)">
567 Wilson Rd, 3250 BPS</p>
<p style="margin:0px; line-height:normal; font-family:'Helvetica Neue'; color:rgb(51,51,51)">
East Lansing, MI 48824</p>
<p style="margin:0px; line-height:normal; font-family:'Helvetica Neue'; color:rgb(62,129,222)">
<span style="color:#333333">(517) 884-5035 —— <a href="mailto:couch@pa.msu.edu"><span style="color:#3e81de">couch@pa.msu.edu</span></a> ——
<a href="http://www.pa.msu.edu/~couch"><span style="color:#3e81de">www.pa.msu.edu/~couch</span></a></span></p>
</div>
</div>
<div name="messageReplySection" style="font-size:14px"><br>
On May 7, 2018, 10:34 AM -0400, Messer, Bronson <bronson@ornl.gov>, wrote:<br>
<blockquote type="cite" style="margin:5px 5px; padding-left:10px; border-left:thin solid #1abc9c">
Let me reinforce Bob’s comments: As an example, for the DOE SC INCITE program, any proposal that referred to previous scaling results without
<div class="">having particular scaling and performance data for the problem proposed would be deemed “not computationally ready.” Such a </div>
<div class="">proposal would be significantly disadvantaged relative to other proposals.</div>
<div class=""><br class="">
</div>
<div class="">Bronson<br class="">
<div><br class="">
<blockquote type="cite" class="" style="margin:5px 5px; padding-left:10px; border-left:thin solid #e67e22">
<div class="">On May 7, 2018, at 9:32 AM, Robert Fisher <<a href="mailto:rfisher1@umassd.edu" class="">rfisher1@umassd.edu</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">Dear Jason :
<div class=""><br class="">
</div>
<div class=""> The gold standard is to include your own metrics on as similar a platform as possible -- ideally the one you are proposing time on. This is because scaling depends not just upon the code you are using, but also of course upon the hardware and
software library stacks implemented upon it. As a result, hard experience teaches that just because a code can scan on one platform, does not necessarily mean it will scale on another platform, even with identical setups.</div>
<div class=""><br class="">
</div>
<div class=""> That said, if the proposal is for a relatively small amount of time on a relatively small number of cores, and if the physics is primarily hydrodynamics, referring to the studies that Lynn mentions might suffice to convince your reviewers. However,
even in those instances, I'd strongly recommend including your own statistics if at all possible, since it shows the review committee that you've done your homework. FLASH has built-in coarse-grained timing diagnostics output at the end of each complete run,<br>
so it's not that that difficult to gather these.</div>
<div class=""><br class="">
</div>
<div class=""> Best wishes,</div>
<div class=""><br class="">
</div>
<div class=""> Bob<br class="">
<br class="">
<div class="gmail_quote">
<div dir="ltr" class="">On Sun, May 6, 2018 at 11:05 PM Jason Galyardt <<a href="mailto:jason.galyardt@gmail.com" class="">jason.galyardt@gmail.com</a>> wrote:<br class="">
</div>
<blockquote class="gmail_quote" style="margin:5px 5px; padding-left:10px; border-left:thin solid #3498db">
<div dir="ltr" class="">
<div class="">Hi all,<br class="">
<br class="">
</div>
<div class="">I'm working on an proposal for a compute time allocation at a major cluster facility. The proposal guidelines request quantitative evidence concerning the parallel performance, stability, and scalability of FLASH. Network and I/O bandwidth benchmarks
are also requested. I realize that there is quite a bit of variability in these performance metrics according to the particular simulation, physics included, solver used, etc. However, I have seen some old (FLASH 2 era) scalability studies; how extensible
are such studies? Is it necessary to profile one's own simulation for such proposals? If so, are there any recommended profiling tools / procedures, aside from those included in FLASH?<br class="">
<br class="">
</div>
<div class="">Thanks,<br class="">
<br class="">
</div>
<div class="">Jason<br class="">
<br class="">
--------------<br class="">
</div>
<div class="">Jason Galyardt, PhD<br class="">
</div>
University of Georgia<br class="">
</div>
</blockquote>
</div>
</div>
</div>
--<br class="">
<div dir="ltr" class="gmail_signature">
<div dir="ltr" class="">Dr. Robert Fisher
<div class="">Associate Professor and Graduate Program Director</div>
<div class="">Physics Department</div>
<div class="">University of Massachusetts Dartmouth</div>
<div class="">285 Old Westport Road</div>
<div class="">North Dartmouth, Ma. 02740</div>
<div class=""><a href="http://www.novastella.org/" class="">http://www.novastella.org</a></div>
<div class=""><a href="mailto:robert.fisher@umassd.edu" class="">robert.fisher@umassd.edu</a></div>
</div>
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</blockquote>
<div></div>
</div>
</div>
</body>
</html>