In Situ Workflows at Exascale: System Software to the
Rescue
Author/Presenter
Event Type
Workshop
Data Analytics
Data management
SIGHPC Workshop
Visualization
TimeSunday, November 12th11:55am -
12:15pm
Location605
DescriptionImplementing an in situ workflow involves several
challenges related to data placement, task scheduling,
efficient communications, scalability, and reliability.
Most of the current implementations provide reasonably
performant solutions to these issues by focusing on
high-performance communications and low-overhead
execution models at the cost of reliability and
flexibility. One of the key design choices in such
infrastructures is between providing a single-program,
integrated environment or a multiple- program, connected
environment, both solutions having their own strengths
and weaknesses. While these approaches might be
appropriate for current production systems, the expected
characteristics of exascale machines will shift current
priorities. After a survey of the trade-offs and
challenges of integrated and connected in situ workflow
solutions available today, we discuss in this paper how
exascale systems will impact those designs. In
particular, we identify missing features of current
system-level software required for the evolution of in
situ workflows toward exascale and how system software
innovations from the Argo Exascale Computing Project can
help address those challenges.
Author/Presenter
