rvGAHP – Push-Based Job Submission Using Reverse SSH
Connections
Author/Presenters
Event Type
Workshop
TimeMonday, November 13th11:45am -
12:10pm
Location501
DescriptionComputational science researchers running large-scale
scientific workflow applications often want to run their
workflows on the largest available compute systems to
improve time to solution. Workflow tools used in
distributed, heterogeneous, high performance computing
environments typically rely on either a push-based or a
pull-based approach for resource provisioning from these
compute systems. However, many large clusters have moved
to two-factor authentication for job submission, making
traditional automated push-based job submission
impossible. On the other hand, pull-based approaches
such as pilot jobs may lead to increased complexity and
a reduction in node-hour efficiency. In this paper, we
describe a new, efficient approach based on HTCondor-G
called reverse GAHP (rvGAHP) that allows us to push jobs
using reverse SSH submissions with better efficiency
than pull-based methods. We successfully used this
approach to perform a large probabilistic seismic hazard
analysis study using SCEC’s CyberShake workflow in March
2017 on the Titan Cray XK7 hybrid system at Oak Ridge
National Laboratory.
Author/Presenters
