Tracking of Online Parameter Fine-Tuning in Scientific
Workflows
Author/Presenter
Event Type
Workshop
TimeMonday, November 13th12:10pm -
12:20pm
Location501
DescriptionIn long-lasting large-scale workflow executions,
computational scientists need to adapt the workflow by
fine-tuning several parameters of complex computational
models. These specific tunings may significantly reduce
overall execution time. In executions that last for
weeks, for instance, they can easily lose track of what
has been tuned at previous simulation stages if the
adaptations are not registered properly. In this work,
we propose a solution for tracking parameter
fine-tunings at runtime. With support of sophisticated
online data analysis, scientists get a detailed view of
the execution, providing insights to determine when and
how to tune parameters. We developed DfAdapter*, a tool
that collects human adaptations in the dataflow, while
the workflow runs with or without a Scientific Workflow
Management System. It controls and stores specific
parameter-tunings in a provenance database, relating the
human adaptation actions with data for: domain, dataflow
provenance, execution, and performance. An extended
PROV-compliant data diagram records the adaptation data.
We evaluate DfAdapter by plugging it into a high
performance workflow built with the libMesh library. The
experiments with real data, from the Oil and Gas domain,
showed that tunings significantly reduced the simulation
time.
* DfAdapter Repository. Available at: https://github.com/hpcdb/DfAdapter
* DfAdapter Repository. Available at: https://github.com/hpcdb/DfAdapter
Author/Presenter
