P58: Wharf: Sharing Docker Images across Hosts from a
Distributed Filesystem
SessionPoster Reception
Authors
Event Type
ACM Student Research Competition
Poster
Reception
TimeTuesday, November 14th5:15pm -
7pm
LocationFour Seasons Ballroom
DescriptionDue to their portability and less overhead compared to
traditional virtual machines, containers are becoming an
attractive solution for running HPC workloads. Docker is
a popular toolset which enables convenient provisioning
and management of containers and their corresponding
images. However, Docker does not natively support
running on shared storage, a crucial requirement in
large-scale HPC clusters which are often diskless or
access data via a shared burst buffer layer. This lack
of distributed storage support can lead to overhead when
running containerized HPC applications. In this work, we
explore how Docker images can be served efficiently from
a shared distributed storage layer. We implement a
distributed layer on top of Docker that allows multiple
Docker daemons to access container images from a shared
filesystem such as IBM Spectrum Scale or NFS. Our design
is independent of the underlying storage layer and
minimizes the synchronization overhead between different
daemons.
