Scalable Reduction Collectives with Data
Partitioning-Based Multi-Leader Design
SessionOptimizing MPI
Event Type
Paper
Programming Systems
TimeThursday, November 16th4:30pm -
5pm
Location405-406-407
DescriptionExisting designs for MPI Allreduce do not take
advantage of the vast parallelism available in modern
multi-/many-core processors like Intel Xeon/Xeon Phi or
the increases in communication throughput and recent
advances in high-end features seen with modern
interconnects like InfiniBand and OmniPath. In this
paper, we propose a high-performance and scalable Data
Partitioning-based Multi-Leader (DPML) solution for MPI
Allreduce that can take advantage of the parallelism
offered by multi-/many-core architectures in conjunction
with high throughput and high-end features offered by
InfiniBand and Omni-Path to significantly enhance the
performance of MPI Allreduce on modern HPC systems. We
also model DPML-based designs to analyze the
communication costs theoretically. Microbenchmark level
evaluations show that the proposed DPML-based designs
are able to deliver up to 3.5 times performance
improvement for MPI Allreduce for multiple HPC systems
at scale. At the application-level, up to 35% and 60%
improvements is seen for HPCG and miniAMR
respectively.
Download PDF:
here
