The Case for Strong Scaling in Deep Learning: Training Large 3D CNNs with Hybrid Parallelism

25 July 2020

Papers citing "The Case for Strong Scaling in Deep Learning: Training Large 3D CNNs with Hybrid Parallelism"

7 / 7 papers shown

Title
A Survey From Distributed Machine Learning to Distributed Deep Learning Mohammad Dehghani Zahra Yazdanparast 20 0 0 11 Jul 2023
RAMP: A Flat Nanosecond Optical Network and MPI Operations for Distributed Deep Learning Systems Alessandro Ottino Joshua L. Benjamin G. Zervas 30 7 0 28 Nov 2022
SOLAR: A Highly Optimized Data Loading Framework for Distributed Training of CNN-based Scientific Surrogates Baixi Sun Xiaodong Yu Chengming Zhang Jiannan Tian Sian Jin K. Iskra Tao Zhou Tekin Bicer Pete Beckman Dingwen Tao 19 1 0 01 Nov 2022
Improving the Robustness of Federated Learning for Severely Imbalanced Datasets Debasrita Chakraborty Ashish Ghosh FedML 13 2 0 28 Apr 2022
TopoOpt: Co-optimizing Network Topology and Parallelization Strategy for Distributed Training Jobs Weiyang Wang Moein Khazraee Zhizhen Zhong M. Ghobadi Zhihao Jia Dheevatsa Mudigere Ying Zhang A. Kewitsch 34 81 0 01 Feb 2022
Clairvoyant Prefetching for Distributed Machine Learning I/O Nikoli Dryden Roman Böhringer Tal Ben-Nun Torsten Hoefler 31 55 0 21 Jan 2021
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 284 2,890 0 15 Sep 2016