Title
AdaScale SGD: A User-Friendly Algorithm for Distributed Training Tyler B. Johnson Pulkit Agrawal Haijie Gu Carlos Guestrin ODL 70 37 0 09 Jul 2020
Blink: Fast and Generic Collectives for Distributed ML Guanhua Wang Shivaram Venkataraman Amar Phanishayee J. Thelin Nikhil R. Devanur Ion Stoica VLM 54 139 0 11 Oct 2019
PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization Thijs Vogels Sai Praneeth Karimireddy Martin Jaggi 90 323 0 31 May 2019
Large Batch Optimization for Deep Learning: Training BERT in 76 minutes Yang You Jing Li Sashank J. Reddi Jonathan Hseu Sanjiv Kumar Srinadh Bhojanapalli Xiaodan Song J. Demmel Kurt Keutzer Cho-Jui Hsieh ODL 230 996 0 01 Apr 2019
Error Feedback Fixes SignSGD and other Gradient Compression Schemes Sai Praneeth Karimireddy Quentin Rebjock Sebastian U. Stich Martin Jaggi 66 502 0 28 Jan 2019
A Distributed Synchronous SGD Algorithm with Global Top- $k$ Sparsification for Low Bandwidth Networks Shaoshuai Shi Qiang-qiang Wang Kaiyong Zhao Zhenheng Tang Yuxin Wang Xiang Huang Xiaowen Chu 66 135 0 14 Jan 2019
Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training Chengyue Wu Song Han Huizi Mao Yu Wang W. Dally 136 1,407 0 05 Dec 2017
Poseidon: An Efficient Communication Architecture for Distributed Deep Learning on GPU Clusters Huatian Zhang Zeyu Zheng Shizhen Xu Wei-Ming Dai Qirong Ho Xiaodan Liang Zhiting Hu Jinliang Wei P. Xie Eric Xing GNN 67 345 0 11 Jun 2017
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour Priya Goyal Piotr Dollár Ross B. Girshick P. Noordhuis Lukasz Wesolowski Aapo Kyrola Andrew Tulloch Yangqing Jia Kaiming He 3DH 126 3,681 0 08 Jun 2017
In-Datacenter Performance Analysis of a Tensor Processing Unit N. Jouppi C. Young Nishant Patil David Patterson Gaurav Agrawal ... Vijay Vasudevan Richard Walter Walter Wang Eric Wilcox Doe Hyun Yoon 235 4,635 0 16 Apr 2017