v1v2 (latest)

GraVAC: Adaptive Compression for Communication-Efficient Distributed DL Training

20 May 2023

Papers citing "GraVAC: Adaptive Compression for Communication-Efficient Distributed DL Training"

26 / 26 papers shown

Title
Scavenger: A Cloud Service for Optimizing Cost and Performance of ML Training S. Tyagi Prateek Sharma 63 5 0 12 Mar 2023
ScaDLES: Scalable Deep Learning over Streaming data at the Edge S. Tyagi Martin Swany 42 6 0 21 Jan 2023
Pufferfish: Communication-efficient Models At No Extra Cost Hongyi Wang Saurabh Agarwal Dimitris Papailiopoulos 74 59 0 05 Mar 2021
On the Utility of Gradient Compression in Distributed Training Systems Saurabh Agarwal Hongyi Wang Shivaram Venkataraman Dimitris Papailiopoulos 77 47 0 28 Feb 2021
An Efficient Statistical-based Gradient Compression Technique for Distributed Training Systems A. Abdelmoniem Ahmed Elzanaty Mohamed-Slim Alouini Marco Canini 111 77 0 26 Jan 2021
Accordion: Adaptive Gradient Communication via Critical Learning Regime Identification Saurabh Agarwal Hongyi Wang Kangwook Lee Shivaram Venkataraman Dimitris Papailiopoulos 78 25 0 29 Oct 2020
Pollux: Co-adaptive Cluster Scheduling for Goodput-Optimized Deep Learning Aurick Qiao Sang Keun Choe Suhas Jayaram Subramanya Willie Neiswanger Qirong Ho Hao Zhang G. Ganger Eric Xing VLM 69 181 0 27 Aug 2020
AdaScale SGD: A User-Friendly Algorithm for Distributed Training Tyler B. Johnson Pulkit Agrawal Haijie Gu Carlos Guestrin ODL 76 37 0 09 Jul 2020
PyTorch Distributed: Experiences on Accelerating Data Parallel Training Shen Li Yanli Zhao R. Varma Omkar Salpekar P. Noordhuis ... Adam Paszke Jeff Smith Brian Vaughan Pritam Damania Soumith Chintala OOD MoE 63 187 0 28 Jun 2020
Is Network the Bottleneck of Distributed Training? Zhen Zhang Chaokun Chang Yanghua Peng Yida Wang R. Arora Xin Jin 89 71 0 17 Jun 2020
The Early Phase of Neural Network Training Jonathan Frankle D. Schwab Ari S. Morcos 87 174 0 24 Feb 2020
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 553 42,639 0 03 Dec 2019
Understanding Top-k Sparsification in Distributed Deep Learning Shaoshuai Shi Xiaowen Chu Ka Chun Cheung Simon See 226 101 0 20 Nov 2019
PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization Thijs Vogels Sai Praneeth Karimireddy Martin Jaggi 90 322 0 31 May 2019
Communication-Efficient Distributed Blockwise Momentum SGD with Error-Feedback Shuai Zheng Ziyue Huang James T. Kwok 54 115 0 27 May 2019
Error Feedback Fixes SignSGD and other Gradient Compression Schemes Sai Praneeth Karimireddy Quentin Rebjock Sebastian U. Stich Martin Jaggi 72 503 0 28 Jan 2019
An Empirical Model of Large-Batch Training Sam McCandlish Jared Kaplan Dario Amodei OpenAI Dota Team 74 280 0 14 Dec 2018
The Convergence of Sparsified Gradient Methods Dan Alistarh Torsten Hoefler M. Johansson Sarit Khirirat Nikola Konstantinov Cédric Renggli 169 493 0 27 Sep 2018
RedSync : Reducing Synchronization Traffic for Distributed Deep Learning Jiarui Fang Haohuan Fu Guangwen Yang Cho-Jui Hsieh GNN 88 25 0 13 Aug 2018
Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training Chengyue Wu Song Han Huizi Mao Yu Wang W. Dally 148 1,410 0 05 Dec 2017
Critical Learning Periods in Deep Neural Networks Alessandro Achille Matteo Rovere Stefano Soatto 66 100 0 24 Nov 2017
Mixed Precision Training Paulius Micikevicius Sharan Narang Jonah Alben G. Diamos Erich Elsen ... Boris Ginsburg Michael Houston Oleksii Kuchaiev Ganesh Venkatesh Hao Wu 178 1,805 0 10 Oct 2017
Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond Levent Sagun Léon Bottou Yann LeCun UQCV 95 236 0 22 Nov 2016
An overview of gradient descent optimization algorithms Sebastian Ruder ODL 209 6,203 0 15 Sep 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 194,510 0 10 Dec 2015
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 1.7K 100,529 0 04 Sep 2014