v1v2v3 (latest)

Natural Compression for Distributed Deep Learning

27 May 2019

Papers citing "Natural Compression for Distributed Deep Learning"

36 / 36 papers shown

Title
LoCoDL: Communication-Efficient Distributed Learning with Local Training and Compression Laurent Condat Artavazd Maranjyan Peter Richtárik 116 5 0 07 Mar 2024
Compressed Private Aggregation for Scalable and Robust Federated Learning over Massive Networks Natalie Lang Nir Shlezinger Rafael G. L. DÓliveira S. E. Rouayheb FedML 138 4 0 01 Aug 2023
Tighter Theory for Local SGD on Identical and Heterogeneous Data Ahmed Khaled Konstantin Mishchenko Peter Richtárik 74 434 0 10 Sep 2019
PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization Thijs Vogels Sai Praneeth Karimireddy Martin Jaggi 90 323 0 31 May 2019
Communication-Efficient Distributed Blockwise Momentum SGD with Error-Feedback Shuai Zheng Ziyue Huang James T. Kwok 54 115 0 27 May 2019
DoubleSqueeze: Parallel Stochastic Gradient Descent with Double-Pass Error-Compensated Compression Hanlin Tang Xiangru Lian Chen Yu Tong Zhang Ji Liu 60 220 0 15 May 2019
Error Feedback Fixes SignSGD and other Gradient Compression Schemes Sai Praneeth Karimireddy Quentin Rebjock Sebastian U. Stich Martin Jaggi 66 502 0 28 Jan 2019
99% of Distributed Optimization is a Waste of Time: The Issue and How to Fix it Konstantin Mishchenko Filip Hanzely Peter Richtárik 51 13 0 27 Jan 2019
Distributed Learning with Compressed Gradient Differences Konstantin Mishchenko Eduard A. Gorbunov Martin Takáč Peter Richtárik 98 201 0 26 Jan 2019
Distributed Learning with Sparse Communications by Identification Dmitry Grishchenko F. Iutzeler J. Malick Massih-Reza Amini 55 19 0 10 Dec 2018
The Convergence of Sparsified Gradient Methods Dan Alistarh Torsten Hoefler M. Johansson Sarit Khirirat Nikola Konstantinov Cédric Renggli 167 494 0 27 Sep 2018
Sparsified SGD with Memory Sebastian U. Stich Jean-Baptiste Cordonnier Martin Jaggi 82 754 0 20 Sep 2018
Distributed learning with compressed gradients Sarit Khirirat Hamid Reza Feyzmahdavian M. Johansson 56 84 0 18 Jun 2018
Local SGD Converges Fast and Communicates Little Sebastian U. Stich FedML 183 1,067 0 24 May 2018
Not All Samples Are Created Equal: Deep Learning with Importance Sampling Angelos Katharopoulos François Fleuret 101 522 0 02 Mar 2018
3LC: Lightweight and Effective Traffic Compression for Distributed Machine Learning Hyeontaek Lim D. Andersen M. Kaminsky 115 70 0 21 Feb 2018
signSGD: Compressed Optimisation for Non-Convex Problems Jeremy Bernstein Yu Wang Kamyar Azizzadenesheli Anima Anandkumar FedML ODL 101 1,050 0 13 Feb 2018
Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training Chengyue Wu Song Han Huizi Mao Yu Wang W. Dally 141 1,409 0 05 Dec 2017
Gradient Sparsification for Communication-Efficient Distributed Optimization Jianqiao Wangni Jialei Wang Ji Liu Tong Zhang 90 528 0 26 Oct 2017
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour Priya Goyal Piotr Dollár Ross B. Girshick P. Noordhuis Lukasz Wesolowski Aapo Kyrola Andrew Tulloch Yangqing Jia Kaiming He 3DH 128 3,685 0 08 Jun 2017
TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning W. Wen Cong Xu Feng Yan Chunpeng Wu Yandan Wang Yiran Chen Hai Helen Li 157 990 0 22 May 2017
Sparse Communication for Distributed Gradient Descent Alham Fikri Aji Kenneth Heafield 66 741 0 17 Apr 2017
Randomized Distributed Mean Estimation: Accuracy vs Communication Jakub Konecný Peter Richtárik FedML 120 102 0 22 Nov 2016
Distributed Mean Estimation with Limited Communication A. Suresh Felix X. Yu Sanjiv Kumar H. B. McMahan FedML 114 366 0 02 Nov 2016
Federated Learning: Strategies for Improving Communication Efficiency Jakub Konecný H. B. McMahan Felix X. Yu Peter Richtárik A. Suresh Dave Bacon FedML 309 4,649 0 18 Oct 2016
Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations Itay Hubara Matthieu Courbariaux Daniel Soudry Ran El-Yaniv Yoshua Bengio MQ 152 1,867 0 22 Sep 2016
Densely Connected Convolutional Networks Gao Huang Zhuang Liu Laurens van der Maaten Kilian Q. Weinberger PINN 3DV 775 36,861 0 25 Aug 2016
AIDE: Fast and Communication Efficient Distributed Optimization Sashank J. Reddi Jakub Konecný Peter Richtárik Barnabás Póczós Alex Smola 58 150 0 24 Aug 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 194,322 0 10 Dec 2015
Deep Learning with Limited Numerical Precision Suyog Gupta A. Agrawal K. Gopalakrishnan P. Narayanan HAI 207 2,049 0 09 Feb 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 2.0K 150,260 0 22 Dec 2014
SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives Aaron Defazio Francis R. Bach Simon Lacoste-Julien ODL 135 1,828 0 01 Jul 2014
Communication Efficient Distributed Optimization using an Approximate Newton-type Method Ohad Shamir Nathan Srebro Tong Zhang 95 555 0 30 Dec 2013
Distributed Coordinate Descent Method for Learning with Big Data Peter Richtárik Martin Takáč 191 255 0 08 Oct 2013
Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming Saeed Ghadimi Guanghui Lan ODL 122 1,555 0 22 Sep 2013
HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent Feng Niu Benjamin Recht Christopher Ré Stephen J. Wright 201 2,274 0 28 Jun 2011