On Biased Compression for Distributed Learning

27 February 2020

Papers citing "On Biased Compression for Distributed Learning"

39 / 39 papers shown

Title
Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis Zhijie Chen Qiaobo Li A. Banerjee FedML 48 0 0 11 Nov 2024
Trustworthiness of Stochastic Gradient Descent in Distributed Learning Hongyang Li Caesar Wu Mohammed Chadli Said Mammar Pascal Bouvry 66 1 0 28 Oct 2024
Communication-efficient Vertical Federated Learning via Compressed Error Feedback Pedro Valdeira João Xavier Cláudia Soares Yuejie Chi FedML 62 4 0 20 Jun 2024
LoCoDL: Communication-Efficient Distributed Learning with Local Training and Compression Laurent Condat Artavazd Maranjyan Peter Richtárik 86 5 0 07 Mar 2024
Non-asymptotic Analysis of Biased Adaptive Stochastic Approximation Sobihan Surendran Antoine Godichon-Baggioni Adeline Fermanian Sylvain Le Corff 73 1 0 05 Feb 2024
Lower Bounds and Accelerated Algorithms in Distributed Stochastic Optimization with Communication Compression Yutong He Xinmeng Huang Yiming Chen W. Yin Kun Yuan 55 7 0 12 May 2023
EF21 with Bells & Whistles: Practical Algorithmic Extensions of Modern Error Feedback Ilyas Fatkhullin Igor Sokolov Eduard A. Gorbunov Zhize Li Peter Richtárik 83 46 0 07 Oct 2021
EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedback Peter Richtárik Igor Sokolov Ilyas Fatkhullin 51 142 0 09 Jun 2021
Accordion: Adaptive Gradient Communication via Critical Learning Regime Identification Saurabh Agarwal Hongyi Wang Kangwook Lee Shivaram Venkataraman Dimitris Papailiopoulos 54 25 0 29 Oct 2020
Linearly Converging Error Compensated SGD Eduard A. Gorbunov D. Kovalev Dmitry Makarenko Peter Richtárik 191 78 0 23 Oct 2020
On the Convergence of SGD with Biased Gradients Ahmad Ajalloeian Sebastian U. Stich 36 85 0 31 Jul 2020
Unified Analysis of Stochastic Gradient Methods for Composite Convex and Smooth Optimization Ahmed Khaled Othmane Sebbouh Nicolas Loizou Robert Mansel Gower Peter Richtárik 83 46 0 20 Jun 2020
A Better Alternative to Error Feedback for Communication-Efficient Distributed Learning Samuel Horváth Peter Richtárik 40 61 0 19 Jun 2020
A Unified Analysis of Stochastic Gradient Methods for Nonconvex Federated Optimization Zhize Li Peter Richtárik FedML 57 36 0 12 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 467 41,106 0 28 May 2020
Advances and Open Problems in Federated Learning Peter Kairouz H. B. McMahan Brendan Avent A. Bellet M. Bennis ... Zheng Xu Qiang Yang Felix X. Yu Han Yu Sen Zhao FedML AI4CE 106 6,177 0 10 Dec 2019
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 237 42,038 0 03 Dec 2019
SCAFFOLD: Stochastic Controlled Averaging for Federated Learning Sai Praneeth Karimireddy Satyen Kale M. Mohri Sashank J. Reddi Sebastian U. Stich A. Suresh FedML 47 345 0 14 Oct 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 253 6,420 0 26 Sep 2019
Tighter Theory for Local SGD on Identical and Heterogeneous Data Ahmed Khaled Konstantin Mishchenko Peter Richtárik 53 432 0 10 Sep 2019
Federated Learning: Challenges, Methods, and Future Directions Tian Li Anit Kumar Sahu Ameet Talwalkar Virginia Smith FedML 83 4,470 0 21 Aug 2019
PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization Thijs Vogels Sai Praneeth Karimireddy Martin Jaggi 54 320 0 31 May 2019
A Unified Theory of SGD: Variance Reduction, Sampling, Quantization and Coordinate Descent Eduard A. Gorbunov Filip Hanzely Peter Richtárik 80 143 0 27 May 2019
Natural Compression for Distributed Deep Learning Samuel Horváth Chen-Yu Ho L. Horvath Atal Narayan Sahu Marco Canini Peter Richtárik 45 151 0 27 May 2019
Error Feedback Fixes SignSGD and other Gradient Compression Schemes Sai Praneeth Karimireddy Quentin Rebjock Sebastian U. Stich Martin Jaggi 49 496 0 28 Jan 2019
Distributed Learning with Compressed Gradient Differences Konstantin Mishchenko Eduard A. Gorbunov Martin Takáč Peter Richtárik 79 197 0 26 Jan 2019
Fast and Faster Convergence of SGD for Over-Parameterized Models and an Accelerated Perceptron Sharan Vaswani Francis R. Bach Mark Schmidt 50 297 0 16 Oct 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 917 93,936 0 11 Oct 2018
The Convergence of Sparsified Gradient Methods Dan Alistarh Torsten Hoefler M. Johansson Sarit Khirirat Nikola Konstantinov Cédric Renggli 112 491 0 27 Sep 2018
Sparsified SGD with Memory Sebastian U. Stich Jean-Baptiste Cordonnier Martin Jaggi 66 743 0 20 Sep 2018
Error Compensated Quantized SGD and its Applications to Large-scale Distributed Optimization Jiaxiang Wu Weidong Huang Junzhou Huang Tong Zhang 59 234 0 21 Jun 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 627 7,080 0 20 Apr 2018
3LC: Lightweight and Effective Traffic Compression for Distributed Machine Learning Hyeontaek Lim D. Andersen M. Kaminsky 106 70 0 21 Feb 2018
Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training Chengyue Wu Song Han Huizi Mao Yu Wang W. Dally 102 1,394 0 05 Dec 2017
TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning W. Wen Cong Xu Feng Yan Chunpeng Wu Yandan Wang Yiran Chen Hai Helen Li 123 985 0 22 May 2017
Federated Learning: Strategies for Improving Communication Efficiency Jakub Konecný H. B. McMahan Felix X. Yu Peter Richtárik A. Suresh Dave Bacon FedML 267 4,620 0 18 Oct 2016
Katyusha: The First Direct Acceleration of Stochastic Gradient Methods Zeyuan Allen-Zhu ODL 85 580 0 18 Mar 2016
Communication-Efficient Learning of Deep Networks from Decentralized Data H. B. McMahan Eider Moore Daniel Ramage S. Hampson Blaise Agüera y Arcas FedML 229 17,328 0 17 Feb 2016
Parallel Coordinate Descent Methods for Big Data Optimization Peter Richtárik Martin Takáč 83 487 0 04 Dec 2012