Communication-efficient SGD: From Local SGD to One-Shot Averaging

v1v2 (latest)

Communication-efficient SGD: From Local SGD to One-Shot Averaging

9 June 2021

Artin Spiridonoff

Alexander Olshevsky

ArXiv (abs)PDF HTML

Papers citing "Communication-efficient SGD: From Local SGD to One-Shot Averaging"

17 / 17 papers shown

Title
WASH: Train your Ensemble with Communication-Efficient Weight Shuffling, then Average Louis Fournier Adel Nabli Masih Aminbeidokhti M. Pedersoli Eugene Belilovsky Edouard Oyallon MoMe FedML 95 3 0 27 May 2024
Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey Feng Liang Zhen Zhang Haifeng Lu Victor C. M. Leung Yanyi Guo Xiping Hu GNN 103 8 0 09 Apr 2024
Distributed Continual Learning with CoCoA in High-dimensional Linear Regression Martin Hellkvist Ayça Özçelikkale Anders Ahlén OOD 83 1 0 04 Dec 2023
On the Communication Complexity of Decentralized Bilevel Optimization Yihan Zhang My T. Thai Jie Wu Hongchang Gao 121 4 0 19 Nov 2023
Dynamic Fair Federated Learning Based on Reinforcement Learning Weikang Chen Junping Du Yingxia Shao Jia Wang Yangxi Zhou FedML 69 3 0 02 Nov 2023
Stability and Generalization for Minibatch SGD and Local SGD Yunwen Lei Tao Sun Mingrui Liu 93 4 0 02 Oct 2023
FusionAI: Decentralized Training and Deploying LLMs with Massive Consumer-Level GPUs Zhenheng Tang Yuxin Wang Xin He Longteng Zhang Xinglin Pan ... Rongfei Zeng Kaiyong Zhao Shaoshuai Shi Bingsheng He Xiaowen Chu 106 30 0 03 Sep 2023
DIGEST: Fast and Communication Efficient Decentralized Learning with Local Updates Peyman Gholami H. Seferoglu FedML 66 12 0 14 Jul 2023
Adaptive Federated Learning via New Entropy Approach Shensheng Zheng Wenhao Yuan Xuehe Wang Ling-Yu Duan FedML OOD 93 1 0 27 Mar 2023
Clustered Federated Learning based on Nonconvex Pairwise Fusion Xue Yu Ziyi Liu Wu Wang Yifan Sun FedML 88 9 0 08 Nov 2022
Local SGD in Overparameterized Linear Regression Mike Nguyen Charly Kirst Nicole Mücke 58 0 0 20 Oct 2022
Fast Composite Optimization and Statistical Recovery in Federated Learning Yajie Bao M. Crawshaw Sha Luo Mingrui Liu FedML 89 17 0 17 Jul 2022
FedAvg with Fine Tuning: Local Updates Lead to Representation Learning Liam Collins Hamed Hassani Aryan Mokhtari Sanjay Shakkottai FedML 83 81 0 27 May 2022
On Arbitrary Compression for Decentralized Consensus and Stochastic Optimization over Directed Networks Taha Toghani César A. Uribe 52 1 0 18 Apr 2022
The Role of Local Steps in Local SGD Tiancheng Qin S. Rasoul Etesami César A. Uribe 65 4 0 14 Mar 2022
Federated Minimax Optimization: Improved Convergence Analyses and Algorithms Pranay Sharma Rohan Panda Gauri Joshi P. Varshney FedML 112 49 0 09 Mar 2022
Communication-Efficient Distributed Deep Learning: A Comprehensive Survey Zhenheng Tang Shaoshuai Shi Wei Wang Yue Liu Xiaowen Chu 83 49 0 10 Mar 2020