Communication-efficient SGD: From Local SGD to One-Shot Averaging

v1v2 (latest)

Communication-efficient SGD: From Local SGD to One-Shot Averaging

9 June 2021

Artin Spiridonoff

Alexander Olshevsky

ArXiv (abs)PDF HTML

Papers citing "Communication-efficient SGD: From Local SGD to One-Shot Averaging"

17 / 17 papers shown

Title
WASH: Train your Ensemble with Communication-Efficient Weight Shuffling, then Average Louis Fournier Adel Nabli Masih Aminbeidokhti M. Pedersoli Eugene Belilovsky Edouard Oyallon MoMe FedML 95 3 0 27 May 2024
Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey Feng Liang Zhen Zhang Haifeng Lu Victor C. M. Leung Yanyi Guo Xiping Hu GNN 103 8 0 09 Apr 2024
Distributed Continual Learning with CoCoA in High-dimensional Linear Regression Martin Hellkvist Ayça Özçelikkale Anders Ahlén OOD 83 1 0 04 Dec 2023
On the Communication Complexity of Decentralized Bilevel Optimization Yihan Zhang My T. Thai Jie Wu Hongchang Gao 119 4 0 19 Nov 2023
Dynamic Fair Federated Learning Based on Reinforcement Learning Weikang Chen Junping Du Yingxia Shao Jia Wang Yangxi Zhou FedML 69 3 0 02 Nov 2023
Stability and Generalization for Minibatch SGD and Local SGD Yunwen Lei Tao Sun Mingrui Liu 93 4 0 02 Oct 2023
FusionAI: Decentralized Training and Deploying LLMs with Massive Consumer-Level GPUs Zhenheng Tang Yuxin Wang Xin He Longteng Zhang Xinglin Pan ... Rongfei Zeng Kaiyong Zhao Shaoshuai Shi Bingsheng He Xiaowen Chu 106 30 0 03 Sep 2023
DIGEST: Fast and Communication Efficient Decentralized Learning with Local Updates Peyman Gholami H. Seferoglu FedML 66 12 0 14 Jul 2023
Adaptive Federated Learning via New Entropy Approach Shensheng Zheng Wenhao Yuan Xuehe Wang Ling-Yu Duan FedML OOD 93 1 0 27 Mar 2023
Clustered Federated Learning based on Nonconvex Pairwise Fusion Xue Yu Ziyi Liu Wu Wang Yifan Sun FedML 86 9 0 08 Nov 2022
Local SGD in Overparameterized Linear Regression Mike Nguyen Charly Kirst Nicole Mücke 58 0 0 20 Oct 2022
Fast Composite Optimization and Statistical Recovery in Federated Learning Yajie Bao M. Crawshaw Sha Luo Mingrui Liu FedML 89 17 0 17 Jul 2022
FedAvg with Fine Tuning: Local Updates Lead to Representation Learning Liam Collins Hamed Hassani Aryan Mokhtari Sanjay Shakkottai FedML 83 81 0 27 May 2022
On Arbitrary Compression for Decentralized Consensus and Stochastic Optimization over Directed Networks Taha Toghani César A. Uribe 52 1 0 18 Apr 2022
The Role of Local Steps in Local SGD Tiancheng Qin S. Rasoul Etesami César A. Uribe 65 4 0 14 Mar 2022
Federated Minimax Optimization: Improved Convergence Analyses and Algorithms Pranay Sharma Rohan Panda Gauri Joshi P. Varshney FedML 112 49 0 09 Mar 2022
Communication-Efficient Distributed Deep Learning: A Comprehensive Survey Zhenheng Tang Shaoshuai Shi Wei Wang Yue Liu Xiaowen Chu 83 49 0 10 Mar 2020

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.