The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication

19 May 2024

Papers citing "The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication"

24 / 24 papers shown

Title
Revisiting LocalSGD and SCAFFOLD: Improved Rates and Missing Analysis Ruichen Luo Sebastian U Stich Samuel Horváth Martin Takáč 116 0 0 08 Jan 2025
FedExP: Speeding Up Federated Averaging via Extrapolation Divyansh Jhunjhunwala Shiqiang Wang Gauri Joshi FedML 57 61 0 23 Jan 2023
On-Demand Sampling: Learning Optimally from Multiple Distributions Nika Haghtalab Michael I. Jordan Eric Zhao FedML 118 36 0 22 Oct 2022
On the Unreasonable Effectiveness of Federated Averaging with Heterogeneous Data Jianyu Wang Rudrajit Das Gauri Joshi Satyen Kale Zheng Xu Tong Zhang FedML 82 40 0 09 Jun 2022
A Stochastic Newton Algorithm for Distributed Convex Optimization Brian Bullins Kumar Kshitij Patel Ohad Shamir Nathan Srebro Blake E. Woodworth 62 16 0 07 Oct 2021
FedChain: Chained Algorithms for Near-Optimal Communication Cost in Federated Learning Charlie Hou K. K. Thekumparampil Giulia Fanti Sewoong Oh FedML 66 14 0 16 Aug 2021
A Field Guide to Federated Optimization Jianyu Wang Zachary B. Charles Zheng Xu Gauri Joshi H. B. McMahan ... Mi Zhang Tong Zhang Chunxiang Zheng Chen Zhu Wennan Zhu FedML 273 421 0 14 Jul 2021
STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learning Prashant Khanduri Pranay Sharma Haibo Yang Min-Fong Hong Jia Liu K. Rajawat P. Varshney FedML 56 63 0 19 Jun 2021
On Large-Cohort Training for Federated Learning Zachary B. Charles Zachary Garrett Zhouyuan Huo Sergei Shmulyian Virginia Smith FedML 66 112 0 15 Jun 2021
The Min-Max Complexity of Distributed Stochastic Convex Optimization with Intermittent Communication Blake E. Woodworth Brian Bullins Ohad Shamir Nathan Srebro 38 49 0 02 Feb 2021
Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning Sai Praneeth Karimireddy Martin Jaggi Satyen Kale M. Mohri Sashank J. Reddi Sebastian U. Stich A. Suresh FedML 136 217 0 08 Aug 2020
On the Outsized Importance of Learning Rates in Local Update Methods Zachary B. Charles Jakub Konecný FedML 78 54 0 02 Jul 2020
Federated Accelerated Stochastic Gradient Descent Honglin Yuan Tengyu Ma FedML 73 180 0 16 Jun 2020
FedSplit: An algorithmic framework for fast federated optimization Reese Pathak Martin J. Wainwright FedML 217 184 0 11 May 2020
A Unified Theory of Decentralized SGD with Changing Topology and Local Updates Anastasia Koloskova Nicolas Loizou Sadra Boreiri Martin Jaggi Sebastian U. Stich FedML 85 514 0 23 Mar 2020
Is Local SGD Better than Minibatch SGD? Blake E. Woodworth Kumar Kshitij Patel Sebastian U. Stich Zhen Dai Brian Bullins H. B. McMahan Ohad Shamir Nathan Srebro FedML 74 254 0 18 Feb 2020
Advances and Open Problems in Federated Learning Peter Kairouz H. B. McMahan Brendan Avent A. Bellet M. Bennis ... Zheng Xu Qiang Yang Felix X. Yu Han Yu Sen Zhao FedML AI4CE 266 6,285 0 10 Dec 2019
Stochastic Newton and Cubic Newton Methods with Simple Local Linear-Quadratic Rates D. Kovalev Konstantin Mishchenko Peter Richtárik ODL 60 45 0 03 Dec 2019
Tighter Theory for Local SGD on Identical and Heterogeneous Data Ahmed Khaled Konstantin Mishchenko Peter Richtárik 74 435 0 10 Sep 2019
Introduction to Online Convex Optimization Elad Hazan OffRL 188 1,939 0 07 Sep 2019
Unified Optimal Analysis of the (Stochastic) Gradient Method Sebastian U. Stich 54 113 0 09 Jul 2019
Don't Use Large Mini-Batches, Use Local SGD Tao R. Lin Sebastian U. Stich Kumar Kshitij Patel Martin Jaggi 119 432 0 22 Aug 2018
Local SGD Converges Fast and Communicates Little Sebastian U. Stich FedML 187 1,069 0 24 May 2018
Communication-Efficient Learning of Deep Networks from Decentralized Data H. B. McMahan Eider Moore Daniel Ramage S. Hampson Blaise Agüera y Arcas FedML 408 17,593 0 17 Feb 2016