v1v2 (latest)

ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!

18 February 2022

Konstantin Mishchenko

Papers citing "ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!"

42 / 42 papers shown

Title
Pseudo-Asynchronous Local SGD: Robust and Efficient Data-Parallel Training Hiroki Naganuma Xinzhi Zhang Man-Chung Yue Ioannis Mitliagkas Philipp A. Witte Russell J. Hewett Yin Tat Lee 222 0 0 25 Apr 2025
Revisiting LocalSGD and SCAFFOLD: Improved Rates and Missing Analysis Ruichen Luo Sebastian U Stich Samuel Horváth Martin Takáč 116 0 0 08 Jan 2025
Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis Zhijie Chen Qiaobo Li A. Banerjee FedML 90 0 0 11 Nov 2024
FedPeWS: Personalized Warmup via Subnetworks for Enhanced Heterogeneous Federated Learning Nurbek Tastan Samuel Horváth Martin Takáč Karthik Nandakumar FedML 128 0 0 03 Oct 2024
A-FedPD: Aligning Dual-Drift is All Federated Primal-Dual Learning Needs Yan Sun Li Shen Dacheng Tao FedML 78 0 0 27 Sep 2024
LoCoDL: Communication-Efficient Distributed Learning with Local Training and Compression Laurent Condat Artavazd Maranjyan Peter Richtárik 116 5 0 07 Mar 2024
Stochastic Controlled Averaging for Federated Learning with Communication Compression Xinmeng Huang Ping Li Xiaoyun Li 96 209 0 16 Aug 2023
Lower Bounds and Accelerated Algorithms in Distributed Stochastic Optimization with Communication Compression Yutong He Xinmeng Huang Yiming Chen W. Yin Kun Yuan 72 7 0 12 May 2023
SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization Kfir Y. Levy Kfir Y. Levy FedML 88 3 0 09 Apr 2023
Stochastic Gradient Descent-Ascent: Unified Theory and New Efficient Methods Aleksandr Beznosikov Eduard A. Gorbunov Hugo Berard Nicolas Loizou 73 50 0 15 Feb 2022
An Improved Analysis of Gradient Tracking for Decentralized Machine Learning Anastasia Koloskova Tao R. Lin Sebastian U. Stich 67 110 0 08 Feb 2022
RelaySum for Decentralized Deep Learning on Heterogeneous Data Thijs Vogels Lie He Anastasia Koloskova Tao R. Lin Sai Praneeth Karimireddy Sebastian U. Stich Martin Jaggi FedML MoE 51 62 0 08 Oct 2021
Lower Bounds and Optimal Algorithms for Smooth and Strongly Convex Decentralized Optimization Over Time-Varying Networks D. Kovalev Elnur Gasanov Peter Richtárik Alexander Gasnikov 53 44 0 08 Jun 2021
Removing Data Heterogeneity Influence Enhances Network Topology Dependence of Decentralized SGD Kun Yuan Sulaiman A. Alghunaim Xinmeng Huang 50 35 0 17 May 2021
ADOM: Accelerated Decentralized Optimization Method for Time-Varying Networks D. Kovalev Egor Shulgin Peter Richtárik Alexander Rogozin Alexander Gasnikov ODL 64 31 0 18 Feb 2021
Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradients A. Mitra Rayana H. Jaafar George J. Pappas Hamed Hassani FedML 101 159 0 14 Feb 2021
Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data Tao R. Lin Sai Praneeth Karimireddy Sebastian U. Stich Martin Jaggi FedML 89 101 0 09 Feb 2021
The Min-Max Complexity of Distributed Stochastic Convex Optimization with Intermittent Communication Blake E. Woodworth Brian Bullins Ohad Shamir Nathan Srebro 38 49 0 02 Feb 2021
Local SGD: Unified Theory and New Efficient Methods Eduard A. Gorbunov Filip Hanzely Peter Richtárik FedML 82 111 0 03 Nov 2020
Lower Bounds and Optimal Algorithms for Personalized Federated Learning Filip Hanzely Slavomír Hanzely Samuel Horváth Peter Richtárik FedML 120 190 0 05 Oct 2020
Distributed Proximal Splitting Algorithms with Rates and Acceleration Laurent Condat Grigory Malinovsky Peter Richtárik 36 21 0 02 Oct 2020
Optimal and Practical Algorithms for Smooth and Strongly Convex Decentralized Optimization D. Kovalev Adil Salim Peter Richtárik 43 83 0 21 Jun 2020
Federated Accelerated Stochastic Gradient Descent Honglin Yuan Tengyu Ma FedML 67 180 0 16 Jun 2020
Minibatch vs Local SGD for Heterogeneous Distributed Learning Blake E. Woodworth Kumar Kshitij Patel Nathan Srebro FedML 114 203 0 08 Jun 2020
From Local SGD to Local Fixed-Point Methods for Federated Learning Grigory Malinovsky D. Kovalev Elnur Gasanov Laurent Condat Peter Richtárik FedML 126 117 0 03 Apr 2020
A Unified Theory of Decentralized SGD with Changing Topology and Local Updates Anastasia Koloskova Nicolas Loizou Sadra Boreiri Martin Jaggi Sebastian U. Stich FedML 85 514 0 23 Mar 2020
Is Local SGD Better than Minibatch SGD? Blake E. Woodworth Kumar Kshitij Patel Sebastian U. Stich Zhen Dai Brian Bullins H. B. McMahan Ohad Shamir Nathan Srebro FedML 74 254 0 18 Feb 2020
Federated Learning of a Mixture of Global and Local Models Filip Hanzely Peter Richtárik FedML 59 385 0 10 Feb 2020
Better Theory for SGD in the Nonconvex World Ahmed Khaled Peter Richtárik 74 185 0 09 Feb 2020
Advances and Open Problems in Federated Learning Peter Kairouz H. B. McMahan Brendan Avent A. Bellet M. Bennis ... Zheng Xu Qiang Yang Felix X. Yu Han Yu Sen Zhao FedML AI4CE 259 6,276 0 10 Dec 2019
Tighter Theory for Local SGD on Identical and Heterogeneous Data Ahmed Khaled Konstantin Mishchenko Peter Richtárik 74 435 0 10 Sep 2019
First Analysis of Local GD on Heterogeneous Data Ahmed Khaled Konstantin Mishchenko Peter Richtárik FedML 72 172 0 10 Sep 2019
SGD: General Analysis and Improved Rates Robert Mansel Gower Nicolas Loizou Xun Qian Alibek Sailanbayev Egor Shulgin Peter Richtárik 84 380 0 27 Jan 2019
Don't Use Large Mini-Batches, Use Local SGD Tao R. Lin Sebastian U. Stich Kumar Kshitij Patel Martin Jaggi 119 432 0 22 Aug 2018
Local SGD Converges Fast and Communicates Little Sebastian U. Stich FedML 186 1,067 0 24 May 2018
D $^2$ : Decentralized Training over Decentralized Data Hanlin Tang Xiangru Lian Ming Yan Ce Zhang Ji Liu 39 352 0 19 Mar 2018
Ray: A Distributed Framework for Emerging AI Applications Philipp Moritz Robert Nishihara Stephanie Wang Alexey Tumanov Richard Liaw ... Melih Elibol Zongheng Yang William Paul Michael I. Jordan Ion Stoica GNN 105 1,266 0 16 Dec 2017
Federated Learning: Strategies for Improving Communication Efficiency Jakub Konecný H. B. McMahan Felix X. Yu Peter Richtárik A. Suresh Dave Bacon FedML 309 4,649 0 18 Oct 2016
Parallel SGD: When does averaging help? Jian Zhang Christopher De Sa Ioannis Mitliagkas Christopher Ré MoMe FedML 77 109 0 23 Jun 2016
NEXT: In-Network Nonconvex Optimization P. Lorenzo G. Scutari 103 508 0 01 Feb 2016
Communication Complexity of Distributed Convex Learning and Optimization Yossi Arjevani Ohad Shamir 97 209 0 05 Jun 2015
Optimal Distributed Online Prediction using Mini-Batches O. Dekel Ran Gilad-Bachrach Ohad Shamir Lin Xiao 273 685 0 07 Dec 2010