An Improved Analysis of Gradient Tracking for Decentralized Machine Learning

8 February 2022

Papers citing "An Improved Analysis of Gradient Tracking for Decentralized Machine Learning"

36 / 36 papers shown

Title
Fully First-Order Methods for Decentralized Bilevel Optimization Xiaoyu Wang Xuxing Chen Shiqian Ma Tong Zhang 88 0 0 25 Oct 2024
SADDLe: Sharpness-Aware Decentralized Deep Learning with Heterogeneous Data Sakshi Choudhary Sai Aparna Aketi Kaushik Roy FedML 80 0 0 22 May 2024
Robust Decentralized Learning with Local Updates and Gradient Tracking Sajjad Ghiasvand Amirhossein Reisizadeh Mahnoosh Alizadeh Ramtin Pedarsani 90 4 0 02 May 2024
Communication-Efficient Federated Optimization over Semi-Decentralized Networks He Wang Yuejie Chi FedML 110 2 0 30 Nov 2023
Distributed Random Reshuffling Methods with Improved Convergence Kun-Yen Huang Linli Zhou Shi Pu 81 4 0 21 Jun 2023
RelaySum for Decentralized Deep Learning on Heterogeneous Data Thijs Vogels Lie He Anastasia Koloskova Tao R. Lin Sai Praneeth Karimireddy Sebastian U. Stich Martin Jaggi FedML MoE 47 61 0 08 Oct 2021
Removing Data Heterogeneity Influence Enhances Network Topology Dependence of Decentralized SGD Kun Yuan Sulaiman A. Alghunaim Xinmeng Huang 50 34 0 17 May 2021
DecentLaM: Decentralized Momentum SGD for Large-batch Deep Training Kun Yuan Yiming Chen Xinmeng Huang Yingya Zhang Pan Pan Yinghui Xu W. Yin MoE 84 64 0 24 Apr 2021
Consensus Control for Decentralized Deep Learning Lingjing Kong Tao R. Lin Anastasia Koloskova Martin Jaggi Sebastian U. Stich 31 79 0 09 Feb 2021
Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data Tao R. Lin Sai Praneeth Karimireddy Sebastian U. Stich Martin Jaggi FedML 73 101 0 09 Feb 2021
A Linearly Convergent Algorithm for Decentralized Optimization: Sending Less Bits for Free! D. Kovalev Anastasia Koloskova Martin Jaggi Peter Richtárik Sebastian U. Stich 62 75 0 03 Nov 2020
A Unified Theory of Decentralized SGD with Changing Topology and Local Updates Anastasia Koloskova Nicolas Loizou Sadra Boreiri Martin Jaggi Sebastian U. Stich FedML 78 506 0 23 Mar 2020
FedDANE: A Federated Newton-Type Method Tian Li Anit Kumar Sahu Manzil Zaheer Maziar Sanjabi Ameet Talwalkar Virginia Smith FedML 138 156 0 07 Jan 2020
Advances and Open Problems in Federated Learning Peter Kairouz H. B. McMahan Brendan Avent A. Bellet M. Bennis ... Zheng Xu Qiang Yang Felix X. Yu Han Yu Sen Zhao FedML AI4CE 232 6,247 0 10 Dec 2019
SCAFFOLD: Stochastic Controlled Averaging for Federated Learning Sai Praneeth Karimireddy Satyen Kale M. Mohri Sashank J. Reddi Sebastian U. Stich A. Suresh FedML 65 346 0 14 Oct 2019
$$\texttt{DeepSqueeze}$: Decentralization Meets Error-Compensated Compression$ $\texttt{DeepSqueeze}$ : Decentralization Meets Error-Compensated Compression Hanlin Tang Xiangru Lian Shuang Qiu Lei Yuan Ce Zhang Tong Zhang Liu 33 49 0 17 Jul 2019
Unified Optimal Analysis of the (Stochastic) Gradient Method Sebastian U. Stich 54 113 0 09 Jul 2019
On the Convergence of FedAvg on Non-IID Data Xiang Li Kaixuan Huang Wenhao Yang Shusen Wang Zhihua Zhang FedML 138 2,333 0 04 Jul 2019
MATCHA: Speeding Up Decentralized SGD via Matching Decomposition Sampling Jianyu Wang Anit Kumar Sahu Zhouyi Yang Gauri Joshi S. Kar 60 163 0 23 May 2019
Distributed stochastic optimization with gradient tracking over strongly-connected networks Ran Xin Anit Kumar Sahu U. Khan S. Kar 68 112 0 18 Mar 2019
Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communication Anastasia Koloskova Sebastian U. Stich Martin Jaggi FedML 79 510 0 01 Feb 2019
Stochastic Gradient Push for Distributed Deep Learning Mahmoud Assran Nicolas Loizou Nicolas Ballas Michael G. Rabbat 73 345 0 27 Nov 2018
New Convergence Aspects of Stochastic Gradient Algorithms Lam M. Nguyen Phuong Ha Nguyen Peter Richtárik K. Scheinberg Martin Takáč Marten van Dijk 102 66 0 10 Nov 2018
COLA: Decentralized Linear Learning Lie He An Bian Martin Jaggi 80 120 0 13 Aug 2018
Distributed Stochastic Gradient Tracking Methods Shi Pu A. Nedić 64 291 0 25 May 2018
D $^2$ : Decentralized Training over Decentralized Data Hanlin Tang Xiangru Lian Ming Yan Ce Zhang Ji Liu 31 350 0 19 Mar 2018
Communication Compression for Decentralized Training Hanlin Tang Shaoduo Gan Ce Zhang Tong Zhang Ji Liu 55 273 0 17 Mar 2018
Network Topology and Communication-Computation Tradeoffs in Decentralized Optimization A. Nedić Alexander Olshevsky Michael G. Rabbat 58 509 0 26 Sep 2017
Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent Xiangru Lian Ce Zhang Huan Zhang Cho-Jui Hsieh Wei Zhang Ji Liu 50 1,227 0 25 May 2017
Federated Optimization: Distributed Machine Learning for On-Device Intelligence Jakub Konecný H. B. McMahan Daniel Ramage Peter Richtárik FedML 126 1,897 0 08 Oct 2016
Geometrically Convergent Distributed Optimization with Uncoordinated Step-Sizes A. Nedić Alexander Olshevsky Wei Shi César A. Uribe 55 140 0 19 Sep 2016
Communication-Efficient Learning of Deep Networks from Decentralized Data H. B. McMahan Eider Moore Daniel Ramage S. Hampson Blaise Agüera y Arcas FedML 394 17,453 0 17 Feb 2016
NEXT: In-Network Nonconvex Optimization P. Lorenzo G. Scutari 91 508 0 01 Feb 2016
Asynchronous Distributed Optimization using a Randomized Alternating Direction Method of Multipliers F. Iutzeler Pascal Bianchi P. Ciblat W. Hachem 73 181 0 12 Mar 2013
Distributed optimization over time-varying directed graphs A. Nedić Alexander Olshevsky 61 998 0 10 Mar 2013
Optimal Distributed Online Prediction using Mini-Batches O. Dekel Ran Gilad-Bachrach Ohad Shamir Lin Xiao 259 685 0 07 Dec 2010