First Analysis of Local GD on Heterogeneous Data

10 September 2019

Ahmed Khaled

Konstantin Mishchenko

Peter Richtárik

FedML

ArXiv PDF HTML

Papers citing "First Analysis of Local GD on Heterogeneous Data"

45 / 45 papers shown

Title
FedPeWS: Personalized Warmup via Subnetworks for Enhanced Heterogeneous Federated Learning Nurbek Tastan Samuel Horváth Martin Takáč Karthik Nandakumar FedML 59 0 0 03 Oct 2024
Semi-Variance Reduction for Fair Federated Learning Saber Malekmohammadi Yaoliang Yu FedML 80 1 0 23 Jun 2024
Kimad: Adaptive Gradient Compression with Bandwidth Awareness Jihao Xin Ivan Ilin Shunkang Zhang Marco Canini Peter Richtárik 37 2 0 13 Dec 2023
Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedup Yan Sun Li Shen Hao Sun Liang Ding Dacheng Tao FedML 24 17 0 30 Jul 2023
Federated Multi-Sequence Stochastic Approximation with Local Hypergradient Estimation Davoud Ataee Tarzanagh Mingchen Li Pranay Sharma Samet Oymak 26 0 0 02 Jun 2023
Local SGD Accelerates Convergence by Exploiting Second Order Information of the Loss Function Linxuan Pan Shenghui Song FedML 25 2 0 24 May 2023
Federated TD Learning over Finite-Rate Erasure Channels: Linear Speedup under Markovian Sampling Nicolò Dal Fabbro A. Mitra George J. Pappas FedML 35 12 0 14 May 2023
Federated Temporal Difference Learning with Linear Function Approximation under Environmental Heterogeneity Han Wang A. Mitra Hamed Hassani George J. Pappas James Anderson FedML 29 21 0 04 Feb 2023
Communication-Efficient Local SGD with Age-Based Worker Selection Feng Zhu Jingjing Zhang Xin Wang 35 1 0 31 Oct 2022
GradSkip: Communication-Accelerated Local Gradient Methods with Better Computational Complexity A. Maranjyan M. Safaryan Peter Richtárik 34 13 0 28 Oct 2022
Sketching for First Order Method: Efficient Algorithm for Low-Bandwidth Channel and Vulnerability Zhao Song Yitan Wang Zheng Yu Licheng Zhang FedML 23 28 0 15 Oct 2022
On the Performance of Gradient Tracking with Local Updates Edward Duc Hien Nguyen Sulaiman A. Alghunaim Kun Yuan César A. Uribe 40 19 0 10 Oct 2022
STSyn: Speeding Up Local SGD with Straggler-Tolerant Synchronization Feng Zhu Jingjing Zhang Xin Wang 33 3 0 06 Oct 2022
Multi-Level Branched Regularization for Federated Learning Jinkyu Kim Geeho Kim Bohyung Han FedML 22 53 0 14 Jul 2022
Federated Latent Class Regression for Hierarchical Data Bin Yang T. Carette Masanobu Jimbo Shinya Maruyama FedML 20 0 0 22 Jun 2022
Federated Random Reshuffling with Compression and Variance Reduction Grigory Malinovsky Peter Richtárik FedML 27 10 0 08 May 2022
FedNest: Federated Bilevel, Minimax, and Compositional Optimization Davoud Ataee Tarzanagh Mingchen Li Christos Thrampoulidis Samet Oymak FedML 42 73 0 04 May 2022
Understanding A Class of Decentralized and Federated Optimization Algorithms: A Multi-Rate Feedback Control Perspective Xinwei Zhang Mingyi Hong N. Elia FedML 18 3 0 27 Apr 2022
ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally! Konstantin Mishchenko Grigory Malinovsky Sebastian U. Stich Peter Richtárik 11 151 0 18 Feb 2022
Speeding up Heterogeneous Federated Learning with Sequentially Trained Superclients Riccardo Zaccone Andrea Rizzardi Debora Caldarola Marco Ciccone Barbara Caputo FedML 58 14 0 26 Jan 2022
On Convergence of Federated Averaging Langevin Dynamics Wei Deng Qian Zhang Yi Ma Zhao Song Guang Lin FedML 30 16 0 09 Dec 2021
On the Convergence of Shallow Neural Network Training with Randomly Masked Neurons Fangshuo Liao Anastasios Kyrillidis 43 16 0 05 Dec 2021
Federated Semi-Supervised Learning with Class Distribution Mismatch Zhiguo Wang Xintong Wang Ruoyu Sun Tsung-Hui Chang FedML 21 12 0 29 Oct 2021
A Stochastic Newton Algorithm for Distributed Convex Optimization Brian Bullins Kumar Kshitij Patel Ohad Shamir Nathan Srebro Blake E. Woodworth 28 15 0 07 Oct 2021
Cost-Effective Federated Learning in Mobile Edge Networks Bing Luo Xiang Li Shiqiang Wang Jianwei Huang Leandros Tassiulas FedML 57 72 0 12 Sep 2021
Iterated Vector Fields and Conservatism, with Applications to Federated Learning Zachary B. Charles Keith Rush 27 6 0 08 Sep 2021
An Operator Splitting View of Federated Learning Saber Malekmohammadi K. Shaloudegi Zeou Hu Yaoliang Yu FedML 26 2 0 12 Aug 2021
Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacy Xinwei Zhang Xiangyi Chen Min-Fong Hong Zhiwei Steven Wu Jinfeng Yi FedML 30 91 0 25 Jun 2021
On Large-Cohort Training for Federated Learning Zachary B. Charles Zachary Garrett Zhouyuan Huo Sergei Shmulyian Virginia Smith FedML 21 113 0 15 Jun 2021
Decentralized Federated Averaging Tao Sun Dongsheng Li Bao Wang FedML 48 207 0 23 Apr 2021
Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradients A. Mitra Rayana H. Jaafar George J. Pappas Hamed Hassani FedML 55 157 0 14 Feb 2021
Vertical federated learning based on DFP and BFGS Wenjie Song Xuan Shen FedML 21 6 0 23 Jan 2021
Federated Learning under Importance Sampling Elsa Rizk Stefan Vlaski Ali H. Sayed FedML 18 52 0 14 Dec 2020
Optimization for Supervised Machine Learning: Randomized Algorithms for Data and Parameters Filip Hanzely 34 0 0 26 Aug 2020
Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization Jianyu Wang Qinghua Liu Hao Liang Gauri Joshi H. Vincent Poor MoMe FedML 16 1,297 0 15 Jul 2020
Minibatch vs Local SGD for Heterogeneous Distributed Learning Blake E. Woodworth Kumar Kshitij Patel Nathan Srebro FedML 22 199 0 08 Jun 2020
FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Data Xinwei Zhang Mingyi Hong S. Dhople W. Yin Yang Liu FedML 21 227 0 22 May 2020
Adaptive Federated Optimization Sashank J. Reddi Zachary B. Charles Manzil Zaheer Zachary Garrett Keith Rush Jakub Konecný Sanjiv Kumar H. B. McMahan FedML 20 1,391 0 29 Feb 2020
Acceleration for Compressed Gradient Descent in Distributed and Federated Optimization Zhize Li D. Kovalev Xun Qian Peter Richtárik FedML AI4CE 29 135 0 26 Feb 2020
Dynamic Federated Learning Elsa Rizk Stefan Vlaski Ali H. Sayed FedML 22 25 0 20 Feb 2020
Distributed Non-Convex Optimization with Sublinear Speedup under Intermittent Client Availability Yikai Yan Chaoyue Niu Yucheng Ding Zhenzhe Zheng Fan Wu Guihai Chen Shaojie Tang Zhihua Wu FedML 49 37 0 18 Feb 2020
On the Convergence of Local Descent Methods in Federated Learning Farzin Haddadpour M. Mahdavi FedML 19 266 0 31 Oct 2019
Gradient Descent with Compressed Iterates Ahmed Khaled Peter Richtárik 21 22 0 10 Sep 2019
On the Convergence of FedAvg on Non-IID Data Xiang Li Kaixuan Huang Wenhao Yang Shusen Wang Zhihua Zhang FedML 70 2,283 0 04 Jul 2019
Adaptive Federated Learning in Resource Constrained Edge Computing Systems Shiqiang Wang Tiffany Tuor Theodoros Salonidis K. Leung C. Makaya T. He Kevin S. Chan 144 1,687 0 14 Apr 2018