Data-heterogeneity-aware Mixing for Decentralized Learning

13 April 2022

Papers citing "Data-heterogeneity-aware Mixing for Decentralized Learning"

41 / 41 papers shown

Title
Distribution-Aware Mobility-Assisted Decentralized Federated Learning Md. Farhamdur Reza Reza Jahani Richeng Jin H. Dai 52 0 0 24 May 2025
An Improved Analysis of Gradient Tracking for Decentralized Machine Learning Anastasia Koloskova Tao R. Lin Sebastian U. Stich 67 110 0 08 Feb 2022
A Unified and Refined Convergence Analysis for Non-Convex Decentralized Learning Sulaiman A. Alghunaim Kun Yuan 78 63 0 19 Oct 2021
RelaySum for Decentralized Deep Learning on Heterogeneous Data Thijs Vogels Lie He Anastasia Koloskova Tao R. Lin Sai Praneeth Karimireddy Sebastian U. Stich Martin Jaggi FedML MoE 51 62 0 08 Oct 2021
A Field Guide to Federated Optimization Jianyu Wang Zachary B. Charles Zheng Xu Gauri Joshi H. B. McMahan ... Mi Zhang Tong Zhang Chunxiang Zheng Chen Zhu Wennan Zhu FedML 273 421 0 14 Jul 2021
Implicit Gradient Alignment in Distributed and Federated Learning Yatin Dandi Luis Barba Martin Jaggi FedML 103 35 0 25 Jun 2021
Removing Data Heterogeneity Influence Enhances Network Topology Dependence of Decentralized SGD Kun Yuan Sulaiman A. Alghunaim Xinmeng Huang 50 35 0 17 May 2021
DecentLaM: Decentralized Momentum SGD for Large-batch Deep Training Kun Yuan Yiming Chen Xinmeng Huang Yingya Zhang Pan Pan Yinghui Xu W. Yin MoE 97 64 0 24 Apr 2021
Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradients A. Mitra Rayana H. Jaafar George J. Pappas Hamed Hassani FedML 101 160 0 14 Feb 2021
Consensus Control for Decentralized Deep Learning Lingjing Kong Tao R. Lin Anastasia Koloskova Martin Jaggi Sebastian U. Stich 41 79 0 09 Feb 2021
Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data Tao R. Lin Sai Praneeth Karimireddy Sebastian U. Stich Martin Jaggi FedML 89 101 0 09 Feb 2021
Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization Jianyu Wang Qinghua Liu Hao Liang Gauri Joshi H. Vincent Poor MoMe FedML 68 1,348 0 15 Jul 2020
Optimal Complexity in Decentralized Training Yucheng Lu Christopher De Sa 94 75 0 15 Jun 2020
Minibatch vs Local SGD for Heterogeneous Distributed Learning Blake E. Woodworth Kumar Kshitij Patel Nathan Srebro FedML 114 204 0 08 Jun 2020
Evolving Normalization-Activation Layers Hanxiao Liu Andrew Brock Karen Simonyan Quoc V. Le 96 80 0 06 Apr 2020
A Unified Theory of Decentralized SGD with Changing Topology and Local Updates Anastasia Koloskova Nicolas Loizou Sadra Boreiri Martin Jaggi Sebastian U. Stich FedML 85 514 0 23 Mar 2020
Decentralized gradient methods: does topology matter? Giovanni Neglia Chuan Xu Don Towsley G. Calbi 64 52 0 28 Feb 2020
Advances and Open Problems in Federated Learning Peter Kairouz H. B. McMahan Brendan Avent A. Bellet M. Bennis ... Zheng Xu Qiang Yang Felix X. Yu Han Yu Sen Zhao FedML AI4CE 264 6,285 0 10 Dec 2019
Communication-Efficient Local Decentralized SGD Methods Xiang Li Wenhao Yang Shusen Wang Zhihua Zhang 74 53 0 21 Oct 2019
SCAFFOLD: Stochastic Controlled Averaging for Federated Learning Sai Praneeth Karimireddy Satyen Kale M. Mohri Sashank J. Reddi Sebastian U. Stich A. Suresh FedML 65 347 0 14 Oct 2019
Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints Felix Sattler K. Müller Wojciech Samek FedML 159 1,008 0 04 Oct 2019
The Non-IID Data Quagmire of Decentralized Machine Learning Kevin Hsieh Amar Phanishayee O. Mutlu Phillip B. Gibbons 163 572 0 01 Oct 2019
Decentralized Deep Learning with Arbitrary Communication Compression Anastasia Koloskova Tao R. Lin Sebastian U. Stich Martin Jaggi FedML 48 235 0 22 Jul 2019
Bayesian Nonparametric Federated Learning of Neural Networks Mikhail Yurochkin Mayank Agarwal S. Ghosh Kristjan Greenewald T. Hoang Y. Khazaeni FedML 129 730 0 28 May 2019
MATCHA: Speeding Up Decentralized SGD via Matching Decomposition Sampling Jianyu Wang Anit Kumar Sahu Zhouyi Yang Gauri Joshi S. Kar 71 163 0 23 May 2019
On the Influence of Bias-Correction on Distributed Stochastic Optimization Kun Yuan Sulaiman A. Alghunaim Bicheng Ying Ali H. Sayed 76 64 0 26 Mar 2019
Semi-supervised Learning in Network-Structured Data via Total Variation Minimization A. Jung A. Hero III Alexandru Mara Saeed Jahromi Ayelet Heimowitz Yonina C. Eldar 58 31 0 28 Jan 2019
Stochastic Gradient Push for Distributed Deep Learning Mahmoud Assran Nicolas Loizou Nicolas Ballas Michael G. Rabbat 79 348 0 27 Nov 2018
Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms Jianyu Wang Gauri Joshi 170 349 0 22 Aug 2018
Distributed Stochastic Gradient Tracking Methods Shi Pu A. Nedić 75 293 0 25 May 2018
D $^2$ : Decentralized Training over Decentralized Data Hanlin Tang Xiangru Lian Ming Yan Ce Zhang Ji Liu 39 352 0 19 Mar 2018
Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning Pinxin Long Tingxiang Fan X. Liao Wenxi Liu Huatian Zhang Jia Pan OOD 77 457 0 28 Sep 2017
Network Topology and Communication-Computation Tradeoffs in Decentralized Optimization A. Nedić Alexander Olshevsky Michael G. Rabbat 74 513 0 26 Sep 2017
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour Priya Goyal Piotr Dollár Ross B. Girshick P. Noordhuis Lukasz Wesolowski Aapo Kyrola Andrew Tulloch Yangqing Jia Kaiming He 3DH 128 3,685 0 08 Jun 2017
Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent Xiangru Lian Ce Zhang Huan Zhang Cho-Jui Hsieh Wei Zhang Ji Liu 50 1,235 0 25 May 2017
Optimal algorithms for smooth and strongly convex distributed optimization in networks Kevin Scaman Francis R. Bach Sébastien Bubeck Y. Lee Laurent Massoulié 74 331 0 28 Feb 2017
Layer Normalization Jimmy Lei Ba J. Kiros Geoffrey E. Hinton 423 10,531 0 21 Jul 2016
NEXT: In-Network Nonconvex Optimization P. Lorenzo G. Scutari 103 509 0 01 Feb 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 194,426 0 10 Dec 2015
Character-level Convolutional Networks for Text Classification Xiang Zhang Jiaqi Zhao Yann LeCun 268 6,130 0 04 Sep 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 2.0K 150,312 0 22 Dec 2014