Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent

25 May 2017

Papers citing "Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent"

39 / 39 papers shown

Title
Dyn-D $^2$ P: Dynamic Differentially Private Decentralized Learning with Provable Utility Guarantee Zehan Zhu Yan Huang Xin Wang Shouling Ji Jinming Xu 62 0 0 10 May 2025
Communication-Efficient Personalized Distributed Learning with Data and Node Heterogeneity Zhuojun Tian Zhaoyang Zhang Yiwei Li Mehdi Bennis 111 0 0 24 Apr 2025
PDSL: Privacy-Preserved Decentralized Stochastic Learning with Heterogeneous Data Distribution Lina Wang Yunsheng Yuan Chunxiao Wang Feng Li FedML 70 0 0 31 Mar 2025
RAMEN: Real-time Asynchronous Multi-agent Neural Implicit Mapping Hongrui Zhao Boris Ivanovic Negar Mehr 122 0 0 26 Feb 2025
Robust Federated Learning in Unreliable Wireless Networks: A Client Selection Approach Yansen Wang Wenkai Ji Jian Zhou Fu Xiao Tsung-Hui Chang 61 0 0 24 Feb 2025
A Bias-Correction Decentralized Stochastic Gradient Algorithm with Momentum Acceleration Yuchen Hu Xi Chen Weidong Liu Xiaojun Mao 78 0 0 31 Jan 2025
Revisiting LocalSGD and SCAFFOLD: Improved Rates and Missing Analysis Ruichen Luo Sebastian U Stich Samuel Horváth Martin Takáč 71 0 0 08 Jan 2025
Gradient Purification: Defense Against Poisoning Attack in Decentralized Federated Learning Bin Li Xiaoye Miao Yongheng Shang Xinkui Zhao AAML 74 0 0 08 Jan 2025
Deep-Relative-Trust-Based Diffusion for Decentralized Deep Learning Muyun Li Aaron Fainman Stefan Vlaski 58 0 0 06 Jan 2025
Disentangling data distribution for Federated Learning Xinyuan Zhao Hanlin Gu Lixin Fan Qiang Yang Yuxing Han OOD FedML 80 0 0 31 Dec 2024
EDiT: A Local-SGD-Based Efficient Distributed Training Method for Large Language Models Jialiang Cheng Ning Gao Yun Yue Zhiling Ye Jiadi Jiang Jian Sha OffRL 111 0 0 10 Dec 2024
Distributed Sign Momentum with Local Steps for Training Transformers Shuhua Yu Ding Zhou Cong Xie An Xu Zhi-Li Zhang Xin Liu S. Kar 92 0 0 26 Nov 2024
Attribute Inference Attacks for Federated Regression Tasks Francesco Diana Othmane Marfoq Chuan Xu Giovanni Neglia F. Giroire Eoin Thomas AAML 462 1 0 19 Nov 2024
Fully First-Order Methods for Decentralized Bilevel Optimization Xiaoyu Wang Xuxing Chen Shiqian Ma Tong Zhang 77 0 0 25 Oct 2024
Fair Decentralized Learning Sayan Biswas Anne-Marie Kermarrec Rishi Sharma Thibaud Trinca M. Vos FedML 90 0 0 03 Oct 2024
Communication-efficient Vertical Federated Learning via Compressed Error Feedback Pedro Valdeira João Xavier Cláudia Soares Yuejie Chi FedML 65 4 0 20 Jun 2024
SADDLe: Sharpness-Aware Decentralized Deep Learning with Heterogeneous Data Sakshi Choudhary Sai Aparna Aketi Kaushik Roy FedML 70 0 0 22 May 2024
FACT or Fiction: Can Truthful Mechanisms Eliminate Federated Free Riding? Marco Bornstein Amrit Singh Bedi Abdirisak Mohamed Furong Huang FedML 64 0 0 22 May 2024
Faster Convergence with Less Communication: Broadcast-Based Subgraph Sampling for Decentralized Learning over Wireless Networks Daniel Pérez Herrera Zheng Chen Erik G. Larsson 96 1 0 24 Jan 2024
Revolutionizing Wireless Networks with Federated Learning: A Comprehensive Review Sajjad Emdadi Mahdimahalleh AI4CE 58 0 0 01 Aug 2023
Distributed Random Reshuffling Methods with Improved Convergence Kun-Yen Huang Linli Zhou Shi Pu 67 4 0 21 Jun 2023
Push-SAGA: A decentralized stochastic algorithm with variance reduction over directed graphs Muhammad I. Qureshi Ran Xin S. Kar U. Khan 60 20 0 13 Aug 2020
Communication optimization strategies for distributed deep neural network training: A survey Shuo Ouyang Dezun Dong Yemao Xu Liquan Xiao 72 12 0 06 Mar 2020
A Structured Self-attentive Sentence Embedding Zhouhan Lin Minwei Feng Cicero Nogueira dos Santos Mo Yu Bing Xiang Bowen Zhou Yoshua Bengio 106 2,132 0 09 Mar 2017
Communication-Efficient Algorithms for Decentralized and Stochastic Optimization Guanghui Lan Soomin Lee Yi Zhou 66 218 0 14 Jan 2017
GaDei: On Scale-up Training As A Service For Deep Learning Wei Zhang Minwei Feng Yunhui Zheng Yufei Ren Yandong Wang ... Peng Liu Bing Xiang Li Zhang Bowen Zhou Fei Wang ALM 37 10 0 18 Nov 2016
Revisiting Distributed Synchronous SGD Jianmin Chen Xinghao Pan R. Monga Samy Bengio Rafal Jozefowicz 64 799 0 04 Apr 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.4K 192,638 0 10 Dec 2015
Staleness-aware Async-SGD for Distributed Deep Learning Wei Zhang Suyog Gupta Xiangru Lian Ji Liu 50 266 0 18 Nov 2015
Model Accuracy and Runtime Tradeoff in Distributed Deep Learning:A Systematic Study Suyog Gupta Wei Zhang Fei Wang 42 170 0 14 Sep 2015
Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization Xiangru Lian Yijun Huang Y. Li Ji Liu 135 499 0 27 Jun 2015
An Asynchronous Mini-Batch Algorithm for Regularized Stochastic Optimization Hamid Reza Feyzmahdavian Arda Aytekin M. Johansson 41 117 0 18 May 2015
Deep learning with Elastic Averaging SGD Sixin Zhang A. Choromańska Yann LeCun FedML 60 609 0 20 Dec 2014
Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming Saeed Ghadimi Guanghui Lan ODL 84 1,538 0 22 Sep 2013
Performance of a Distributed Stochastic Approximation Algorithm Pascal Bianchi G. Fort W. Hachem 66 117 0 07 Mar 2012
HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent Feng Niu Benjamin Recht Christopher Ré Stephen J. Wright 142 2,272 0 28 Jun 2011
Distributed Delayed Stochastic Optimization Alekh Agarwal John C. Duchi 113 626 0 28 Apr 2011
Optimal Distributed Online Prediction using Mini-Batches O. Dekel Ran Gilad-Bachrach Ohad Shamir Lin Xiao 241 683 0 07 Dec 2010
Distributed Autonomous Online Learning: Regrets and Intrinsic Privacy-Preserving Properties Feng Yan S. Sundaram S. V. N. Vishwanathan Y. Qi 75 268 0 21 Jun 2010