On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimization

9 May 2019

Papers citing "On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimization"

50 / 79 papers shown

Title
PDSL: Privacy-Preserved Decentralized Stochastic Learning with Heterogeneous Data Distribution Lina Wang Yunsheng Yuan Chunxiao Wang Feng Li FedML 43 0 0 31 Mar 2025
A Bias-Correction Decentralized Stochastic Gradient Algorithm with Momentum Acceleration Yuchen Hu Xi Chen Weidong Liu Xiaojun Mao 57 0 0 31 Jan 2025
On the Performance Analysis of Momentum Method: A Frequency Domain Perspective Xianliang Li Jun Luo Zhiwei Zheng Hanxiao Wang Li Luo Lingkun Wen Linlong Wu Sheng Xu 74 0 0 29 Nov 2024
Distributed Sign Momentum with Local Steps for Training Transformers Shuhua Yu Ding Zhou Cong Xie An Xu Zhi-Li Zhang Xin Liu S. Kar 69 0 0 26 Nov 2024
DEPT: Decoupled Embeddings for Pre-training Language Models Alex Iacob Lorenzo Sani Meghdad Kurmanji William F. Shen Xinchi Qiu Dongqi Cai Yan Gao Nicholas D. Lane VLM 168 0 0 07 Oct 2024
Hierarchical Learning and Computing over Space-Ground Integrated Networks Jingyang Zhu Yuanming Shi Yong Zhou Chunxiao Jiang Linling Kuang 31 2 0 26 Aug 2024
Ordered Momentum for Asynchronous SGD Chang-Wei Shi Yi-Rui Yang Wu-Jun Li ODL 59 0 0 27 Jul 2024
A New Theoretical Perspective on Data Heterogeneity in Federated Optimization Jiayi Wang Shiqiang Wang Rong-Rong Chen Mingyue Ji FedML 30 1 0 22 Jul 2024
DIMAT: Decentralized Iterative Merging-And-Training for Deep Learning Models Nastaran Saadati Minh Pham Nasla Saleem Joshua R. Waite Aditya Balu Zhanhong Jiang Chinmay Hegde Soumik Sarkar MoMe 47 1 0 11 Apr 2024
Convergence of Decentralized Stochastic Subgradient-based Methods for Nonsmooth Nonconvex functions Siyuan Zhang Nachuan Xiao Xin Liu 61 1 0 18 Mar 2024
Non-Convex Stochastic Composite Optimization with Polyak Momentum Yuan Gao Anton Rodomanov Sebastian U. Stich 37 6 0 05 Mar 2024
FedDRO: Federated Compositional Optimization for Distributionally Robust Learning Prashant Khanduri Chengyin Li Rafi Ibn Sultan Yao Qiang Joerg Kliewer Dongxiao Zhu 42 0 0 21 Nov 2023
Asynchronous Federated Learning with Bidirectional Quantized Communications and Buffered Aggregation Tomàs Ortega Hamid Jafarkhani FedML 30 6 0 01 Aug 2023
Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedup Yan Sun Li Shen Hao Sun Liang Ding Dacheng Tao FedML 24 17 0 30 Jul 2023
On the Optimal Batch Size for Byzantine-Robust Distributed Learning Yi-Rui Yang Chang-Wei Shi Wu-Jun Li FedML AAML 24 0 0 23 May 2023
Beyond Exponential Graph: Communication-Efficient Topologies for Decentralized Learning via Finite-time Convergence Yuki Takezawa Ryoma Sato Han Bao Kenta Niwa M. Yamada 31 9 0 19 May 2023
Lower Bounds and Accelerated Algorithms in Distributed Stochastic Optimization with Communication Compression Yutong He Xinmeng Huang Yiming Chen W. Yin Kun Yuan 31 7 0 12 May 2023
CoDeC: Communication-Efficient Decentralized Continual Learning Sakshi Choudhary Sai Aparna Aketi Gobinda Saha Kaushik Roy CLL 50 3 0 27 Mar 2023
Decentralized Learning Made Practical with Client Sampling M. Vos Akash Dhasade Anne-Marie Kermarrec Erick Lavoie J. Pouwelse Rishi Sharma 29 1 0 27 Feb 2023
On the Convergence of Federated Averaging with Cyclic Client Participation Yae Jee Cho Pranay Sharma Gauri Joshi Zheng Xu Satyen Kale Tong Zhang FedML 44 27 0 06 Feb 2023
Distributed Stochastic Optimization under a General Variance Condition Kun-Yen Huang Xiao Li Shin-Yi Pu FedML 40 6 0 30 Jan 2023
Graph Federated Learning with Hidden Representation Sharing Shuang Wu Mingxuan Zhang Yuantong Li Carl Yang Pan Li FedML 27 1 0 23 Dec 2022
Decentralized Stochastic Gradient Descent Ascent for Finite-Sum Minimax Problems Hongchang Gao 24 16 0 06 Dec 2022
Analysis of Error Feedback in Federated Non-Convex Optimization with Biased Compression Xiaoyun Li Ping Li FedML 34 4 0 25 Nov 2022
FedCut: A Spectral Analysis Framework for Reliable Detection of Byzantine Colluders Hanlin Gu Lixin Fan Xingxing Tang Qiang Yang AAML FedML 22 1 0 24 Nov 2022
FL Games: A Federated Learning Framework for Distribution Shifts Sharut Gupta Kartik Ahuja Mohammad Havaei N. Chatterjee Yoshua Bengio OOD FedML 29 18 0 31 Oct 2022
GradSkip: Communication-Accelerated Local Gradient Methods with Better Computational Complexity A. Maranjyan M. Safaryan Peter Richtárik 34 13 0 28 Oct 2022
Hierarchical Federated Learning with Momentum Acceleration in Multi-Tier Networks Zhengjie Yang Sen Fu Wei Bao Dong Yuan Albert Y. Zomaya FedML 44 5 0 26 Oct 2022
FeDXL: Provable Federated Learning for Deep X-Risk Optimization Zhishuai Guo R. L. Jin Jiebo Luo Tianbao Yang FedML 47 8 0 26 Oct 2022
On the Stability Analysis of Open Federated Learning Systems Youbang Sun H. Fernando Tianyi Chen Shahin Shahrampour FedML 29 1 0 25 Sep 2022
Tensor Decomposition based Personalized Federated Learning Qing Wang Jing Jin Xiaofeng Liu Huixuan Zong Yunfeng Shao Yinchuan Li FedML 27 3 0 27 Aug 2022
Adam Can Converge Without Any Modification On Update Rules Yushun Zhang Congliang Chen Naichen Shi Ruoyu Sun Zhimin Luo 18 63 0 20 Aug 2022
NET-FLEET: Achieving Linear Convergence Speedup for Fully Decentralized Federated Learning with Heterogeneous Data Xin Zhang Minghong Fang Zhuqing Liu Haibo Yang Jia-Wei Liu Zhengyuan Zhu FedML 25 14 0 17 Aug 2022
Partial Connection Based on Channel Attention for Differentiable Neural Architecture Search Yu Xue Jiafeng Qin 27 55 0 01 Aug 2022
On the Convergence of Momentum-Based Algorithms for Federated Bilevel Optimization Problems Hongchang Gao FedML 28 1 0 28 Apr 2022
Federated Progressive Sparsification (Purge, Merge, Tune)+ Dimitris Stripelis Umang Gupta Greg Ver Steeg J. Ambite FedML 23 9 0 26 Apr 2022
Closing the Generalization Gap of Cross-silo Federated Medical Image Segmentation An Xu Wenqi Li Pengfei Guo Dong Yang H. Roth Ali Hatamizadeh Can Zhao Daguang Xu Heng-Chiao Huang Ziyue Xu FedML 36 51 0 18 Mar 2022
Federated Minimax Optimization: Improved Convergence Analyses and Algorithms Pranay Sharma Rohan Panda Gauri Joshi P. Varshney FedML 21 47 0 09 Mar 2022
Personalized Federated Learning with Exact Stochastic Gradient Descent Sotirios Nikoloutsopoulos I. Koutsopoulos Michalis K. Titsias FedML 31 9 0 20 Feb 2022
Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam Yucheng Lu Conglong Li Minjia Zhang Christopher De Sa Yuxiong He OffRL AI4CE 24 20 0 12 Feb 2022
TOFU: Towards Obfuscated Federated Updates by Encoding Weight Updates into Gradients from Proxy Data Isha Garg M. Nagaraj Kaushik Roy FedML 31 1 0 21 Jan 2022
Communication-Efficient TeraByte-Scale Model Training Framework for Online Advertising Weijie Zhao Xuewu Jiao Mingqing Hu Xiaoyun Li Xinming Zhang Ping Li 3DV 37 8 0 05 Jan 2022
Collaborative Learning over Wireless Networks: An Introductory Overview Emre Ozfatura Deniz Gunduz H. Vincent Poor 27 11 0 07 Dec 2021
A Novel Convergence Analysis for Algorithms of the Adam Family Zhishuai Guo Yi Tian Xu W. Yin R. L. Jin Tianbao Yang 39 47 0 07 Dec 2021
BlueFog: Make Decentralized Algorithms Practical for Optimization and Deep Learning Bicheng Ying Kun Yuan Hanbin Hu Yiming Chen W. Yin FedML 39 27 0 08 Nov 2021
What Do We Mean by Generalization in Federated Learning? Honglin Yuan Warren Morningstar Lin Ning K. Singhal OOD FedML 41 71 0 27 Oct 2021
Exponential Graph is Provably Efficient for Decentralized Deep Training Bicheng Ying Kun Yuan Yiming Chen Hanbin Hu Pan Pan W. Yin FedML 39 83 0 26 Oct 2021
Accelerate Distributed Stochastic Descent for Nonconvex Optimization with Momentum Guojing Cong Tianyi Liu 16 0 0 01 Oct 2021
Order Optimal Bounds for One-Shot Federated Learning over non-Convex Loss Functions Arsalan Sharifnassab Saber Salehkaleybar S. J. Golestani FedML 6 0 0 19 Aug 2021
Decentralized Federated Learning: Balancing Communication and Computing Costs Wei Liu Li Chen Wenyi Zhang FedML 19 106 0 26 Jul 2021