Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learning

17 July 2018

Papers citing "Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learning"

50 / 106 papers shown

Title
Communication-Efficient Federated Fine-Tuning of Language Models via Dynamic Update Schedules Michail Theologitis V. Samoladas Antonios Deligiannakis 34 0 0 07 May 2025
Cluster-Aware Multi-Round Update for Wireless Federated Learning in Heterogeneous Environments Pengcheng Sun Erwu Liu Wei Ni Kanglei Yu Rui-cang Wang Abbas Jamalipour FedML 26 0 0 06 May 2025
Biased Federated Learning under Wireless Heterogeneity Muhammad Faraz Ul Abrar Nicolò Michelusi FedML 46 0 0 08 Mar 2025
Communication-Efficient Device Scheduling for Federated Learning Using Lyapunov Optimization Jake B. Perazzone Shiqiang Wang Mingyue Ji Kevin S. Chan FedML 75 0 0 01 Mar 2025
Revisiting LocalSGD and SCAFFOLD: Improved Rates and Missing Analysis Ruichen Luo Sebastian U Stich Samuel Horváth Martin Takáč 40 0 0 08 Jan 2025
EDiT: A Local-SGD-Based Efficient Distributed Training Method for Large Language Models Jialiang Cheng Ning Gao Yun Yue Zhiling Ye Jiadi Jiang Jian Sha OffRL 77 0 0 10 Dec 2024
Aiding Global Convergence in Federated Learning via Local Perturbation and Mutual Similarity Information Emanuel Buttaci Giuseppe Carlo Calafiore FedML 27 0 0 07 Oct 2024
No Need to Talk: Asynchronous Mixture of Language Models Anastasiia Filippova Angelos Katharopoulos David Grangier Ronan Collobert MoE 41 0 0 04 Oct 2024
Selective Aggregation for Low-Rank Adaptation in Federated Learning Pengxin Guo Shuang Zeng Y. Wang Huijie Fan Feifei Wang Liangqiong Qu FedML 44 10 0 02 Oct 2024
A New Theoretical Perspective on Data Heterogeneity in Federated Optimization Jiayi Wang Shiqiang Wang Rong-Rong Chen Mingyue Ji FedML 30 1 0 22 Jul 2024
PAFedFV: Personalized and Asynchronous Federated Learning for Finger Vein Recognition Hengyu Mu Jian Guo Chong Han Lijuan Sun FedML 26 5 0 20 Apr 2024
FedDRO: Federated Compositional Optimization for Distributionally Robust Learning Prashant Khanduri Chengyin Li Rafi Ibn Sultan Yao Qiang Joerg Kliewer Dongxiao Zhu 39 0 0 21 Nov 2023
ALI-DPFL: Differentially Private Federated Learning with Adaptive Local Iterations Xinpeng Ling Jie Fu Kuncan Wang Haitao Liu Zhili Chen FedML 39 2 0 21 Aug 2023
Revolutionizing Wireless Networks with Federated Learning: A Comprehensive Review Sajjad Emdadi Mahdimahalleh AI4CE 35 0 0 01 Aug 2023
Improved Convergence Analysis and SNR Control Strategies for Federated Learning in the Presence of Noise Antesh Upadhyay Abolfazl Hashemi 42 9 0 14 Jul 2023
A Survey From Distributed Machine Learning to Distributed Deep Learning Mohammad Dehghani Zahra Yazdanparast 23 0 0 11 Jul 2023
Benchmarking Algorithms for Federated Domain Generalization Ruqi Bai S. Bagchi David I. Inouye FedML 40 12 0 11 Jul 2023
Federated Multi-Sequence Stochastic Approximation with Local Hypergradient Estimation Davoud Ataee Tarzanagh Mingchen Li Pranay Sharma Samet Oymak 26 0 0 02 Jun 2023
Local SGD Accelerates Convergence by Exploiting Second Order Information of the Loss Function Linxuan Pan Shenghui Song FedML 25 2 0 24 May 2023
On the Optimal Batch Size for Byzantine-Robust Distributed Learning Yi-Rui Yang Chang-Wei Shi Wu-Jun Li FedML AAML 21 0 0 23 May 2023
On the Local Cache Update Rules in Streaming Federated Learning Heqiang Wang Jieming Bian Jie Xu 24 4 0 28 Mar 2023
Cloudless-Training: A Framework to Improve Efficiency of Geo-Distributed ML Training W. Tan Xiao Shi Cunchi Lv Xiaofang Zhao FedML 25 1 0 09 Mar 2023
FLINT: A Platform for Federated Learning Integration Ewen N. Wang Ajaykumar Kannan Yuefeng Liang Boyi Chen Mosharaf Chowdhury 40 24 0 24 Feb 2023
A Survey of Trustworthy Federated Learning with Perspectives on Security, Robustness, and Privacy Yifei Zhang Dun Zeng Jinglong Luo Zenglin Xu Irwin King FedML 84 47 0 21 Feb 2023
Delay Sensitive Hierarchical Federated Learning with Stochastic Local Updates Abdulmoneam Ali A. Arafa FedML 34 4 0 09 Feb 2023
Differentially Private Federated Clustering over Non-IID Data Yiwei Li Shuai Wang Chong-Yung Chi Tony Q. S. Quek FedML 33 12 0 03 Jan 2023
Graph Federated Learning with Hidden Representation Sharing Shuang Wu Mingxuan Zhang Yuantong Li Carl Yang Pan Li FedML 24 1 0 23 Dec 2022
Federated Learning with Flexible Control Shiqiang Wang Jake B. Perazzone Mingyue Ji Kevin S. Chan FedML 28 17 0 16 Dec 2022
Communication-Efficient Local SGD with Age-Based Worker Selection Feng Zhu Jingjing Zhang Xin Wang 32 1 0 31 Oct 2022
FeDXL: Provable Federated Learning for Deep X-Risk Optimization Zhishuai Guo R. L. Jin Jiebo Luo Tianbao Yang FedML 47 8 0 26 Oct 2022
STSyn: Speeding Up Local SGD with Straggler-Tolerant Synchronization Feng Zhu Jingjing Zhang Xin Wang 33 3 0 06 Oct 2022
PersA-FL: Personalized Asynchronous Federated Learning Taha Toghani Soomin Lee César A. Uribe FedML 34 6 0 03 Oct 2022
Unbounded Gradients in Federated Learning with Buffered Asynchronous Aggregation Taha Toghani César A. Uribe FedML 35 14 0 03 Oct 2022
NET-FLEET: Achieving Linear Convergence Speedup for Fully Decentralized Federated Learning with Heterogeneous Data Xin Zhang Minghong Fang Zhuqing Liu Haibo Yang Jia-Wei Liu Zhengyuan Zhu FedML 22 14 0 17 Aug 2022
On the Convergence of Multi-Server Federated Learning with Overlapping Area Zhe Qu Xingyu Li Jie Xu Bo Tang Zhuo Lu Yao-Hong Liu FedML 50 14 0 16 Aug 2022
A Fast Blockchain-based Federated Learning Framework with Compressed Communications Laizhong Cui Xiaoxin Su Yipeng Zhou FedML 19 23 0 12 Aug 2022
Hypernetwork-based Personalized Federated Learning for Multi-Institutional CT Imaging Ziyuan Yang Wenjun Xia Zexin Lu Yingyu Chen Xiaoxia Li Yi Zhang FedML OOD 16 28 0 08 Jun 2022
Edge Learning for B5G Networks with Distributed Signal Processing: Semantic Communication, Edge Computing, and Wireless Sensing Wei Xu Zhaohui Yang Derrick Wing Kwan Ng Marco Levorato Yonina C. Eldar Mérouane Debbah 28 398 0 01 Jun 2022
Communication-Efficient Distributionally Robust Decentralized Learning Matteo Zecchin Marios Kountouris David Gesbert 22 9 0 31 May 2022
DELTA: Diverse Client Sampling for Fasting Federated Learning Lung-Chuang Wang Yongxin Guo Tao R. Lin Xiaoying Tang FedML 23 22 0 27 May 2022
FedAvg with Fine Tuning: Local Updates Lead to Representation Learning Liam Collins Hamed Hassani Aryan Mokhtari Sanjay Shakkottai FedML 32 75 0 27 May 2022
Over-the-Air Federated Learning with Joint Adaptive Computation and Power Control Haibo Yang Pei-Yuan Qiu Jia Liu Aylin Yener 21 18 0 12 May 2022
FedNest: Federated Bilevel, Minimax, and Compositional Optimization Davoud Ataee Tarzanagh Mingchen Li Christos Thrampoulidis Samet Oymak FedML 40 73 0 04 May 2022
Local Stochastic Bilevel Optimization with Momentum-Based Variance Reduction Junyi Li Feihu Huang Heng-Chiao Huang FedML 19 27 0 03 May 2022
FedGiA: An Efficient Hybrid Algorithm for Federated Learning Shenglong Zhou Geoffrey Ye Li FedML 31 16 0 03 May 2022
On the Convergence of Momentum-Based Algorithms for Federated Bilevel Optimization Problems Hongchang Gao FedML 28 1 0 28 Apr 2022
FedCau: A Proactive Stop Policy for Communication and Computation Efficient Federated Learning Afsaneh Mahmoudi H. S. Ghadikolaei José Hélio da Cruz Júnior Carlo Fischione 30 9 0 16 Apr 2022
FedVQCS: Federated Learning via Vector Quantized Compressed Sensing Yong-Nam Oh Yo-Seb Jeon Mingzhe Chen Walid Saad FedML 27 10 0 16 Apr 2022
FedCos: A Scene-adaptive Federated Optimization Enhancement for Performance Improvement Hao Zhang Tingting Wu Siyao Cheng Jie Liu FedML 35 11 0 07 Apr 2022
SHED: A Newton-type algorithm for federated learning based on incremental Hessian eigenvector sharing Nicolò Dal Fabbro S. Dey M. Rossi Luca Schenato FedML 29 14 0 11 Feb 2022