Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learning

17 July 2018

Papers citing "Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learning"

50 / 106 papers shown

Title
Comparative assessment of federated and centralized machine learning Ibrahim Abdul Majeed Sagar Kaushik Aniruddha Bardhan Venkata Siva Kumar Tadi Hwang-Ki Min K. Kumaraguru Rajasekhara Reddy Duvvuru Muni FedML 25 6 0 03 Feb 2022
On the Convergence of Heterogeneous Federated Learning with Arbitrary Adaptive Online Model Pruning Hanhan Zhou Tian-Shing Lan Guru Venkataramani Wenbo Ding FedML 32 6 0 27 Jan 2022
Communication-Efficient Stochastic Zeroth-Order Optimization for Federated Learning Wenzhi Fang Ziyi Yu Yuning Jiang Yuanming Shi Colin N. Jones Yong Zhou FedML 78 57 0 24 Jan 2022
Optimal Rate Adaption in Federated Learning with Compressed Communications Laizhong Cui Xiaoxin Su Yipeng Zhou Jiangchuan Liu FedML 42 38 0 13 Dec 2021
On Convergence of Federated Averaging Langevin Dynamics Wei Deng Qian Zhang Yi Ma Zhao Song Guang Lin FedML 30 16 0 09 Dec 2021
Optimization-Based GenQSGD for Federated Edge Learning Yangchen Li Ying Cui Vincent K. N. Lau FedML 6 6 0 25 Oct 2021
TESSERACT: Gradient Flip Score to Secure Federated Learning Against Model Poisoning Attacks Atul Sharma Wei Chen Joshua C. Zhao Qiang Qiu Somali Chaterji S. Bagchi FedML AAML 54 5 0 19 Oct 2021
Trade-offs of Local SGD at Scale: An Empirical Study Jose Javier Gonzalez Ortiz Jonathan Frankle Michael G. Rabbat Ari S. Morcos Nicolas Ballas FedML 43 19 0 15 Oct 2021
Federated Learning from Small Datasets Michael Kamp Jonas Fischer Jilles Vreeken FedML 32 26 0 07 Oct 2021
Cross-Silo Federated Learning for Multi-Tier Networks with Vertical and Horizontal Data Partitioning Anirban Das Timothy Castiglia Shiqiang Wang S. Patterson FedML 13 19 0 19 Aug 2021
FedJAX: Federated learning simulation with JAX Jae Hun Ro A. Suresh Ke Wu FedML 33 48 0 04 Aug 2021
Decentralized Federated Learning: Balancing Communication and Computing Costs Wei Liu Li Chen Wenyi Zhang FedML 19 106 0 26 Jul 2021
A Field Guide to Federated Optimization Jianyu Wang Zachary B. Charles Zheng Xu Gauri Joshi H. B. McMahan ... Mi Zhang Tong Zhang Chunxiang Zheng Chen Zhu Wennan Zhu FedML 187 412 0 14 Jul 2021
BAGUA: Scaling up Distributed Learning with System Relaxations Shaoduo Gan Xiangru Lian Rui Wang Jianbin Chang Chengjun Liu ... Jiawei Jiang Binhang Yuan Sen Yang Ji Liu Ce Zhang 23 30 0 03 Jul 2021
Federated Noisy Client Learning Huazhu Fu Li Li Bo Han Chengzhong Xu Ling Shao FedML 25 26 0 24 Jun 2021
Federated Learning with Buffered Asynchronous Aggregation John Nguyen Kshitiz Malik Hongyuan Zhan Ashkan Yousefpour Michael G. Rabbat Mani Malek Dzmitry Huba FedML 33 288 0 11 Jun 2021
SpreadGNN: Serverless Multi-task Federated Learning for Graph Neural Networks Chaoyang He Emir Ceyani Keshav Balasubramanian M. Annavaram Salman Avestimehr FedML 25 50 0 04 Jun 2021
FL-NTK: A Neural Tangent Kernel-based Framework for Federated Learning Convergence Analysis Baihe Huang Xiaoxiao Li Zhao Song Xin Yang FedML 25 16 0 11 May 2021
Slashing Communication Traffic in Federated Learning by Transmitting Clustered Model Updates Laizhong Cui Xiaoxin Su Yipeng Zhou Yi Pan FedML 38 36 0 10 May 2021
Sample-based and Feature-based Federated Learning for Unconstrained and Constrained Nonconvex Optimization via Mini-batch SSCA Ying Cui Yangchen Li Chencheng Ye FedML 16 7 0 13 Apr 2021
Distributed Learning in Wireless Networks: Recent Progress and Future Challenges Mingzhe Chen Deniz Gündüz Kaibin Huang Walid Saad M. Bennis Aneta Vulgarakis Feljan H. Vincent Poor 38 401 0 05 Apr 2021
Wirelessly Powered Federated Edge Learning: Optimal Tradeoffs Between Convergence and Power Transfer Qunsong Zeng Yuqing Du Kaibin Huang 37 35 0 24 Feb 2021
Bandit-based Communication-Efficient Client Selection Strategies for Federated Learning Yae Jee Cho Samarth Gupta Gauri Joshi Osman Yağan FedML 13 67 0 14 Dec 2020
Federated Learning under Importance Sampling Elsa Rizk Stefan Vlaski Ali H. Sayed FedML 16 52 0 14 Dec 2020
Federated Composite Optimization Honglin Yuan Manzil Zaheer Sashank J. Reddi FedML 32 58 0 17 Nov 2020
Demystifying Why Local Aggregation Helps: Convergence Analysis of Hierarchical SGD Jiayi Wang Shiqiang Wang Rong-Rong Chen Mingyue Ji FedML 36 51 0 24 Oct 2020
Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithms Maruan Al-Shedivat Jennifer Gillenwater Eric P. Xing Afshin Rostamizadeh FedML 19 109 0 11 Oct 2020
Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies Yae Jee Cho Jianyu Wang Gauri Joshi FedML 44 400 0 03 Oct 2020
Federated Learning with Nesterov Accelerated Gradient Zhengjie Yang Wei Bao Dong Yuan Nguyen H. Tran Albert Y. Zomaya FedML 19 29 0 18 Sep 2020
Periodic Stochastic Gradient Descent with Momentum for Decentralized Training Hongchang Gao Heng-Chiao Huang 23 25 0 24 Aug 2020
Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications Jihong Park S. Samarakoon Anis Elgabli Joongheon Kim M. Bennis Seong-Lyun Kim Mérouane Debbah 34 161 0 06 Aug 2020
Stochastic Normalized Gradient Descent with Momentum for Large-Batch Training Shen-Yi Zhao Chang-Wei Shi Yin-Peng Xie Wu-Jun Li ODL 20 8 0 28 Jul 2020
Fast-Convergent Federated Learning Hung T. Nguyen Vikash Sehwag Seyyedali Hosseinalipour Christopher G. Brinton M. Chiang H. Vincent Poor FedML 26 192 0 26 Jul 2020
DBS: Dynamic Batch Size For Distributed Deep Neural Network Training Qing Ye Yuhao Zhou Mingjia Shi Yanan Sun Jiancheng Lv 22 11 0 23 Jul 2020
Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization Jianyu Wang Qinghua Liu Hao Liang Gauri Joshi H. Vincent Poor MoMe FedML 16 1,297 0 15 Jul 2020
Federated Learning with Compression: Unified Analysis and Sharp Guarantees Farzin Haddadpour Mohammad Mahdi Kamani Aryan Mokhtari M. Mahdavi FedML 36 271 0 02 Jul 2020
FedGAN: Federated Generative Adversarial Networks for Distributed Data M. Rasouli Tao Sun Ram Rajagopal FedML 30 143 0 12 Jun 2020
FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Data Xinwei Zhang Mingyi Hong S. Dhople W. Yin Yang Liu FedML 21 227 0 22 May 2020
Communication-Efficient Distributed Stochastic AUC Maximization with Deep Neural Networks Zhishuai Guo Mingrui Liu Zhuoning Yuan Li Shen Wei Liu Tianbao Yang 33 42 0 05 May 2020
A Unified Theory of Decentralized SGD with Changing Topology and Local Updates Anastasia Koloskova Nicolas Loizou Sadra Boreiri Martin Jaggi Sebastian U. Stich FedML 41 493 0 23 Mar 2020
Communication optimization strategies for distributed deep neural network training: A survey Shuo Ouyang Dezun Dong Yemao Xu Liquan Xiao 30 12 0 06 Mar 2020
Buffered Asynchronous SGD for Byzantine Learning Yi-Rui Yang Wu-Jun Li FedML 29 5 0 02 Mar 2020
Adaptive Federated Optimization Sashank J. Reddi Zachary B. Charles Manzil Zaheer Zachary Garrett Keith Rush Jakub Konecný Sanjiv Kumar H. B. McMahan FedML 20 1,391 0 29 Feb 2020
Dynamic Federated Learning Elsa Rizk Stefan Vlaski Ali H. Sayed FedML 22 25 0 20 Feb 2020
Is Local SGD Better than Minibatch SGD? Blake E. Woodworth Kumar Kshitij Patel Sebastian U. Stich Zhen Dai Brian Bullins H. B. McMahan Ohad Shamir Nathan Srebro FedML 34 253 0 18 Feb 2020
Distributed Non-Convex Optimization with Sublinear Speedup under Intermittent Client Availability Yikai Yan Chaoyue Niu Yucheng Ding Zhenzhe Zheng Fan Wu Guihai Chen Shaojie Tang Zhihua Wu FedML 49 37 0 18 Feb 2020
Stochastic Weight Averaging in Parallel: Large-Batch Training that Generalizes Well Vipul Gupta S. Serrano D. DeCoste MoMe 38 55 0 07 Jan 2020
Variance Reduced Local SGD with Lower Communication Complexity Xian-Feng Liang Shuheng Shen Jingchang Liu Zhen Pan Enhong Chen Yifei Cheng FedML 24 152 0 30 Dec 2019
Abnormal Client Behavior Detection in Federated Learning Suyi Li Yong Cheng Yang Liu Wei Wang Tianjian Chen AAML 8 134 0 22 Oct 2019
Communication-Efficient Local Decentralized SGD Methods Xiang Li Wenhao Yang Shusen Wang Zhihua Zhang 30 53 0 21 Oct 2019