On the convergence properties of a $K$ -step averaging stochastic gradient descent algorithm for nonconvex optimization

3 August 2017

Papers citing "On the convergence properties of a $K$-step averaging stochastic gradient descent algorithm for nonconvex optimization"

50 / 141 papers shown

Title
From Deterioration to Acceleration: A Calibration Approach to Rehabilitating Step Asynchronism in Federated Optimization Feijie Wu Song Guo Yining Qi Zhihao Qu Haobo Zhang Jiewei Zhang Ziming Liu 25 11 0 17 Dec 2021
Efficient Device Scheduling with Multi-Job Federated Learning Chen Zhou Ji Liu Juncheng Jia Jingbo Zhou Yang Zhou H. Dai Dejing Dou FedML 12 40 0 11 Dec 2021
Collaborative Learning over Wireless Networks: An Introductory Overview Emre Ozfatura Deniz Gunduz H. Vincent Poor 27 11 0 07 Dec 2021
Loss Landscape Dependent Self-Adjusting Learning Rates in Decentralized Stochastic Gradient Descent Wei Zhang Mingrui Liu Yu Feng Xiaodong Cui Brian Kingsbury Yuhai Tu 19 3 0 02 Dec 2021
Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspective Margalit Glasgow Honglin Yuan Tengyu Ma FedML 27 43 0 05 Nov 2021
Large-Scale Deep Learning Optimizations: A Comprehensive Survey Xiaoxin He Fuzhao Xue Xiaozhe Ren Yang You 27 14 0 01 Nov 2021
What Do We Mean by Generalization in Federated Learning? Honglin Yuan Warren Morningstar Lin Ning K. Singhal OOD FedML 41 71 0 27 Oct 2021
Asynchronous Decentralized Distributed Training of Acoustic Models Xiaodong Cui Wei Zhang Abdullah Kayi Mingrui Liu Ulrich Finkler Brian Kingsbury G. Saon David S. Kung 24 3 0 21 Oct 2021
Variance Reduction based Experience Replay for Policy Optimization Hua Zheng Wei Xie M. Feng OffRL 33 2 0 17 Oct 2021
Trade-offs of Local SGD at Scale: An Empirical Study Jose Javier Gonzalez Ortiz Jonathan Frankle Michael G. Rabbat Ari S. Morcos Nicolas Ballas FedML 37 19 0 15 Oct 2021
Adaptive Elastic Training for Sparse Deep Learning on Heterogeneous Multi-GPU Servers Yujing Ma Florin Rusu Kesheng Wu A. Sim 38 3 0 13 Oct 2021
A Stochastic Newton Algorithm for Distributed Convex Optimization Brian Bullins Kumar Kshitij Patel Ohad Shamir Nathan Srebro Blake E. Woodworth 28 15 0 07 Oct 2021
Accelerate Distributed Stochastic Descent for Nonconvex Optimization with Momentum Guojing Cong Tianyi Liu 16 0 0 01 Oct 2021
Toward Communication Efficient Adaptive Gradient Method Xiangyi Chen Xiaoyun Li P. Li FedML 34 41 0 10 Sep 2021
Order Optimal Bounds for One-Shot Federated Learning over non-Convex Loss Functions Arsalan Sharifnassab Saber Salehkaleybar S. J. Golestani FedML 6 0 0 19 Aug 2021
On Bridging Generic and Personalized Federated Learning for Image Classification Hong-You Chen Wei-Lun Chao FedML 22 21 0 02 Jul 2021
Personalized Federated Learning with Gaussian Processes Idan Achituve Aviv Shamsian Aviv Navon Gal Chechik Ethan Fetaya FedML 32 98 0 29 Jun 2021
CD-SGD: Distributed Stochastic Gradient Descent with Compression and Delay Compensation Enda Yu Dezun Dong Yemao Xu Shuo Ouyang Xiangke Liao 16 5 0 21 Jun 2021
STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learning Prashant Khanduri Pranay Sharma Haibo Yang Min-Fong Hong Jia Liu K. Rajawat P. Varshney FedML 21 63 0 19 Jun 2021
Towards Heterogeneous Clients with Elastic Federated Learning Zichen Ma Yu Lu Zihan Lu Wenye Li Jinfeng Yi Shuguang Cui FedML 19 3 0 17 Jun 2021
CFedAvg: Achieving Efficient Communication and Fast Convergence in Non-IID Federated Learning Haibo Yang Jia Liu Elizabeth S. Bentley FedML 21 17 0 14 Jun 2021
Communication-efficient SGD: From Local SGD to One-Shot Averaging Artin Spiridonoff Alexander Olshevsky I. Paschalidis FedML 29 20 0 09 Jun 2021
Fast Federated Learning by Balancing Communication Trade-Offs Milad Khademi Nori Sangseok Yun Il Kim FedML 20 53 0 23 May 2021
Federated Learning with Unreliable Clients: Performance Analysis and Mechanism Design Chuan Ma Jun Li Ming Ding Kang Wei Wen Chen H. Vincent Poor FedML 24 28 0 10 May 2021
The Gradient Convergence Bound of Federated Multi-Agent Reinforcement Learning with Efficient Communication Xing Xu Rongpeng Li Zhifeng Zhao Honggang Zhang 38 12 0 24 Mar 2021
Personalized Federated Learning using Hypernetworks Aviv Shamsian Aviv Navon Ethan Fetaya Gal Chechik FedML 41 324 0 08 Mar 2021
FedPower: Privacy-Preserving Distributed Eigenspace Estimation Xiaoxun Guo Xiang Li Xiangyu Chang Shusen Wang Zhihua Zhang FedML 16 3 0 01 Mar 2021
Wirelessly Powered Federated Edge Learning: Optimal Tradeoffs Between Convergence and Power Transfer Qunsong Zeng Yuqing Du Kaibin Huang 34 35 0 24 Feb 2021
Stragglers Are Not Disaster: A Hybrid Federated Learning Algorithm with Delayed Gradients Xingyu Li Zhe Qu Bo Tang Zhuo Lu FedML 73 31 0 12 Feb 2021
Federated Acoustic Modeling For Automatic Speech Recognition Xiaodong Cui Songtao Lu Brian Kingsbury 16 33 0 08 Feb 2021
A Bayesian Federated Learning Framework with Online Laplace Approximation Liang Liu Xi Jiang Feng Zheng Hong Chen Guo-Jun Qi Heng-Chiao Huang Ling Shao FedML 43 53 0 03 Feb 2021
Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learning Haibo Yang Minghong Fang Jia Liu FedML 17 249 0 27 Jan 2021
Federated Learning over Noisy Channels: Convergence Analysis and Design Examples Xizixiang Wei Cong Shen FedML 36 15 0 06 Jan 2021
Straggler-Resilient Federated Learning: Leveraging the Interplay Between Statistical Accuracy and System Heterogeneity Amirhossein Reisizadeh Isidoros Tziotis Hamed Hassani Aryan Mokhtari Ramtin Pedarsani FedML 172 99 0 28 Dec 2020
Cost-Effective Federated Learning Design Bing Luo Xiang Li Shiqiang Wang Jianwei Huang Leandros Tassiulas FedML 14 177 0 15 Dec 2020
Federated Learning under Importance Sampling Elsa Rizk Stefan Vlaski Ali H. Sayed FedML 16 52 0 14 Dec 2020
Federated Learning in Unreliable and Resource-Constrained Cellular Wireless Networks M. Salehi E. Hossain FedML 51 82 0 09 Dec 2020
Second-Order Guarantees in Federated Learning Stefan Vlaski Elsa Rizk Ali H. Sayed FedML 12 7 0 02 Dec 2020
Federated Composite Optimization Honglin Yuan Manzil Zaheer Sashank J. Reddi FedML 32 58 0 17 Nov 2020
Distributed Sparse SGD with Majority Voting Kerem Ozfatura Emre Ozfatura Deniz Gunduz FedML 40 4 0 12 Nov 2020
Towards a Scalable and Distributed Infrastructure for Deep Learning Applications Bita Hasheminezhad S. Shirzad Nanmiao Wu Patrick Diehl Hannes Schulz Hartmut Kaiser GNN AI4CE 27 4 0 06 Oct 2020
VirtualFlow: Decoupling Deep Learning Models from the Underlying Hardware Andrew Or Haoyu Zhang M. Freedman 12 9 0 20 Sep 2020
FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning Hong-You Chen Wei-Lun Chao FedML 14 255 0 04 Sep 2020
FedSKETCH: Communication-Efficient and Private Federated Learning via Sketching Farzin Haddadpour Belhal Karimi Ping Li Xiaoyun Li FedML 52 31 0 11 Aug 2020
Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization Jianyu Wang Qinghua Liu Hao Liang Gauri Joshi H. Vincent Poor MoMe FedML 16 1,297 0 15 Jul 2020
Adaptive Periodic Averaging: A Practical Approach to Reducing Communication in Distributed Learning Peng Jiang G. Agrawal 27 5 0 13 Jul 2020
Federated Learning with Compression: Unified Analysis and Sharp Guarantees Farzin Haddadpour Mohammad Mahdi Kamani Aryan Mokhtari M. Mahdavi FedML 33 271 0 02 Jul 2020
DEED: A General Quantization Scheme for Communication Efficiency in Bits Tian-Chun Ye Peijun Xiao Ruoyu Sun FedML MQ 28 2 0 19 Jun 2020
Federated Learning With Quantized Global Model Updates M. Amiri Deniz Gunduz Sanjeev R. Kulkarni H. Vincent Poor FedML 18 130 0 18 Jun 2020
Federated Accelerated Stochastic Gradient Descent Honglin Yuan Tengyu Ma FedML 17 171 0 16 Jun 2020