Asynchronous Stochastic Gradient Descent with Delay Compensation

27 September 2016

Papers citing "Asynchronous Stochastic Gradient Descent with Delay Compensation"

41 / 41 papers shown

Title
Cluster-Aware Multi-Round Update for Wireless Federated Learning in Heterogeneous Environments Pengcheng Sun Erwu Liu Wei Ni Kanglei Yu Rui-cang Wang Abbas Jamalipour FedML 26 0 0 06 May 2025
Nesterov Method for Asynchronous Pipeline Parallel Optimization Thalaiyasingam Ajanthan Sameera Ramasinghe Yan Zuo Gil Avraham Alexander Long 26 0 0 02 May 2025
Pseudo-Asynchronous Local SGD: Robust and Efficient Data-Parallel Training Hiroki Naganuma Xinzhi Zhang Man-Chung Yue Ioannis Mitliagkas Philipp A. Witte Russell J. Hewett Yin Tat Lee 65 0 0 25 Apr 2025
Orthogonal Calibration for Asynchronous Federated Learning Jiayun Zhang Shuheng Li Haiyu Huang Xiaofan Yu Rajesh K. Gupta Jingbo Shang FedML 65 0 0 21 Feb 2025
Asynchronous Stochastic Gradient Descent with Decoupled Backpropagation and Layer-Wise Updates Cabrel Teguemne Fokam Khaleelulla Khan Nazeer Lukas König David Kappel Anand Subramoney 35 0 0 08 Oct 2024
Ordered Momentum for Asynchronous SGD Chang-Wei Shi Yi-Rui Yang Wu-Jun Li ODL 62 0 0 27 Jul 2024
Distributed Stochastic Gradient Descent with Staleness: A Stochastic Delay Differential Equation Based Framework Siyuan Yu Wei Chen H. V. Poor 32 0 0 17 Jun 2024
Revolutionizing Wireless Networks with Federated Learning: A Comprehensive Review Sajjad Emdadi Mahdimahalleh AI4CE 38 0 0 01 Aug 2023
Distributed Optimization Methods for Multi-Robot Systems: Part II -- A Survey O. Shorinwa Trevor Halsted Javier Yu Mac Schwager 24 19 0 26 Jan 2023
STSyn: Speeding Up Local SGD with Straggler-Tolerant Synchronization Feng Zhu Jingjing Zhang Xin Wang 33 3 0 06 Oct 2022
Efficient Adaptive Federated Optimization of Federated Learning for IoT Zunming Chen Hongyan Cui Ensen Wu Yu Xi 27 0 0 23 Jun 2022
FedGiA: An Efficient Hybrid Algorithm for Federated Learning Shenglong Zhou Geoffrey Ye Li FedML 31 16 0 03 May 2022
A review of Federated Learning in Intrusion Detection Systems for IoT Aitor Belenguer J. Navaridas J. A. Pascual 28 15 0 26 Apr 2022
Wireless-Enabled Asynchronous Federated Fourier Neural Network for Turbulence Prediction in Urban Air Mobility (UAM) Tengchan Zeng Omid Semiari Walid Saad M. Bennis 34 3 0 26 Dec 2021
Collaborative Learning over Wireless Networks: An Introductory Overview Emre Ozfatura Deniz Gunduz H. Vincent Poor 30 11 0 07 Dec 2021
BAGUA: Scaling up Distributed Learning with System Relaxations Shaoduo Gan Xiangru Lian Rui Wang Jianbin Chang Chengjun Liu ... Jiawei Jiang Binhang Yuan Sen Yang Ji Liu Ce Zhang 25 30 0 03 Jul 2021
Federated Learning with Buffered Asynchronous Aggregation John Nguyen Kshitiz Malik Hongyuan Zhan Ashkan Yousefpour Michael G. Rabbat Mani Malek Dzmitry Huba FedML 33 288 0 11 Jun 2021
Towards Demystifying Serverless Machine Learning Training Jiawei Jiang Shaoduo Gan Yue Liu Fanlin Wang Gustavo Alonso Ana Klimovic Ankit Singla Wentao Wu Ce Zhang 19 121 0 17 May 2021
AutoSpace: Neural Architecture Search with Less Human Interference Daquan Zhou Xiaojie Jin Xiaochen Lian Linjie Yang Yujing Xue Qibin Hou Jiashi Feng 26 10 0 22 Mar 2021
Distributed Deep Learning Using Volunteer Computing-Like Paradigm Medha Atre B. Jha Ashwini Rao 18 11 0 16 Mar 2021
Adaptive Transmission Scheduling in Wireless Networks for Asynchronous Federated Learning Hyun-Suk Lee Jang-Won Lee 81 53 0 02 Mar 2021
1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed Hanlin Tang Shaoduo Gan A. A. Awan Samyam Rajbhandari Conglong Li Xiangru Lian Ji Liu Ce Zhang Yuxiong He AI4CE 45 84 0 04 Feb 2021
FedAR: Activity and Resource-Aware Federated Learning Model for Distributed Mobile Robots Ahmed Imteaj M. Amini 83 51 0 11 Jan 2021
Hogwild! over Distributed Local Data Sets with Linearly Increasing Mini-Batch Sizes Marten van Dijk Nhuong V. Nguyen Toan N. Nguyen Lam M. Nguyen Quoc Tran-Dinh Phuong Ha Nguyen FedML 42 10 0 27 Oct 2020
A Survey on Large-scale Machine Learning Meng Wang Weijie Fu Xiangnan He Shijie Hao Xindong Wu 19 109 0 10 Aug 2020
Accelerating Federated Learning over Reliability-Agnostic Clients in Mobile Edge Computing Systems Wentai Wu Ligang He Weiwei Lin Rui Mao 25 78 0 28 Jul 2020
DBS: Dynamic Batch Size For Distributed Deep Neural Network Training Qing Ye Yuhao Zhou Mingjia Shi Yanan Sun Jiancheng Lv 22 11 0 23 Jul 2020
Asynchronous Federated Learning with Reduced Number of Rounds and with Differential Privacy from Less Aggregated Gaussian Noise Marten van Dijk Nhuong V. Nguyen Toan N. Nguyen Lam M. Nguyen Quoc Tran-Dinh Phuong Ha Nguyen FedML 16 28 0 17 Jul 2020
Buffered Asynchronous SGD for Byzantine Learning Yi-Rui Yang Wu-Jun Li FedML 29 5 0 02 Mar 2020
Taming Momentum in a Distributed Asynchronous Environment Ido Hakimi Saar Barkai Moshe Gabel Assaf Schuster 13 23 0 26 Jul 2019
Fully Decoupled Neural Network Learning Using Delayed Gradients Huiping Zhuang Yi Wang Qinglai Liu Shuai Zhang Zhiping Lin FedML 14 29 0 21 Jun 2019
Dynamic Mini-batch SGD for Elastic Distributed Training: Learning in the Limbo of Resources Yanghua Peng Hang Zhang Yifei Ma Tong He Zhi-Li Zhang Sheng Zha Mu Li 25 23 0 26 Apr 2019
Asynchronous Federated Optimization Cong Xie Oluwasanmi Koyejo Indranil Gupta FedML 15 561 0 10 Mar 2019
Gradient Scheduling with Global Momentum for Non-IID Data Distributed Asynchronous Training Chengjie Li Ruixuan Li Yining Qi Yuhua Li Pan Zhou Song Guo Keqin Li 27 15 0 21 Feb 2019
A Hitchhiker's Guide On Distributed Training of Deep Neural Networks K. Chahal Manraj Singh Grover Kuntal Dey 3DH OOD 6 53 0 28 Oct 2018
Error Compensated Quantized SGD and its Applications to Large-scale Distributed Optimization Jiaxiang Wu Weidong Huang Junzhou Huang Tong Zhang 18 234 0 21 Jun 2018
Taming Convergence for Asynchronous Stochastic Gradient Descent with Unbounded Delay in Non-Convex Learning Xin Zhang Jia-Wei Liu Zhengyuan Zhu 16 17 0 24 May 2018
Stochastic Gradient Descent on Highly-Parallel Architectures Yujing Ma Florin Rusu Martin Torres 24 6 0 24 Feb 2018
Scale out for large minibatch SGD: Residual network training on ImageNet-1K with improved accuracy and reduced time to train V. Codreanu Damian Podareanu V. Saletore 39 55 0 12 Nov 2017
Convergence Analysis of Distributed Stochastic Gradient Descent with Shuffling Qi Meng Wei-neng Chen Yue Wang Zhi-Ming Ma Tie-Yan Liu FedML 18 101 0 29 Sep 2017
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 183 1,185 0 30 Nov 2014