Toward Understanding the Impact of Staleness in Distributed Machine Learning

8 October 2018

Papers citing "Toward Understanding the Impact of Staleness in Distributed Machine Learning"

36 / 36 papers shown

Title
ZenFlow: Enabling Stall-Free Offloading Training via Asynchronous Updates Tingfeng Lan Yusen Wu Bin Ma Zhaoyuan Su Rui Yang Tekin Bicer Dong Li Yue Cheng 2 0 0 18 May 2025
MAB-Based Channel Scheduling for Asynchronous Federated Learning in Non-Stationary Environments Zehan Li Yubo Yang Tao Yang X. Wu Ziyu Guo Bo Hu 64 0 0 03 Mar 2025
Membership Inference Attacks and Defenses in Federated Learning: A Survey Li Bai Haibo Hu Qingqing Ye Haoyang Li Leixia Wang Jianliang Xu FedML 82 13 0 09 Dec 2024
DPDR: Gradient Decomposition and Reconstruction for Differentially Private Deep Learning Yixuan Liu Li Xiong Yuhan Liu Yujie Gu Ruixuan Liu Hong Chen 40 1 0 04 Jun 2024
MSPipe: Efficient Temporal GNN Training via Staleness-Aware Pipeline Guangming Sheng Junwei Su Chao Huang Chuan Wu 33 5 0 23 Feb 2024
Client Orchestration and Cost-Efficient Joint Optimization for NOMA-Enabled Hierarchical Federated Learning Bibo Wu Fang Fang Xianbin Wang Donghong Cai Shu Fu Zhiguo Ding 34 0 0 03 Nov 2023
Arena: A Learning-based Synchronization Scheme for Hierarchical Federated Learning--Technical Report Tianyu Qi Yufeng Zhan Peng Li Jingcai Guo Yuanqing Xia FedML 27 12 0 20 Aug 2023
Defending Against Poisoning Attacks in Federated Learning with Blockchain Nanqing Dong Zhipeng Wang Jiahao Sun Michael C. Kampffmeyer William Knottenbelt Eric P. Xing OOD AAML 33 18 0 02 Jul 2023
Straggler-Resilient Decentralized Learning via Adaptive Asynchronous Updates Guojun Xiong Gang Yan Shiqiang Wang Jian Li 18 3 0 11 Jun 2023
Joint Age-based Client Selection and Resource Allocation for Communication-Efficient Federated Learning over NOMA Networks Bibo Wu Fang Fang Xianbin Wang 41 19 0 18 Apr 2023
TimelyFL: Heterogeneity-aware Asynchronous Federated Learning with Adaptive Partial Training Tuo Zhang Lei Gao Sunwoo Lee Mi Zhang Salman Avestimehr FedML 34 28 0 14 Apr 2023
Convergence Acceleration in Wireless Federated Learning: A Stackelberg Game Approach Kaidi Wang Y. Ma Mahdi Boloursaz Mashhadi C. Foh Rahim Tafazolli Z. Ding FedML 52 11 0 14 Sep 2022
Layer-Wise Partitioning and Merging for Efficient and Scalable Deep Learning S. Akintoye Liangxiu Han H. Lloyd Xin Zhang Darren Dancey Haoming Chen Daoqiang Zhang FedML 34 5 0 22 Jul 2022
FedSS: Federated Learning with Smart Selection of clients Ammar Tahir Yongzhou Chen Prashanti Nilayam FedML 13 4 0 10 Jul 2022
AFAFed -- Protocol analysis E. Baccarelli M. Scarpiniti Alireza Momenzadeh S. S. Ahrabi FedML 4 0 0 29 Jun 2022
Accelerating Asynchronous Federated Learning Convergence via Opportunistic Mobile Relaying Jieming Bian Jie Xu FedML 23 6 0 09 Jun 2022
Nebula-I: A General Framework for Collaboratively Training Deep Learning Models on Low-Bandwidth Cloud Clusters Yang Xiang Zhihua Wu Weibao Gong Siyu Ding Xianjie Mo ... Yue Yu Ge Li Yu Sun Yanjun Ma Dianhai Yu 24 4 0 19 May 2022
Energy Minimization for Federated Asynchronous Learning on Battery-Powered Mobile Devices via Application Co-running Cong Wang Bin Hu Hongyi Wu 23 5 0 29 Apr 2022
Towards Efficient and Stable K-Asynchronous Federated Learning with Unbounded Stale Gradients on Non-IID Data Zihao Zhou Yanan Li Xuebin Ren Shusen Yang 25 29 0 02 Mar 2022
Federated Stochastic Gradient Descent Begets Self-Induced Momentum Howard H. Yang Zuozhu Liu Yaru Fu Tony Q.S. Quek H. Vincent Poor FedML 8 5 0 17 Feb 2022
Wireless-Enabled Asynchronous Federated Fourier Neural Network for Turbulence Prediction in Urban Air Mobility (UAM) Tengchan Zeng Omid Semiari Walid Saad M. Bennis 34 3 0 26 Dec 2021
Scheduling Optimization Techniques for Neural Network Training Hyungjun Oh Junyeol Lee HyeongJu Kim Jiwon Seo 21 0 0 03 Oct 2021
AdaptCL: Efficient Collaborative Learning with Dynamic and Adaptive Pruning Guangmeng Zhou Ke Xu Qi Li Yang Liu Yi Zhao 21 8 0 27 Jun 2021
SpreadGNN: Serverless Multi-task Federated Learning for Graph Neural Networks Chaoyang He Emir Ceyani Keshav Balasubramanian M. Annavaram Salman Avestimehr FedML 25 50 0 04 Jun 2021
Straggler-Resilient Distributed Machine Learning with Dynamic Backup Workers Guojun Xiong Gang Yan Rahul Singh Jian Li 28 12 0 11 Feb 2021
Semi-Synchronous Federated Learning for Energy-Efficient Training and Accelerated Convergence in Cross-Silo Settings Dimitris Stripelis J. Ambite FedML 13 38 0 04 Feb 2021
Crossover-SGD: A gossip-based communication in distributed deep learning for alleviating large mini-batch problem and enhancing scalability Sangho Yeo Minho Bae Minjoong Jeong Oh-Kyoung Kwon Sangyoon Oh 11 3 0 30 Dec 2020
Distributed Machine Learning for Wireless Communication Networks: Techniques, Architectures, and Applications Shuyan Hu Xiaojing Chen Wei Ni Ekram Hossain Xin Wang AI4CE 47 111 0 02 Dec 2020
Accelerating Federated Learning in Heterogeneous Data and Computational Environments Dimitris Stripelis J. Ambite FedML 12 11 0 25 Aug 2020
DS-Sync: Addressing Network Bottlenecks with Divide-and-Shuffle Synchronization for Distributed DNN Training Weiyan Wang Cengguang Zhang Liu Yang Kai Chen Kun Tan 29 12 0 07 Jul 2020
Dynamic backup workers for parallel machine learning Chuan Xu Giovanni Neglia Nicola Sebastianelli 15 11 0 30 Apr 2020
Elastic Consistency: A General Consistency Model for Distributed Stochastic Gradient Descent Giorgi Nadiradze Ilia Markov Bapi Chatterjee Vyacheslav Kungurtsev Dan Alistarh FedML 22 14 0 16 Jan 2020
Sparsification as a Remedy for Staleness in Distributed Asynchronous SGD Rosa Candela Giulio Franzese Maurizio Filippone Pietro Michiardi 18 1 0 21 Oct 2019
At Stability's Edge: How to Adjust Hyperparameters to Preserve Minima Selection in Asynchronous Training of Neural Networks? Niv Giladi Mor Shpigel Nacson Elad Hoffer Daniel Soudry 20 22 0 26 Sep 2019
Taming Momentum in a Distributed Asynchronous Environment Ido Hakimi Saar Barkai Moshe Gabel Assaf Schuster 11 23 0 26 Jul 2019
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 308 2,890 0 15 Sep 2016