Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization

27 June 2015

Papers citing "Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization"

50 / 80 papers shown

Title
Cluster-Aware Multi-Round Update for Wireless Federated Learning in Heterogeneous Environments Pengcheng Sun Erwu Liu Wei Ni Kanglei Yu Rui-cang Wang Abbas Jamalipour FedML 26 0 0 06 May 2025
Pseudo-Asynchronous Local SGD: Robust and Efficient Data-Parallel Training Hiroki Naganuma Xinzhi Zhang Man-Chung Yue Ioannis Mitliagkas Philipp A. Witte Russell J. Hewett Yin Tat Lee 65 0 0 25 Apr 2025
Balancing Label Imbalance in Federated Environments Using Only Mixup and Artificially-Labeled Noise Kyle Rui Sang Tahseen Rabbani Furong Huang FedML 36 0 0 20 Sep 2024
Faster Stochastic Optimization with Arbitrary Delays via Asynchronous Mini-Batching Amit Attia Ofir Gaash Tomer Koren 40 0 0 14 Aug 2024
Ordered Momentum for Asynchronous SGD Chang-Wei Shi Yi-Rui Yang Wu-Jun Li ODL 59 0 0 27 Jul 2024
Asynchronous Federated Stochastic Optimization for Heterogeneous Objectives Under Arbitrary Delays Charikleia Iakovidou Kibaek Kim FedML 35 2 0 16 May 2024
Convergence Analysis of Decentralized ASGD Mauro Dalle Lucca Tosi Martin Theobald 29 2 0 07 Sep 2023
Revolutionizing Wireless Networks with Federated Learning: A Comprehensive Review Sajjad Emdadi Mahdimahalleh AI4CE 35 0 0 01 Aug 2023
Theoretically Principled Federated Learning for Balancing Privacy and Utility Xiaojin Zhang Wenjie Li Kai Chen Shutao Xia Qian Yang FedML 22 9 0 24 May 2023
Stability and Convergence of Distributed Stochastic Approximations with large Unbounded Stochastic Information Delays Adrian Redder Arunselvan Ramaswamy Holger Karl 20 1 0 11 May 2023
Performance and Energy Consumption of Parallel Machine Learning Algorithms Xidong Wu Preston Brazzle Stephen Cahoon 39 0 0 01 May 2023
Considerations on the Theory of Training Models with Differential Privacy Marten van Dijk Phuong Ha Nguyen FedML 8 2 0 08 Mar 2023
DoCoFL: Downlink Compression for Cross-Device Federated Learning Ron Dorfman S. Vargaftik Y. Ben-Itzhak Kfir Y. Levy FedML 29 18 0 01 Feb 2023
Analysis of Error Feedback in Federated Non-Convex Optimization with Biased Compression Xiaoyun Li Ping Li FedML 34 4 0 25 Nov 2022
STSyn: Speeding Up Local SGD with Straggler-Tolerant Synchronization Feng Zhu Jingjing Zhang Xin Wang 33 3 0 06 Oct 2022
Semi-Synchronous Personalized Federated Learning over Mobile Edge Networks Chaoqun You Daquan Feng Kun Guo Howard H. Yang Tony Q. S. Quek 38 12 0 27 Sep 2022
SYNTHESIS: A Semi-Asynchronous Path-Integrated Stochastic Gradient Method for Distributed Learning in Computing Clusters Zhuqing Liu Xin Zhang Jia-Wei Liu 32 1 0 17 Aug 2022
FederatedScope: A Flexible Federated Learning Platform for Heterogeneity Yuexiang Xie Zhen Wang Dawei Gao Daoyuan Chen Liuyi Yao Weirui Kuang Yaliang Li Bolin Ding Jingren Zhou FedML 21 88 0 11 Apr 2022
Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam Yucheng Lu Conglong Li Minjia Zhang Christopher De Sa Yuxiong He OffRL AI4CE 24 20 0 12 Feb 2022
Multiplayer Performative Prediction: Learning in Decision-Dependent Games Adhyyan Narang Evan Faulkner Dmitriy Drusvyatskiy Maryam Fazel Lillian J. Ratliff 18 42 0 10 Jan 2022
DSAG: A mixed synchronous-asynchronous iterative method for straggler-resilient learning A. Severinson E. Rosnes S. E. Rouayheb Alexandre Graell i Amat 16 2 0 27 Nov 2021
Accelerate Distributed Stochastic Descent for Nonconvex Optimization with Momentum Guojing Cong Tianyi Liu 16 0 0 01 Oct 2021
Accelerating Perturbed Stochastic Iterates in Asynchronous Lock-Free Optimization Kaiwen Zhou Anthony Man-Cho So James Cheng 19 1 0 30 Sep 2021
Mobility-Aware Cluster Federated Learning in Hierarchical Wireless Networks Chenyuan Feng H. Yang Deshun Hu Zhiwei Zhao Tony Q. S. Quek Geyong Min 36 74 0 20 Aug 2021
Order Optimal Bounds for One-Shot Federated Learning over non-Convex Loss Functions Arsalan Sharifnassab Saber Salehkaleybar S. J. Golestani FedML 6 0 0 19 Aug 2021
Decentralized Federated Learning: Balancing Communication and Computing Costs Wei Liu Li Chen Wenyi Zhang FedML 19 106 0 26 Jul 2021
Improved Learning Rates for Stochastic Optimization: Two Theoretical Viewpoints Shaojie Li Yong Liu 23 13 0 19 Jul 2021
Federated Learning with Buffered Asynchronous Aggregation John Nguyen Kshitiz Malik Hongyuan Zhan Ashkan Yousefpour Michael G. Rabbat Mani Malek Dzmitry Huba FedML 33 288 0 11 Jun 2021
Fast Federated Learning in the Presence of Arbitrary Device Unavailability Xinran Gu Kaixuan Huang Jingzhao Zhang Longbo Huang FedML 29 95 0 08 Jun 2021
DataLens: Scalable Privacy Preserving Training via Gradient Compression and Aggregation Wei Ping Fan Wu Yunhui Long Luka Rimanic Ce Zhang Bo-wen Li FedML 45 63 0 20 Mar 2021
Distributed Deep Learning Using Volunteer Computing-Like Paradigm Medha Atre B. Jha Ashwini Rao 18 11 0 16 Mar 2021
Parareal Neural Networks Emulating a Parallel-in-time Algorithm Zhanyu Ma Jiyang Xie Jingyi Yu AI4CE 18 9 0 16 Mar 2021
EventGraD: Event-Triggered Communication in Parallel Machine Learning Soumyadip Ghosh B. Aquino V. Gupta FedML 21 8 0 12 Mar 2021
Consistent Lock-free Parallel Stochastic Gradient Descent for Fast and Stable Convergence Karl Bäckström Ivan Walulya Marina Papatriantafilou P. Tsigas 26 5 0 17 Feb 2021
Integrating Deep Learning in Domain Sciences at Exascale Rick Archibald E. Chow E. DÁzevedo Jack J. Dongarra M. Eisenbach ... Florent Lopez Daniel Nichols S. Tomov Kwai Wong Junqi Yin PINN 23 5 0 23 Nov 2020
Hogwild! over Distributed Local Data Sets with Linearly Increasing Mini-Batch Sizes Marten van Dijk Nhuong V. Nguyen Toan N. Nguyen Lam M. Nguyen Quoc Tran-Dinh Phuong Ha Nguyen FedML 39 10 0 27 Oct 2020
Async-RED: A Provably Convergent Asynchronous Block Parallel Stochastic Method using Deep Denoising Priors Yu Sun Jiaming Liu Yiran Sun B. Wohlberg Ulugbek S. Kamilov 37 15 0 03 Oct 2020
DBS: Dynamic Batch Size For Distributed Deep Neural Network Training Qing Ye Yuhao Zhou Mingjia Shi Yanan Sun Jiancheng Lv 19 11 0 23 Jul 2020
Asynchronous Federated Learning with Reduced Number of Rounds and with Differential Privacy from Less Aggregated Gaussian Noise Marten van Dijk Nhuong V. Nguyen Toan N. Nguyen Lam M. Nguyen Quoc Tran-Dinh Phuong Ha Nguyen FedML 13 28 0 17 Jul 2020
Efficient Learning of Generative Models via Finite-Difference Score Matching Tianyu Pang Kun Xu Chongxuan Li Yang Song Stefano Ermon Jun Zhu DiffM 31 53 0 07 Jul 2020
MixML: A Unified Analysis of Weakly Consistent Parallel Learning Yucheng Lu J. Nash Christopher De Sa FedML 32 12 0 14 May 2020
Pipelined Backpropagation at Scale: Training Large Models without Batches Atli Kosson Vitaliy Chiley Abhinav Venigalla Joel Hestness Urs Koster 35 33 0 25 Mar 2020
Faster On-Device Training Using New Federated Momentum Algorithm Zhouyuan Huo Qian Yang Bin Gu Heng-Chiao Huang FedML 22 47 0 06 Feb 2020
Intermittent Pulling with Local Compensation for Communication-Efficient Federated Learning Yining Qi Zhihao Qu Song Guo Xin Gao Ruixuan Li Baoliu Ye FedML 18 8 0 22 Jan 2020
Asynchronous Federated Learning with Differential Privacy for Edge Intelligence Yanan Li Shusen Yang Xuebin Ren Cong Zhao FedML 19 33 0 17 Dec 2019
Distributed Inexact Successive Convex Approximation ADMM: Analysis-Part I Sandeep Kumar K. Rajawat Daniel P. Palomar 16 4 0 21 Jul 2019
Fully Decoupled Neural Network Learning Using Delayed Gradients Huiping Zhuang Yi Wang Qinglai Liu Shuai Zhang Zhiping Lin FedML 9 29 0 21 Jun 2019
MD-GAN: Multi-Discriminator Generative Adversarial Networks for Distributed Datasets Corentin Hardy Erwan Le Merrer B. Sericola GAN 27 181 0 09 Nov 2018
Toward Understanding the Impact of Staleness in Distributed Machine Learning Wei-Ming Dai Yi Zhou Nanqing Dong H. M. Zhang Eric P. Xing 19 79 0 08 Oct 2018
Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms Jianyu Wang Gauri Joshi 30 348 0 22 Aug 2018