Local SGD Converges Fast and Communicates Little

24 May 2018

Papers citing "Local SGD Converges Fast and Communicates Little"

50 / 629 papers shown

Title
FedAQ: Communication-Efficient Federated Edge Learning via Joint Uplink and Downlink Adaptive Quantization Linping Qu Shenghui Song Chi-Ying Tsui MQ FedML 18 4 0 26 Jun 2024
Scalable Dual Coordinate Descent for Kernel Methods Zishan Shao Aditya Devarakonda 16 1 0 26 Jun 2024
Communication-Efficient Adaptive Batch Size Strategies for Distributed Local Gradient Methods Tim Tsz-Kit Lau Weijian Li Chenwei Xu Han Liu Mladen Kolar 44 1 0 20 Jun 2024
ACCO: Accumulate While You Communicate for Communication-Overlapped Sharded LLM Training Adel Nabli Louis Fournier Pierre Erbacher Louis Serrano Eugene Belilovsky Edouard Oyallon FedML 46 1 0 03 Jun 2024
Local Methods with Adaptivity via Scaling Saveliy Chezhegov Sergey Skorik Nikolas Khachaturov Danil Shalagin A. Avetisyan Aleksandr Beznosikov Martin Takáč Yaroslav Kholodov Alexander Gasnikov 53 2 0 02 Jun 2024
Communication-Efficient Distributed Deep Learning via Federated Dynamic Averaging Michail Theologitis Georgios Frangias Georgios Anestis V. Samoladas Antonios Deligiannakis FedML 37 0 0 31 May 2024
Federated Learning with Bilateral Curation for Partially Class-Disjoint Data Ziqing Fan Ruipeng Zhang Jiangchao Yao Bo Han Ya-Qin Zhang Yanfeng Wang FedML 34 12 0 29 May 2024
FedSAC: Dynamic Submodel Allocation for Collaborative Fairness in Federated Learning Zihui Wang Zheng Wang Lingjuan Lyu Zhaopeng Peng Zhicheng Yang Chenglu Wen Rongshan Yu Cheng-i Wang Xiaoliang Fan FedML 35 2 0 28 May 2024
WASH: Train your Ensemble with Communication-Efficient Weight Shuffling, then Average Louis Fournier Adel Nabli Masih Aminbeidokhti M. Pedersoli Eugene Belilovsky Edouard Oyallon MoMe FedML 46 3 0 27 May 2024
Near-Optimal Distributed Minimax Optimization under the Second-Order Similarity Qihao Zhou Haishan Ye Luo Luo 26 0 0 25 May 2024
Distributed Continual Learning Long Le Marcel Hussing Eric Eaton CLL FedML 36 2 0 23 May 2024
Worldwide Federated Training of Language Models Alexandru Iacob Lorenzo Sani Bill Marino Preslav Aleksandrov William F. Shen Nicholas D. Lane FedML 35 2 0 23 May 2024
Decentralized Federated Learning Over Imperfect Communication Channels Weicai Li Tiejun Lv Wei Ni Jingbo Zhao Ekram Hossain H. Vincent Poor FedML 34 3 0 21 May 2024
The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication Kumar Kshitij Patel Margalit Glasgow Ali Zindari Lingxiao Wang Sebastian U. Stich Ziheng Cheng Nirmit Joshi Nathan Srebro 48 6 0 19 May 2024
The Future of Large Language Model Pre-training is Federated Lorenzo Sani Alexandru Iacob Zeyu Cao Bill Marino Yan Gao ... Wanru Zhao William F. Shen Preslav Aleksandrov Xinchi Qiu Nicholas D. Lane AI4CE 35 13 0 17 May 2024
Asynchronous Federated Stochastic Optimization for Heterogeneous Objectives Under Arbitrary Delays Charikleia Iakovidou Kibaek Kim FedML 35 2 0 16 May 2024
FedSC: Provable Federated Self-supervised Learning with Spectral Contrastive Objective over Non-i.i.d. Data Shusen Jing Anlan Yu Shuai Zhang Songyang Zhang FedML 31 1 0 07 May 2024
Interpretable Data Fusion for Distributed Learning: A Representative Approach via Gradient Matching Mengchen Fan Baocheng Geng Keren Li Xueqian Wang Pramod K. Varshney FaML FedML 29 0 0 06 May 2024
Adaptive Heterogeneous Client Sampling for Federated Learning over Wireless Networks Bing Luo Wenli Xiao Shiqiang Wang Jianwei Huang Leandros Tassiulas FedML 33 4 0 22 Apr 2024
PAFedFV: Personalized and Asynchronous Federated Learning for Finger Vein Recognition Hengyu Mu Jian Guo Chong Han Lijuan Sun FedML 24 5 0 20 Apr 2024
Improved Generalization Bounds for Communication Efficient Federated Learning Peyman Gholami H. Seferoglu FedML AI4CE 23 6 0 17 Apr 2024
FedFa: A Fully Asynchronous Training Paradigm for Federated Learning Haotian Xu Zhaorui Zhang Sheng Di Benben Liu Khalid Ayedh Alharthi Jiannong Cao FedML 25 5 0 17 Apr 2024
DIMAT: Decentralized Iterative Merging-And-Training for Deep Learning Models Nastaran Saadati Minh Pham Nasla Saleem Joshua R. Waite Aditya Balu Zhanhong Jiang Chinmay Hegde Soumik Sarkar MoMe 45 1 0 11 Apr 2024
Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey Feng Liang Zhen Zhang Haifeng Lu Victor C. M. Leung Yanyi Guo Xiping Hu GNN 37 6 0 09 Apr 2024
Approximate Gradient Coding for Privacy-Flexible Federated Learning with Non-IID Data Okko Makkonen Sampo Niemelä Camilla Hollanti Serge Kas Hanna 35 0 0 04 Apr 2024
Optimal Batch Allocation for Wireless Federated Learning Jaeyoung Song Sang-Woon Jeon 26 0 0 03 Apr 2024
Stragglers-Aware Low-Latency Synchronous Federated Learning via Layer-Wise Model Updates Natalie Lang Alejandro Cohen Nir Shlezinger FedML 53 4 0 27 Mar 2024
A Selective Review on Statistical Methods for Massive Data Computation: Distributed Computing, Subsampling, and Minibatch Techniques Xuetong Li Yuan Gao Hong Chang Danyang Huang Yingying Ma ... Ke Xu Jing Zhou Xuening Zhu Yingqiu Zhu Hansheng Wang 44 7 0 17 Mar 2024
Cyclic Data Parallelism for Efficient Parallelism of Deep Neural Networks Louis Fournier Edouard Oyallon 43 0 0 13 Mar 2024
MUSIC: Accelerated Convergence for Distributed Optimization With Inexact and Exact Methods Mou Wu Haibin Liao Zhengtao Ding Yonggang Xiao 21 2 0 05 Mar 2024
On the Convergence of Federated Learning Algorithms without Data Similarity Ali Beikmohammadi Sarit Khirirat Sindri Magnússon FedML 33 1 0 29 Feb 2024
Communication Efficient ConFederated Learning: An Event-Triggered SAGA Approach Bin Wang Jun Fang Hongbin Li Yonina C. Eldar FedML 28 1 0 28 Feb 2024
Training Neural Networks from Scratch with Parallel Low-Rank Adapters Minyoung Huh Brian Cheung Jeremy Bernstein Phillip Isola Pulkit Agrawal 35 10 0 26 Feb 2024
Random Aggregate Beamforming for Over-the-Air Federated Learning in Large-Scale Networks Chunmei Xu Shengheng Liu Yongming Huang Björn E. Ottersten Dusist Niyato FedML 21 2 0 20 Feb 2024
Achieving Linear Speedup in Asynchronous Federated Learning with Heterogeneous Clients Xiaolu Wang Zijian Li Shi Jin Jun Zhang FedML 21 3 0 17 Feb 2024
FedLion: Faster Adaptive Federated Optimization with Fewer Communication Zhiwei Tang Tsung-Hui Chang 29 5 0 15 Feb 2024
Federated Learning Can Find Friends That Are Advantageous N. Tupitsa Samuel Horváth Martin Takávc Eduard A. Gorbunov FedML 46 2 0 07 Feb 2024
FedCore: Straggler-Free Federated Learning with Distributed Coresets Hongpeng Guo Haotian Gu Xiaoyang Wang Bo Chen Eun Kyung Lee Tamar Eilam Deming Chen K. Nahrstedt FedML 24 1 0 31 Jan 2024
Decomposable Submodular Maximization in Federated Setting Akbar Rafiey FedML 30 1 0 31 Jan 2024
CO2: Efficient Distributed Training with Full Communication-Computation Overlap Weigao Sun Zhen Qin Weixuan Sun Shidi Li Dong Li Xuyang Shen Yu Qiao Yiran Zhong OffRL 58 10 0 29 Jan 2024
Asynchronous Local-SGD Training for Language Modeling Bo Liu Rachita Chhaparia Arthur Douillard Satyen Kale Andrei A. Rusu Jiajun Shen Arthur Szlam MarcÁurelio Ranzato FedML 40 10 0 17 Jan 2024
Joint Probability Selection and Power Allocation for Federated Learning Ouiame Marnissi Hajar Elhammouti El Houcine Bergou FedML 14 2 0 15 Jan 2024
Lotto: Secure Participant Selection against Adversarial Servers in Federated Learning Zhifeng Jiang Peng Ye Shiqi He Wei Wang Ruichuan Chen Bo Li 28 2 0 05 Jan 2024
Federated Hyperdimensional Computing Kazim Ergun Rishikanth Chandrasekaran Tajana Simunic FedML 15 0 0 26 Dec 2023
Contractive error feedback for gradient compression Bingcong Li Shuai Zheng Parameswaran Raman Anshumali Shrivastava G. Giannakis 15 0 0 13 Dec 2023
Speed Up Federated Learning in Heterogeneous Environment: A Dynamic Tiering Approach Seyed Mahmoud Sajjadi Mohammadabadi Syed Zawad Feng Yan Lei Yang FedML 27 7 0 09 Dec 2023
Merging by Matching Models in Task Parameter Subspaces Derek Tam Mohit Bansal Colin Raffel MoMe 21 10 0 07 Dec 2023
Distributed Continual Learning with CoCoA in High-dimensional Linear Regression Martin Hellkvist Ayça Özçelikkale Anders Ahlén OOD 33 1 0 04 Dec 2023
Can We Learn Communication-Efficient Optimizers? Charles-Étienne Joseph Benjamin Thérien A. Moudgil Boris Knyazev Eugene Belilovsky 40 1 0 02 Dec 2023
Communication-Efficient Heterogeneous Federated Learning with Generalized Heavy-Ball Momentum Riccardo Zaccone Carlo Masone Marco Ciccone FedML 24 2 0 30 Nov 2023