Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computations

6 June 2019

Suhas Diggavi

Papers citing "Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computations"

50 / 84 papers shown

Title
Communication-Efficient Federated Fine-Tuning of Language Models via Dynamic Update Schedules Michail Theologitis V. Samoladas Antonios Deligiannakis 34 0 0 07 May 2025
Convergence Analysis of Asynchronous Federated Learning with Gradient Compression for Non-Convex Optimization Diying Yang Yingwei Hou Danyang Xiao Weigang Wu FedML 39 0 0 28 Apr 2025
TAGC: Optimizing Gradient Communication in Distributed Transformer Training Igor Polyakov Alexey Dukhanov Egor Spirin 43 0 0 08 Apr 2025
Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis Zhijie Chen Qiaobo Li A. Banerjee FedML 37 0 0 11 Nov 2024
Selective Aggregation for Low-Rank Adaptation in Federated Learning Pengxin Guo Shuang Zeng Y. Wang Huijie Fan Feifei Wang Liangqiong Qu FedML 44 10 0 02 Oct 2024
VeLoRA: Memory Efficient Training using Rank-1 Sub-Token Projections Roy Miles Pradyumna Reddy Ismail Elezi Jiankang Deng VLM 43 3 0 28 May 2024
SignSGD with Federated Voting Chanho Park H. Vincent Poor Namyoon Lee FedML 40 1 0 25 Mar 2024
LoCoDL: Communication-Efficient Distributed Learning with Local Training and Compression Laurent Condat A. Maranjyan Peter Richtárik 47 4 0 07 Mar 2024
Fed-CVLC: Compressing Federated Learning Communications with Variable-Length Codes Xiaoxin Su Yipeng Zhou Laizhong Cui John C. S. Lui Jiangchuan Liu FedML 35 1 0 06 Feb 2024
RS-DGC: Exploring Neighborhood Statistics for Dynamic Gradient Compression on Remote Sensing Image Interpretation Weiying Xie Zixuan Wang Jitao Ma Daixun Li Yunsong Li 30 0 0 29 Dec 2023
Sparse Training for Federated Learning with Regularized Error Correction Ran Greidi Kobi Cohen FedML 30 1 0 21 Dec 2023
Layered Randomized Quantization for Communication-Efficient and Privacy-Preserving Distributed Learning Guangfeng Yan Tan Li Tian-Shing Lan Kui Wu Linqi Song 19 6 0 12 Dec 2023
AirFL-Mem: Improving Communication-Learning Trade-Off by Long-Term Memory Haifeng Wen Hong Xing Osvaldo Simeone 40 0 0 25 Oct 2023
Communication Compression for Byzantine Robust Learning: New Efficient Algorithms and Improved Rates Ahmad Rammal Kaja Gruntkowska Nikita Fedin Eduard A. Gorbunov Peter Richtárik 45 5 0 15 Oct 2023
Information-Theoretically Private Federated Submodel Learning with Storage Constrained Databases Sajani Vithana S. Ulukus FedML 20 0 0 12 Jul 2023
Benchmarking Algorithms for Federated Domain Generalization Ruqi Bai S. Bagchi David I. Inouye FedML 40 12 0 11 Jul 2023
Federated Multi-Sequence Stochastic Approximation with Local Hypergradient Estimation Davoud Ataee Tarzanagh Mingchen Li Pranay Sharma Samet Oymak 26 0 0 02 Jun 2023
Convergence Analysis of Over-the-Air FL with Compression and Power Control via Clipping Haifeng Wen Hong Xing Osvaldo Simeone 30 1 0 18 May 2023
Lower Bounds and Accelerated Algorithms in Distributed Stochastic Optimization with Communication Compression Yutong He Xinmeng Huang Yiming Chen W. Yin Kun Yuan 31 7 0 12 May 2023
FedAVO: Improving Communication Efficiency in Federated Learning with African Vultures Optimizer Md Zarif Hossain Ahmed Imteaj FedML 29 5 0 02 May 2023
Federated Gradient Matching Pursuit Halyun Jeong Deanna Needell Jing Qin FedML 37 1 0 20 Feb 2023
Similarity, Compression and Local Steps: Three Pillars of Efficient Communications for Distributed Variational Inequalities Aleksandr Beznosikov Martin Takáč Alexander Gasnikov 29 10 0 15 Feb 2023
On the Convergence of Federated Averaging with Cyclic Client Participation Yae Jee Cho Pranay Sharma Gauri Joshi Zheng Xu Satyen Kale Tong Zhang FedML 44 27 0 06 Feb 2023
Does Federated Learning Really Need Backpropagation? H. Feng Tianyu Pang Chao Du Wei Chen Shuicheng Yan Min-Bin Lin FedML 36 10 0 28 Jan 2023
Optimus-CC: Efficient Large NLP Model Training with 3D Parallelism Aware Communication Compression Jaeyong Song Jinkyu Yim Jaewon Jung Hongsun Jang H. Kim Youngsok Kim Jinho Lee GNN 24 25 0 24 Jan 2023
Network Adaptive Federated Learning: Congestion and Lossy Compression Parikshit Hegde G. Veciana Aryan Mokhtari FedML 20 4 0 11 Jan 2023
Federated Learning with Flexible Control Shiqiang Wang Jake B. Perazzone Mingyue Ji Kevin S. Chan FedML 28 17 0 16 Dec 2022
SplitGP: Achieving Both Generalization and Personalization in Federated Learning Dong-Jun Han Do-Yeon Kim Minseok Choi Christopher G. Brinton Jaekyun Moon FedML 26 31 0 16 Dec 2022
GlueFL: Reconciling Client Sampling and Model Masking for Bandwidth Efficient Federated Learning Shiqi He Qifan Yan Feijie Wu Lanjun Wang Mathias Lécuyer Ivan Beschastnikh FedML 42 7 0 03 Dec 2022
Analysis of Error Feedback in Federated Non-Convex Optimization with Biased Compression Xiaoyun Li Ping Li FedML 34 4 0 25 Nov 2022
Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challenges Enrique Tomás Martínez Beltrán Mario Quiles Pérez Pedro Miguel Sánchez Sánchez Sergio López Bernal Gérome Bovet M. Pérez Gregorio Martínez Pérez Alberto Huertas Celdrán FedML 35 222 0 15 Nov 2022
Privacy-Aware Compression for Federated Learning Through Numerical Mechanism Design Chuan Guo Kamalika Chaudhuri Pierre Stock Michael G. Rabbat FedML 33 7 0 08 Nov 2022
FeDXL: Provable Federated Learning for Deep X-Risk Optimization Zhishuai Guo R. L. Jin Jiebo Luo Tianbao Yang FedML 47 8 0 26 Oct 2022
Private Read Update Write (PRUW) in Federated Submodel Learning (FSL): Communication Efficient Schemes With and Without Sparsification Sajani Vithana S. Ulukus FedML 20 19 0 09 Sep 2022
A Fast Blockchain-based Federated Learning Framework with Compressed Communications Laizhong Cui Xiaoxin Su Yipeng Zhou FedML 22 23 0 12 Aug 2022
Federated Optimization Algorithms with Random Reshuffling and Gradient Compression Abdurakhmon Sadiev Grigory Malinovsky Eduard A. Gorbunov Igor Sokolov Ahmed Khaled Konstantin Burlachenko Peter Richtárik FedML 16 21 0 14 Jun 2022
Private Federated Submodel Learning with Sparsification Sajani Vithana S. Ulukus FedML 26 10 0 31 May 2022
Federated Random Reshuffling with Compression and Variance Reduction Grigory Malinovsky Peter Richtárik FedML 27 10 0 08 May 2022
Communication-Efficient Adaptive Federated Learning Yujia Wang Lu Lin Jinghui Chen FedML 27 71 0 05 May 2022
Byzantine Fault Tolerance in Distributed Machine Learning : a Survey Djamila Bouhata Hamouma Moumen Moumen Hamouma Ahcène Bounceur AI4CE 27 7 0 05 May 2022
FedNest: Federated Bilevel, Minimax, and Compositional Optimization Davoud Ataee Tarzanagh Mingchen Li Christos Thrampoulidis Samet Oymak FedML 42 73 0 04 May 2022
FedSynth: Gradient Compression via Synthetic Data in Federated Learning Shengyuan Hu Jack Goetz Kshitiz Malik Hongyuan Zhan Zhe Liu Yue Liu DD FedML 37 38 0 04 Apr 2022
Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam Yucheng Lu Conglong Li Minjia Zhang Christopher De Sa Yuxiong He OffRL AI4CE 24 20 0 12 Feb 2022
Distributed Learning With Sparsified Gradient Differences Yicheng Chen Rick S. Blum Martin Takáč Brian M. Sadler 31 15 0 05 Feb 2022
TOFU: Towards Obfuscated Federated Updates by Encoding Weight Updates into Gradients from Proxy Data Isha Garg M. Nagaraj Kaushik Roy FedML 31 1 0 21 Jan 2022
Decentralized Multi-Task Stochastic Optimization With Compressed Communications Navjot Singh Xuanyu Cao Suhas Diggavi Tamer Basar 18 9 0 23 Dec 2021
FastSGD: A Fast Compressed SGD Framework for Distributed Machine Learning Keyu Yang Lu Chen Zhihao Zeng Yunjun Gao 20 9 0 08 Dec 2021
Communication-Efficient Distributed Learning via Sparse and Adaptive Stochastic Gradient Xiaoge Deng Dongsheng Li Tao Sun Xicheng Lu FedML 26 0 0 08 Dec 2021
What Do We Mean by Generalization in Federated Learning? Honglin Yuan Warren Morningstar Lin Ning K. Singhal OOD FedML 41 71 0 27 Oct 2021
Leveraging Spatial and Temporal Correlations in Sparsified Mean Estimation Divyansh Jhunjhunwala Ankur Mallick Advait Gadhikar S. Kadhe Gauri Joshi 24 10 0 14 Oct 2021