Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training

5 December 2017

Song Han

Papers citing "Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training"

50 / 615 papers shown

Title
ZenFlow: Enabling Stall-Free Offloading Training via Asynchronous Updates Tingfeng Lan Yusen Wu Bin Ma Zhaoyuan Su Rui Yang Tekin Bicer Dong Li Yue Cheng 2 0 0 18 May 2025
Communication-Efficient Wireless Federated Fine-Tuning for Large-Scale AI Models Bumjun Kim Wan Choi 34 0 0 01 May 2025
BackSlash: Rate Constrained Optimized Training of Large Language Models Jun Wu Jiangtao Wen Yuxing Han 39 0 0 23 Apr 2025
TAGC: Optimizing Gradient Communication in Distributed Transformer Training Igor Polyakov Alexey Dukhanov Egor Spirin 46 0 0 08 Apr 2025
Towards Optimal Heterogeneous Client Sampling in Multi-Model Federated Learning Haoran Zhang Zejun Gong Zekai Li Marie Siew Carlee Joe-Wong Rachid El-Azouzi 31 0 0 07 Apr 2025
FedSAUC: A Similarity-Aware Update Control for Communication-Efficient Federated Learning in Edge Computing Ming-Lun Lee Han-Chang Chou Yan-AnnChen FedML 36 6 0 07 Apr 2025
UniCoRN: Latent Diffusion-based Unified Controllable Image Restoration Network across Multiple Degradations Debabrata Mandal Soumitri Chattopadhyay Guansen Tong Praneeth Chakravarthula DiffM 57 0 0 20 Mar 2025
Trading-off Accuracy and Communication Cost in Federated Learning Mattia Jacopo Villani Emanuele Natale Frederik Mallmann-Trenn FedML 80 0 0 18 Mar 2025
Accelerated Distributed Optimization with Compression and Error Feedback Yuan Gao Anton Rodomanov Jeremy Rack Sebastian U. Stich 51 0 0 11 Mar 2025
Secure Aggregation in Federated Learning using Multiparty Homomorphic Encryption Erfan Hosseini Shuangyi Chen Ashish Khisti FedML 31 0 0 01 Mar 2025
Delayed Random Partial Gradient Averaging for Federated Learning Xinyi Hu FedML 43 0 0 31 Dec 2024
MARINA-P: Superior Performance in Non-smooth Federated Optimization with Adaptive Stepsizes Igor Sokolov Peter Richtárik 77 1 0 22 Dec 2024
Fed-AugMix: Balancing Privacy and Utility via Data Augmentation HaoYang Li Wei Chen Xiaojin Zhang FedML 78 0 0 18 Dec 2024
Adaptive Client Selection with Personalization for Communication Efficient Federated Learning Allan M. de Souza Filipe Maciel Joahannes B. D. da Costa L. Bittencourt E. Cerqueira A. A. F. Loureiro Leandro A. Villas FedML 101 16 0 26 Nov 2024
Lion Cub: Minimizing Communication Overhead in Distributed Lion Satoki Ishikawa Tal Ben-Nun B. Van Essen Rio Yokota Nikoli Dryden 82 0 0 25 Nov 2024
Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis Zhijie Chen Qiaobo Li A. Banerjee FedML 37 0 0 11 Nov 2024
Optimal Defenses Against Gradient Reconstruction Attacks Yuxiao Chen Gamze Gürsoy Qi Lei FedML AAML 31 0 0 06 Nov 2024
$r$ Age- $k$ : Communication-Efficient Federated Learning Using Age Factor Matin Mortaheb Priyanka Kaswan S. Ulukus FedML 36 1 0 29 Oct 2024
Trustworthiness of Stochastic Gradient Descent in Distributed Learning Hongyang Li Caesar Wu Mohammed Chadli Said Mammar Pascal Bouvry 56 1 0 28 Oct 2024
LDAdam: Adaptive Optimization from Low-Dimensional Gradient Statistics Thomas Robert M. Safaryan Ionut-Vlad Modoranu Dan Alistarh ODL 36 2 0 21 Oct 2024
FusionLLM: A Decentralized LLM Training System on Geo-distributed GPUs with Adaptive Compression Zhenheng Tang Xueze Kang Yiming Yin Xinglin Pan Yuxin Wang ... Shaohuai Shi Amelie Chi Zhou Bo Li Bingsheng He Xiaowen Chu AI4CE 81 8 0 16 Oct 2024
A Comprehensive Survey on Joint Resource Allocation Strategies in Federated Edge Learning Jingbo Zhang Qiong Wu Pingyi Fan Q. Fan FedML 33 0 0 10 Oct 2024
Private and Communication-Efficient Federated Learning based on Differentially Private Sketches Meifan Zhang Zhanhong Xie Lihua Yin FedML 29 1 0 08 Oct 2024
No Need to Talk: Asynchronous Mixture of Language Models Anastasiia Filippova Angelos Katharopoulos David Grangier Ronan Collobert MoE 44 0 0 04 Oct 2024
Temporal Predictive Coding for Gradient Compression in Distributed Learning Adrian Edin Zheng Chen Michel Kieffer Mikael Johansson 32 1 0 03 Oct 2024
Novel Gradient Sparsification Algorithm via Bayesian Inference Ali Bereyhi B. Liang G. Boudreau Ali Afana 34 2 0 23 Sep 2024
Data Poisoning and Leakage Analysis in Federated Learning Wenqi Wei Tiansheng Huang Zachary Yahn Anoop Singhal Margaret Loper Ling Liu FedML SILM 33 0 0 19 Sep 2024
Advances in APPFL: A Comprehensive and Extensible Federated Learning Framework Zilinghan Li Shilan He Ze Yang Minseok Ryu Kibaek Kim Ravi K. Madduri FedML 60 6 0 17 Sep 2024
S-STE: Continuous Pruning Function for Efficient 2:4 Sparse Pre-training Yuezhou Hu Jun-Jie Zhu Jianfei Chen 40 0 0 13 Sep 2024
FedFQ: Federated Learning with Fine-Grained Quantization Haowei Li Weiying Xie Hangyu Ye Jitao Ma Shuran Ma Yunsong Li FedML MQ 33 1 0 16 Aug 2024
Rina: Enhancing Ring-AllReduce with In-network Aggregation in Distributed Model Training Zixuan Chen Xuandong Liu Minglin Li Yinfan Hu Hao Mei Huifeng Xing Hao Wang Wanxin Shi Sen Liu Yang Xu 32 0 0 29 Jul 2024
Ordered Momentum for Asynchronous SGD Chang-Wei Shi Yi-Rui Yang Wu-Jun Li ODL 59 0 0 27 Jul 2024
Toward INT4 Fixed-Point Training via Exploring Quantization Error for Gradients Dohyung Kim Junghyup Lee Jeimin Jeon Jaehyeon Moon Bumsub Ham MQ 37 0 0 17 Jul 2024
Scalable and Reliable Over-the-Air Federated Edge Learning Maximilian Egger Christoph Hofmeister Cem Kaya Rawad Bitar Antonia Wachter-Zeh 44 1 0 16 Jul 2024
CCVA-FL: Cross-Client Variations Adaptive Federated Learning for Medical Imaging Sunny Gupta Amit Sethi MedIm 24 0 0 16 Jul 2024
Foundation Model Engineering: Engineering Foundation Models Just as Engineering Software Dezhi Ran Mengzhou Wu Wei Yang Tao Xie AI4CE 39 1 0 11 Jul 2024
A Joint Approach to Local Updating and Gradient Compression for Efficient Asynchronous Federated Learning Jiajun Song Jiajun Luo Rongwei Lu Shuzhao Xie Bin Chen Zhi Wang FedML 28 0 0 06 Jul 2024
VcLLM: Video Codecs are Secretly Tensor Codecs Ceyu Xu Yongji Wu Xinyu Yang Beidi Chen Matthew Lentz Danyang Zhuo Lisa Wu Wills 50 0 0 29 Jun 2024
FedAQ: Communication-Efficient Federated Edge Learning via Joint Uplink and Downlink Adaptive Quantization Linping Qu Shenghui Song Chi-Ying Tsui MQ FedML 26 4 0 26 Jun 2024
Grass: Compute Efficient Low-Memory LLM Training with Structured Sparse Gradients Aashiq Muhamed Oscar Li David Woodruff Mona Diab Virginia Smith 61 7 0 25 Jun 2024
Breaking Secure Aggregation: Label Leakage from Aggregated Gradients in Federated Learning Zhibo Wang Zhiwei Chang Jiahui Hu Xiaoyi Pang Jiacheng Du Yongle Chen Kui Ren FedML 29 1 0 22 Jun 2024
A Resource-Adaptive Approach for Federated Learning under Resource-Constrained Environments Ruirui Zhang Xingze Wu Yifei Zou Zhenzhen Xie Peng Li Xiuzhen Cheng Dongxiao Yu FedML 36 0 0 19 Jun 2024
Communication-Efficient Federated Knowledge Graph Embedding with Entity-Wise Top-K Sparsification Xiaoxiong Zhang Zhiwei Zeng Xin Zhou Dusit Niyato Zhiqi Shen FedML 31 1 0 19 Jun 2024
DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving Yuxuan Tong Xiwen Zhang Rui Wang R. Wu Junxian He AIMat LRM 46 32 0 18 Jun 2024
EncCluster: Scalable Functional Encryption in Federated Learning through Weight Clustering and Probabilistic Filters Vasileios Tsouvalas Samaneh Mohammadi Ali Balador T. Ozcelebi Francesco Flammini N. Meratnia FedML 38 0 0 13 Jun 2024
Asynchronous Byzantine Federated Learning Bart Cox Abele Malan Lydia Y. Chen Jérémie Decouchant 50 1 0 03 Jun 2024
FedAdOb: Privacy-Preserving Federated Deep Learning with Adaptive Obfuscation Hanlin Gu Jiahuan Luo Yan Kang Yuan Yao Gongxi Zhu Bowen Li Jie Li Lixin Fan Qiang Yang 47 0 0 03 Jun 2024
Communication-Efficient Distributed Deep Learning via Federated Dynamic Averaging Michail Theologitis Georgios Frangias Georgios Anestis V. Samoladas Antonios Deligiannakis FedML 40 0 0 31 May 2024
Federated Learning with Multi-resolution Model Broadcast Henrik Rydén Reza Moosavi Erik G. Larsson 42 1 0 30 May 2024
VeLoRA: Memory Efficient Training using Rank-1 Sub-Token Projections Roy Miles Pradyumna Reddy Ismail Elezi Jiankang Deng VLM 43 3 0 28 May 2024