v1v2v3 (latest)

Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training

5 December 2017

Song Han

ArXiv (abs)PDF HTML Github (222★)

Papers citing "Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training"

50 / 625 papers shown

Title
SparDL: Distributed Deep Learning Training with Efficient Sparse Communication Minjun Zhao Yichen Yin Yuren Mao Qing Liu Lu Chen Yunjun Gao 48 1 0 03 Apr 2023
AI-Generated Content (AIGC): A Survey Jiayang Wu Wensheng Gan Zefeng Chen Shicheng Wan Hong Lin 3DV 92 133 0 26 Mar 2023
Delay-Aware Hierarchical Federated Learning F. Lin Seyyedali Hosseinalipour Nicolò Michelusi Christopher G. Brinton FedML 94 12 0 22 Mar 2023
Efficient and Secure Federated Learning for Financial Applications Tao Liu Zhi Wang Hui He Wei Shi Liangliang Lin Wei Shi Ran An Chenhao Li FedML 67 27 0 15 Mar 2023
Complement Sparsification: Low-Overhead Model Pruning for Federated Learning Xiaopeng Jiang Cristian Borcea FedML 83 17 0 10 Mar 2023
Cloudless-Training: A Framework to Improve Efficiency of Geo-Distributed ML Training W. Tan Xiao Shi Cunchi Lv Xiaofang Zhao FedML 58 1 0 09 Mar 2023
FedREP: A Byzantine-Robust, Communication-Efficient and Privacy-Preserving Framework for Federated Learning Yi-Rui Yang Kun Wang Wulu Li FedML 85 3 0 09 Mar 2023
Communication-efficient Federated Learning with Single-Step Synthetic Features Compressor for Faster Convergence Yuhao Zhou Mingjia Shi Yuanxi Li Qing Ye Yanan Sun Jiancheng Lv 49 4 0 27 Feb 2023
DeAR: Accelerating Distributed Deep Learning with Fine-Grained All-Reduce Pipelining Lin Zhang Shaoshuai Shi Xiaowen Chu Wei Wang Yue Liu Chengjian Liu 75 11 0 24 Feb 2023
Advancements in Federated Learning: Models, Methods, and Privacy Hui Chen Huandong Wang Qingyue Long Depeng Jin Yong Li FedML 105 16 0 22 Feb 2023
Magnitude Matters: Fixing SIGNSGD Through Magnitude-Aware Sparsification in the Presence of Data Heterogeneity Richeng Jin Xiaofan He C. Zhong Zhaoyang Zhang Tony Q.S. Quek H. Dai FedML 53 1 0 19 Feb 2023
Multimodal Federated Learning via Contrastive Representation Ensemble Qiying Yu Yang Liu Yimu Wang Ke Xu Jingjing Liu 82 90 0 17 Feb 2023
THC: Accelerating Distributed Deep Learning Using Tensor Homomorphic Compression Minghao Li Ran Ben-Basat S. Vargaftik Chon-In Lao Ke Xu Michael Mitzenmacher Minlan Yu Harvard University 94 19 0 16 Feb 2023
Sparse-SignSGD with Majority Vote for Communication-Efficient Distributed Learning Chanho Park Namyoon Lee FedML 54 4 0 15 Feb 2023
Expediting Distributed DNN Training with Device Topology-Aware Graph Deployment Shiwei Zhang Xiaodong Yi Lansong Diao Chuan Wu Siyu Wang W. Lin GNN 39 5 0 13 Feb 2023
FedPass: Privacy-Preserving Vertical Federated Deep Learning with Adaptive Obfuscation Hanlin Gu Jiahuan Luo Yan Kang Lixin Fan Qiang Yang FedML 94 13 0 30 Jan 2023
SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient Max Ryabinin Tim Dettmers Michael Diskin Alexander Borzunov MoE 111 38 0 27 Jan 2023
Optimus-CC: Efficient Large NLP Model Training with 3D Parallelism Aware Communication Compression Jaeyong Song Jinkyu Yim Jaewon Jung Hongsun Jang H. Kim Youngsok Kim Jinho Lee GNN 74 28 0 24 Jan 2023
M22: A Communication-Efficient Algorithm for Federated Learning Inspired by Rate-Distortion Yangyi Liu Stefano Rini Sadaf Salehkalaibar Jun Chen FedML 48 4 0 23 Jan 2023
ScaDLES: Scalable Deep Learning over Streaming data at the Edge S. Tyagi Martin Swany 52 6 0 21 Jan 2023
Does compressing activations help model parallel training? S. Bian Dacheng Li Hongyi Wang Eric P. Xing Shivaram Venkataraman 72 9 0 06 Jan 2023
Temporal Difference Learning with Compressed Updates: Error-Feedback meets Reinforcement Learning A. Mitra George J. Pappas Hamed Hassani 76 12 0 03 Jan 2023
Mutual Information Regularization for Vertical Federated Learning Tianyuan Zou Yang Liu Ya-Qin Zhang AAML FedML 68 6 0 01 Jan 2023
Deep Hierarchy Quantization Compression algorithm based on Dynamic Sampling W. Jiang Gang Liu Xiaofeng Chen Yipeng Zhou FedML 28 0 0 30 Dec 2022
A Survey on Federated Recommendation Systems Zehua Sun Yonghui Xu Yang Liu Weiliang He Lanju Kong Fangzhao Wu Yiheng Jiang Li-zhen Cui FedML 113 68 0 27 Dec 2022
Adaptive Control of Client Selection and Gradient Compression for Efficient Federated Learning Zhida Jiang Yang Xu Hong-Ze Xu Zhiyuan Wang Chen Qian 54 9 0 19 Dec 2022
ResFed: Communication Efficient Federated Learning by Transmitting Deep Compressed Residuals Rui Song Liguo Zhou Lingjuan Lyu Andreas Festag Alois Knoll FedML 81 5 0 11 Dec 2022
Client Selection for Federated Bayesian Learning Jiarong Yang Yuan Liu Rahif Kassab FedML 66 12 0 11 Dec 2022
Scalable Graph Convolutional Network Training on Distributed-Memory Systems G. Demirci Aparajita Haldar Hakan Ferhatosmanoglu GNN 100 9 0 09 Dec 2022
Vertical Federated Learning: A Structured Literature Review Afsana Khan M. T. Thij A. Wilbik FedML 112 10 0 01 Dec 2022
HashVFL: Defending Against Data Reconstruction Attacks in Vertical Federated Learning Pengyu Qiu Xuhong Zhang S. Ji Chong Fu Xing Yang Ting Wang FedML AAML 128 13 0 01 Dec 2022
Analysis of Error Feedback in Federated Non-Convex Optimization with Biased Compression Xiaoyun Li Ping Li FedML 75 5 0 25 Nov 2022
Vertical Federated Learning: Concepts, Advances and Challenges Yang Liu Yan Kang Tianyuan Zou Yanhong Pu Yuanqin He Xiaozhou Ye Ye Ouyang Yaqin Zhang Qian Yang FedML 187 176 0 23 Nov 2022
FedDCT: Federated Learning of Large Convolutional Neural Networks on Resource Constrained Devices using Divide and Collaborative Training Quan Nguyen Hieu H. Pham Kok-Seng Wong Phi Le Nguyen Truong Thao Nguyen Minh N. Do FedML 100 7 0 20 Nov 2022
Improving Federated Learning Communication Efficiency with Global Momentum Fusion for Gradient Compression Schemes Chun-Chih Kuo Ted T. Kuo Chia-Yu Lin FedML 120 1 0 17 Nov 2022
Optimal Privacy Preserving for Federated Learning in Mobile Edge Computing Hai M. Nguyen N. Chu Diep N. Nguyen D. Hoang Van-Dinh Nguyen Minh Hoàng Hà E. Dutkiewicz Marwan Krunz FedML 61 1 0 14 Nov 2022
Knowledge Distillation for Federated Learning: a Practical Guide Alessio Mora Irene Tenison Paolo Bellavista Irina Rish FedML 66 31 0 09 Nov 2022
QuantPipe: Applying Adaptive Post-Training Quantization for Distributed Transformer Pipelines in Dynamic Edge Environments Hong Wang Connor Imes Souvik Kundu Peter A. Beerel S. Crago J. Walters MQ 59 7 0 08 Nov 2022
HFedMS: Heterogeneous Federated Learning with Memorable Data Semantics in Industrial Metaverse Shenglai Zeng Zonghang Li Hongfang Yu Zhihao Zhang Long Luo Yue Liu Dusit Niyato 105 44 0 07 Nov 2022
On the Interaction Between Differential Privacy and Gradient Compression in Deep Learning Jimmy J. Lin 37 0 0 01 Nov 2022
Adaptive Compression for Communication-Efficient Distributed Training Maksim Makarenko Elnur Gasanov Rustem Islamov Abdurakhmon Sadiev Peter Richtárik 116 16 0 31 Oct 2022
L-GreCo: Layerwise-Adaptive Gradient Compression for Efficient and Accurate Deep Learning Mohammadreza Alimohammadi I. Markov Elias Frantar Dan Alistarh 80 5 0 31 Oct 2022
FedGRec: Federated Graph Recommender System with Lazy Update of Latent Embeddings Junyi Li Heng-Chiao Huang FedML 48 6 0 25 Oct 2022
Federated Learning and Meta Learning: Approaches, Applications, and Directions Xiaonan Liu Yansha Deng Arumugam Nallanathan M. Bennis 116 38 0 24 Oct 2022
Sparse Random Networks for Communication-Efficient Federated Learning Berivan Isik Francesco Pase Deniz Gunduz Tsachy Weissman M. Zorzi FedML 120 53 0 30 Sep 2022
Empirical Analysis on Top-k Gradient Sparsification for Distributed Deep Learning in a Supercomputing Environment Daegun Yoon Sangyoon Oh 85 0 0 18 Sep 2022
Concealing Sensitive Samples against Gradient Leakage in Federated Learning Jing Wu Munawar Hayat Min Zhou Mehrtash Harandi FedML 54 11 0 13 Sep 2022
Convergence of Batch Updating Methods with Approximate Gradients and/or Noisy Measurements: Theory and Computational Results Tadipatri Uday M. Vidyasagar 37 0 0 12 Sep 2022
A simplified convergence theory for Byzantine resilient stochastic gradient descent Lindon Roberts E. Smyth 80 3 0 25 Aug 2022
Federated Learning via Decentralized Dataset Distillation in Resource-Constrained Edge Environments Rui Song Dai Liu Da Chen Andreas Festag Carsten Trinitis Martin Schulz Alois C. Knoll DD FedML 119 66 0 24 Aug 2022