Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training

5 December 2017

Song Han

Papers citing "Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training"

50 / 616 papers shown

Title
Magnitude Matters: Fixing SIGNSGD Through Magnitude-Aware Sparsification in the Presence of Data Heterogeneity Richeng Jin Xiaofan He C. Zhong Zhaoyang Zhang Tony Q.S. Quek H. Dai FedML 24 1 0 19 Feb 2023
Multimodal Federated Learning via Contrastive Representation Ensemble Qiying Yu Yang Liu Yimu Wang Ke Xu Jingjing Liu 37 81 0 17 Feb 2023
THC: Accelerating Distributed Deep Learning Using Tensor Homomorphic Compression Minghao Li Ran Ben-Basat S. Vargaftik Chon-In Lao Ke Xu Michael Mitzenmacher Minlan Yu Harvard University 26 15 0 16 Feb 2023
Sparse-SignSGD with Majority Vote for Communication-Efficient Distributed Learning Chanho Park Namyoon Lee FedML 35 3 0 15 Feb 2023
Expediting Distributed DNN Training with Device Topology-Aware Graph Deployment Shiwei Zhang Xiaodong Yi Lansong Diao Chuan Wu Siyu Wang W. Lin GNN 22 5 0 13 Feb 2023
FedPass: Privacy-Preserving Vertical Federated Deep Learning with Adaptive Obfuscation Hanlin Gu Jiahuan Luo Yan Kang Lixin Fan Qiang Yang FedML 36 13 0 30 Jan 2023
SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient Max Ryabinin Tim Dettmers Michael Diskin Alexander Borzunov MoE 30 31 0 27 Jan 2023
Optimus-CC: Efficient Large NLP Model Training with 3D Parallelism Aware Communication Compression Jaeyong Song Jinkyu Yim Jaewon Jung Hongsun Jang H. Kim Youngsok Kim Jinho Lee GNN 24 25 0 24 Jan 2023
M22: A Communication-Efficient Algorithm for Federated Learning Inspired by Rate-Distortion Yangyi Liu Stefano Rini Sadaf Salehkalaibar Jun Chen FedML 21 4 0 23 Jan 2023
ScaDLES: Scalable Deep Learning over Streaming data at the Edge S. Tyagi Martin Swany 22 6 0 21 Jan 2023
Does compressing activations help model parallel training? S. Bian Dacheng Li Hongyi Wang Eric P. Xing Shivaram Venkataraman 24 5 0 06 Jan 2023
Temporal Difference Learning with Compressed Updates: Error-Feedback meets Reinforcement Learning A. Mitra George J. Pappas Hamed Hassani 31 12 0 03 Jan 2023
Mutual Information Regularization for Vertical Federated Learning Tianyuan Zou Yang Liu Ya-Qin Zhang AAML FedML 35 7 0 01 Jan 2023
Deep Hierarchy Quantization Compression algorithm based on Dynamic Sampling W. Jiang Gang Liu Xiaofeng Chen Yipeng Zhou FedML 19 0 0 30 Dec 2022
A Survey on Federated Recommendation Systems Zehua Sun Yonghui Xu Yong-Jin Liu Weiliang He Lanju Kong Fangzhao Wu Y. Jiang Li-zhen Cui FedML 29 60 0 27 Dec 2022
Adaptive Control of Client Selection and Gradient Compression for Efficient Federated Learning Zhida Jiang Yang Xu Hong-Ze Xu Zhiyuan Wang Chen Qian 20 9 0 19 Dec 2022
ResFed: Communication Efficient Federated Learning by Transmitting Deep Compressed Residuals Rui Song Liguo Zhou Lingjuan Lyu Andreas Festag Alois Knoll FedML 34 5 0 11 Dec 2022
Client Selection for Federated Bayesian Learning Jiarong Yang Yuan Liu Rahif Kassab FedML 41 11 0 11 Dec 2022
Scalable Graph Convolutional Network Training on Distributed-Memory Systems G. Demirci Aparajita Haldar Hakan Ferhatosmanoglu GNN 36 9 0 09 Dec 2022
Vertical Federated Learning: A Structured Literature Review Afsana Khan M. T. Thij A. Wilbik FedML 55 10 0 01 Dec 2022
HashVFL: Defending Against Data Reconstruction Attacks in Vertical Federated Learning Pengyu Qiu Xuhong Zhang S. Ji Chong Fu Xing Yang Ting Wang FedML AAML 32 12 0 01 Dec 2022
Analysis of Error Feedback in Federated Non-Convex Optimization with Biased Compression Xiaoyun Li Ping Li FedML 34 4 0 25 Nov 2022
Vertical Federated Learning: Concepts, Advances and Challenges Yang Liu Yan Kang Tianyuan Zou Yanhong Pu Yuanqin He Xiaozhou Ye Ye Ouyang Yaqin Zhang Qian Yang FedML 64 162 0 23 Nov 2022
FedDCT: Federated Learning of Large Convolutional Neural Networks on Resource Constrained Devices using Divide and Collaborative Training Quan Nguyen Hieu H. Pham Kok-Seng Wong Phi Le Nguyen Truong Thao Nguyen Minh N. Do FedML 27 7 0 20 Nov 2022
Improving Federated Learning Communication Efficiency with Global Momentum Fusion for Gradient Compression Schemes Chun-Chih Kuo Ted T. Kuo Chia-Yu Lin FedML 18 1 0 17 Nov 2022
Optimal Privacy Preserving for Federated Learning in Mobile Edge Computing Hai M. Nguyen N. Chu Diep N. Nguyen D. Hoang Van-Dinh Nguyen Minh Hoàng Hà E. Dutkiewicz Marwan Krunz FedML 27 1 0 14 Nov 2022
Knowledge Distillation for Federated Learning: a Practical Guide Alessio Mora Irene Tenison Paolo Bellavista Irina Rish FedML 25 17 0 09 Nov 2022
QuantPipe: Applying Adaptive Post-Training Quantization for Distributed Transformer Pipelines in Dynamic Edge Environments Hong Wang Connor Imes Souvik Kundu P. Beerel S. Crago J. Walters MQ 21 7 0 08 Nov 2022
HFedMS: Heterogeneous Federated Learning with Memorable Data Semantics in Industrial Metaverse Shenglai Zeng Zonghang Li Hongfang Yu Zhihao Zhang Long Luo Bo-wen Li Dusit Niyato 44 42 0 07 Nov 2022
On the Interaction Between Differential Privacy and Gradient Compression in Deep Learning Jimmy J. Lin 19 0 0 01 Nov 2022
Adaptive Compression for Communication-Efficient Distributed Training Maksim Makarenko Elnur Gasanov Rustem Islamov Abdurakhmon Sadiev Peter Richtárik 44 14 0 31 Oct 2022
L-GreCo: Layerwise-Adaptive Gradient Compression for Efficient and Accurate Deep Learning Mohammadreza Alimohammadi I. Markov Elias Frantar Dan Alistarh 35 5 0 31 Oct 2022
FedGRec: Federated Graph Recommender System with Lazy Update of Latent Embeddings Junyi Li Heng-Chiao Huang FedML 24 6 0 25 Oct 2022
Federated Learning and Meta Learning: Approaches, Applications, and Directions Xiaonan Liu Yansha Deng Arumugam Nallanathan M. Bennis 64 32 0 24 Oct 2022
Sparse Random Networks for Communication-Efficient Federated Learning Berivan Isik Francesco Pase Deniz Gunduz Tsachy Weissman M. Zorzi FedML 70 52 0 30 Sep 2022
Empirical Analysis on Top-k Gradient Sparsification for Distributed Deep Learning in a Supercomputing Environment Daegun Yoon Sangyoon Oh 26 0 0 18 Sep 2022
Concealing Sensitive Samples against Gradient Leakage in Federated Learning Jing Wu Munawar Hayat Min Zhou Mehrtash Harandi FedML 16 10 0 13 Sep 2022
Convergence of Batch Updating Methods with Approximate Gradients and/or Noisy Measurements: Theory and Computational Results Tadipatri Uday M. Vidyasagar 28 0 0 12 Sep 2022
A simplified convergence theory for Byzantine resilient stochastic gradient descent Lindon Roberts E. Smyth 31 3 0 25 Aug 2022
Federated Learning via Decentralized Dataset Distillation in Resource-Constrained Edge Environments Rui Song Dai Liu Da Chen Andreas Festag Carsten Trinitis Martin Schulz Alois C. Knoll DD FedML 28 62 0 24 Aug 2022
Joint Privacy Enhancement and Quantization in Federated Learning Natalie Lang Elad Sofer Tomer Shaked Nir Shlezinger FedML 37 46 0 23 Aug 2022
A Fast Blockchain-based Federated Learning Framework with Compressed Communications Laizhong Cui Xiaoxin Su Yipeng Zhou FedML 22 23 0 12 Aug 2022
Scalable neural quantum states architecture for quantum chemistry Tianchen Zhao J. Stokes S. Veerapaneni 18 30 0 11 Aug 2022
Quantized Adaptive Subgradient Algorithms and Their Applications Ke Xu Jianqiao Wangni Yifan Zhang Deheng Ye Jiaxiang Wu P. Zhao 36 0 0 11 Aug 2022
FedOBD: Opportunistic Block Dropout for Efficiently Training Large-scale Neural Networks through Federated Learning Yuanyuan Chen Zichen Chen Pengcheng Wu Han Yu AI4CE 22 18 0 10 Aug 2022
Towards Efficient Communications in Federated Learning: A Contemporary Survey Zihao Zhao Yuzhu Mao Yang Liu Linqi Song Ouyang Ye Xinlei Chen Wenbo Ding FedML 59 60 0 02 Aug 2022
Parameter-Parallel Distributed Variational Quantum Algorithm Yun-Fei Niu Shuo Zhang Chen Ding Wansu Bao Heliang Huang 27 4 0 31 Jul 2022
BiFeat: Supercharge GNN Training via Graph Feature Quantization Yuxin Ma Ping Gong Jun Yi Z. Yao Cheng-rong Li Yuxiong He Feng Yan GNN 21 6 0 29 Jul 2022
CFLIT: Coexisting Federated Learning and Information Transfer Zehong Lin Hang Liu Y. Zhang 14 11 0 26 Jul 2022
Reconciling Security and Communication Efficiency in Federated Learning Karthik Prasad Sayan Ghosh Graham Cormode Ilya Mironov Ashkan Yousefpour Pierre Stock FedML 35 8 0 26 Jul 2022