Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training

5 December 2017

Song Han

Papers citing "Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training"

50 / 616 papers shown

Title
Federated Graph Machine Learning: A Survey of Concepts, Techniques, and Applications Xingbo Fu Binchi Zhang Yushun Dong Chen Chen Jundong Li FedML OOD AI4CE 42 35 0 24 Jul 2022
Training Transformers Together Alexander Borzunov Max Ryabinin Tim Dettmers Quentin Lhoest Lucile Saulnier Michael Diskin Yacine Jernite Thomas Wolf ViT 31 8 0 07 Jul 2022
Fundamental Limits of Communication Efficiency for Model Aggregation in Distributed Learning: A Rate-Distortion Approach Naifu Zhang M. Tao Jia Wang Fan Xu 19 13 0 28 Jun 2022
Efficient Adaptive Federated Optimization of Federated Learning for IoT Zunming Chen Hongyan Cui Ensen Wu Yu Xi 27 0 0 23 Jun 2022
sqSGD: Locally Private and Communication Efficient Federated Learning Yan Feng Tao Xiong Ruofan Wu Lingjuan Lv Leilei Shi FedML 31 2 0 21 Jun 2022
Compressed-VFL: Communication-Efficient Learning with Vertically Partitioned Data Timothy Castiglia Anirban Das Shiqiang Wang S. Patterson FedML 27 48 0 16 Jun 2022
Matching Pursuit Based Scheduling for Over-the-Air Federated Learning Ali Bereyhi Adela Vagollari S. Asaad R. Muller W. Gerstacker H. Vincent Poor 28 6 0 14 Jun 2022
Communication-Efficient Federated Learning over MIMO Multiple Access Channels Yo-Seb Jeon Mohammad Mohammadi Amiri Namyoon Lee 19 18 0 12 Jun 2022
Communication-Efficient Robust Federated Learning with Noisy Labels Junyi Li Jian Pei Heng Huang FedML 34 18 0 11 Jun 2022
Lower Bounds and Nearly Optimal Algorithms in Distributed Learning with Communication Compression Xinmeng Huang Yiming Chen W. Yin Kun Yuan 30 25 0 08 Jun 2022
Towards Fair Federated Recommendation Learning: Characterizing the Inter-Dependence of System and Data Heterogeneity Kiwan Maeng Haiyu Lu Luca Melis John Nguyen Michael G. Rabbat Carole-Jean Wu FedML 45 31 0 30 May 2022
ByteComp: Revisiting Gradient Compression in Distributed Training Zhuang Wang Yanghua Peng Yibo Zhu T. Ng 18 2 0 28 May 2022
Stochastic Gradient Methods with Compressed Communication for Decentralized Saddle Point Problems Chhavi Sharma Vishnu Narayanan P. Balamurugan 16 2 0 28 May 2022
QUIC-FL: Quick Unbiased Compression for Federated Learning Ran Ben-Basat S. Vargaftik Amit Portnoy Gil Einziger Y. Ben-Itzhak Michael Mitzenmacher FedML 72 13 0 26 May 2022
BlockDFL: A Blockchain-based Fully Decentralized Peer-to-Peer Federated Learning Framework Zhen Qin Xueqiang Yan Mengchu Zhou Shuiguang Deng 25 12 0 21 May 2022
Federated learning: Applications, challenges and future directions Subrato Bharati Hossain Mondal Prajoy Podder V. B. Surya Prasath FedML 39 53 0 18 May 2022
On Distributed Adaptive Optimization with Gradient Compression Xiaoyun Li Belhal Karimi Ping Li 20 25 0 11 May 2022
Federated Random Reshuffling with Compression and Variance Reduction Grigory Malinovsky Peter Richtárik FedML 29 10 0 08 May 2022
Over-the-Air Federated Multi-Task Learning via Model Sparsification and Turbo Compressed Sensing Haoming Ma Xiaojun Yuan Z. Ding Dian Fan Jun Fang 15 1 0 08 May 2022
Training Mixed-Domain Translation Models via Federated Learning Peyman Passban Tanya Roosta Rahul Gupta Ankit R. Chadha Clement Chung FedML AI4CE 29 18 0 03 May 2022
MiCS: Near-linear Scaling for Training Gigantic Model on Public Cloud Zhen Zhang Shuai Zheng Yida Wang Justin Chiu George Karypis Trishul Chilimbi Mu Li Xin Jin 21 39 0 30 Apr 2022
A review of Federated Learning in Intrusion Detection Systems for IoT Aitor Belenguer J. Navaridas J. A. Pascual 28 15 0 26 Apr 2022
Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications Han Cai Ji Lin Chengyue Wu Zhijian Liu Haotian Tang Hanrui Wang Ligeng Zhu Song Han 27 108 0 25 Apr 2022
FedVQCS: Federated Learning via Vector Quantized Compressed Sensing Yong-Nam Oh Yo-Seb Jeon Mingzhe Chen Walid Saad FedML 33 11 0 16 Apr 2022
Adaptive Differential Filters for Fast and Communication-Efficient Federated Learning Daniel Becking H. Kirchhoffer G. Tech Paul Haase Karsten Müller H. Schwarz Wojciech Samek FedML 29 4 0 09 Apr 2022
Accelerating Federated Edge Learning via Topology Optimization Shanfeng Huang Zezhong Zhang Shuai Wang Rui-cang Wang Kaibin Huang FedML 25 13 0 01 Apr 2022
Auditing Privacy Defenses in Federated Learning via Generative Gradient Leakage Zhuohang Li Jiaxin Zhang Lu Liu Jian-Dong Liu FedML 38 114 0 29 Mar 2022
Addressing Client Drift in Federated Continual Learning with Adaptive Optimization Yeshwanth Venkatesha Youngeun Kim Hyoungseob Park Yuhang Li Priyadarshini Panda FedML 13 8 0 24 Mar 2022
Local Stochastic Factored Gradient Descent for Distributed Quantum State Tomography J. Kim Taha Toghani César A. Uribe Anastasios Kyrillidis 35 3 0 22 Mar 2022
Similarity-based Label Inference Attack against Training and Inference of Split Learning Junlin Liu Xinchen Lyu Qimei Cui Xiaofeng Tao FedML 35 26 0 10 Mar 2022
Correlated quantization for distributed mean estimation and optimization A. Suresh Ziteng Sun Jae Hun Ro Felix X. Yu 33 12 0 09 Mar 2022
The Fundamental Price of Secure Aggregation in Differentially Private Federated Learning Wei-Ning Chen Christopher A. Choquette-Choo Peter Kairouz A. Suresh FedML 42 63 0 07 Mar 2022
Differentially Private Federated Learning with Local Regularization and Sparsification Anda Cheng Peisong Wang Xi Sheryl Zhang Jian Cheng FedML 28 71 0 07 Mar 2022
Linear Stochastic Bandits over a Bit-Constrained Channel A. Mitra Hamed Hassani George J. Pappas 44 8 0 02 Mar 2022
DropIT: Dropping Intermediate Tensors for Memory-Efficient DNN Training Joya Chen Kai Xu Yuhui Wang Yifei Cheng Angela Yao 19 7 0 28 Feb 2022
No One Left Behind: Inclusive Federated Learning over Heterogeneous Devices Ruixuan Liu Fangzhao Wu Chuhan Wu Yanlin Wang Lingjuan Lyu Hong Chen Xing Xie FedML 19 70 0 16 Feb 2022
OLIVE: Oblivious Federated Learning on Trusted Execution Environment against the risk of sparsification Fumiyuki Kato Yang Cao Masatoshi Yoshikawa FedML 29 5 0 15 Feb 2022
BEAS: Blockchain Enabled Asynchronous & Secure Federated Machine Learning A. Mondal Harpreet Virk Debayan Gupta 42 15 0 06 Feb 2022
Distributed Learning With Sparsified Gradient Differences Yicheng Chen Rick S. Blum Martin Takáč Brian M. Sadler 31 15 0 05 Feb 2022
Improved Information Theoretic Generalization Bounds for Distributed and Federated Learning L. P. Barnes Alex Dytso H. V. Poor FedML 41 16 0 04 Feb 2022
3PC: Three Point Compressors for Communication-Efficient Distributed Training and a Better Theory for Lazy Aggregation Peter Richtárik Igor Sokolov Ilyas Fatkhullin Elnur Gasanov Zhize Li Eduard A. Gorbunov 23 31 0 02 Feb 2022
TopoOpt: Co-optimizing Network Topology and Parallelization Strategy for Distributed Training Jobs Weiyang Wang Moein Khazraee Zhizhen Zhong M. Ghobadi Zhihao Jia Dheevatsa Mudigere Ying Zhang A. Kewitsch 39 85 0 01 Feb 2022
Faster Convergence of Local SGD for Over-Parameterized Models Tiancheng Qin S. Rasoul Etesami César A. Uribe FedML 38 6 0 30 Jan 2022
Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges Nuria Rodríguez-Barroso Daniel Jiménez López M. V. Luzón Francisco Herrera Eugenio Martínez-Cámara FedML 37 213 0 20 Jan 2022
DEFER: Distributed Edge Inference for Deep Neural Networks Arjun Parthasarathy Bhaskar Krishnamachari 24 14 0 18 Jan 2022
Egeria: Efficient DNN Training with Knowledge-Guided Layer Freezing Yiding Wang D. Sun Kai Chen Fan Lai Mosharaf Chowdhury 33 44 0 17 Jan 2022
Optimizing the Communication-Accuracy Trade-off in Federated Learning with Rate-Distortion Theory Nicole Mitchell Johannes Ballé Zachary B. Charles Jakub Konecný FedML 19 21 0 07 Jan 2022
Automatic Configuration for Optimal Communication Scheduling in DNN Training Yiqing Ma Hao Wang Yiming Zhang Kai Chen 22 12 0 27 Dec 2021
Sparsified Secure Aggregation for Privacy-Preserving Federated Learning Irem Ergun Hasin Us Sami Başak Güler FedML 41 26 0 23 Dec 2021
FedLGA: Towards System-Heterogeneity of Federated Learning via Local Gradient Approximation Xingyu Li Zhe Qu Bo Tang Zhuo Lu FedML 33 25 0 22 Dec 2021