v1v2v3 (latest)

Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training

5 December 2017

Song Han

ArXiv (abs)PDF HTML Github (222★)

Papers citing "Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training"

50 / 625 papers shown

Title
Joint Privacy Enhancement and Quantization in Federated Learning Natalie Lang Elad Sofer Tomer Shaked Nir Shlezinger FedML 92 48 0 23 Aug 2022
A Fast Blockchain-based Federated Learning Framework with Compressed Communications Laizhong Cui Xiaoxin Su Yipeng Zhou FedML 60 24 0 12 Aug 2022
Scalable neural quantum states architecture for quantum chemistry Tianchen Zhao J. Stokes S. Veerapaneni 103 34 0 11 Aug 2022
Quantized Adaptive Subgradient Algorithms and Their Applications Ke Xu Jianqiao Wangni Yifan Zhang Deheng Ye Jiaxiang Wu P. Zhao 54 0 0 11 Aug 2022
FedOBD: Opportunistic Block Dropout for Efficiently Training Large-scale Neural Networks through Federated Learning Yuanyuan Chen Zichen Chen Pengcheng Wu Han Yu AI4CE 79 19 0 10 Aug 2022
Towards Efficient Communications in Federated Learning: A Contemporary Survey Zihao Zhao Yuzhu Mao Yang Liu Linqi Song Ouyang Ye Xinlei Chen Wenbo Ding FedML 105 64 0 02 Aug 2022
Parameter-Parallel Distributed Variational Quantum Algorithm Yun-Fei Niu Shuo Zhang Chen Ding Wansu Bao Heliang Huang 52 4 0 31 Jul 2022
BiFeat: Supercharge GNN Training via Graph Feature Quantization Yuxin Ma Ping Gong Jun Yi Z. Yao Cheng-rong Li Yuxiong He Feng Yan GNN 46 6 0 29 Jul 2022
CFLIT: Coexisting Federated Learning and Information Transfer Zehong Lin Hang Liu Y. Zhang 43 11 0 26 Jul 2022
Reconciling Security and Communication Efficiency in Federated Learning Karthik Prasad Sayan Ghosh Graham Cormode Ilya Mironov Ashkan Yousefpour Pierre Stock FedML 69 9 0 26 Jul 2022
Federated Graph Machine Learning: A Survey of Concepts, Techniques, and Applications Xingbo Fu Binchi Zhang Yushun Dong Chen Chen Jundong Li FedML OOD AI4CE 113 38 0 24 Jul 2022
Training Transformers Together Alexander Borzunov Max Ryabinin Tim Dettmers Quentin Lhoest Lucile Saulnier Michael Diskin Yacine Jernite Thomas Wolf ViT 63 10 0 07 Jul 2022
Fundamental Limits of Communication Efficiency for Model Aggregation in Distributed Learning: A Rate-Distortion Approach Naifu Zhang M. Tao Jia Wang Fan Xu 49 14 0 28 Jun 2022
Efficient Adaptive Federated Optimization of Federated Learning for IoT Zunming Chen Hongyan Cui Ensen Wu Yu Xi 74 0 0 23 Jun 2022
sqSGD: Locally Private and Communication Efficient Federated Learning Yan Feng Tao Xiong Ruofan Wu Lingjuan Lv Leilei Shi FedML 74 2 0 21 Jun 2022
Compressed-VFL: Communication-Efficient Learning with Vertically Partitioned Data Timothy Castiglia Anirban Das Shiqiang Wang S. Patterson FedML 66 50 0 16 Jun 2022
Matching Pursuit Based Scheduling for Over-the-Air Federated Learning Ali Bereyhi Adela Vagollari S. Asaad R. Muller W. Gerstacker H. Vincent Poor 66 6 0 14 Jun 2022
Communication-Efficient Federated Learning over MIMO Multiple Access Channels Yo-Seb Jeon Mohammad Mohammadi Amiri Namyoon Lee 52 18 0 12 Jun 2022
Communication-Efficient Robust Federated Learning with Noisy Labels Junyi Li Jian Pei Heng Huang FedML 83 18 0 11 Jun 2022
Lower Bounds and Nearly Optimal Algorithms in Distributed Learning with Communication Compression Xinmeng Huang Yiming Chen W. Yin Kun Yuan 92 27 0 08 Jun 2022
Towards Fair Federated Recommendation Learning: Characterizing the Inter-Dependence of System and Data Heterogeneity Kiwan Maeng Haiyu Lu Luca Melis John Nguyen Michael G. Rabbat Carole-Jean Wu FedML 110 32 0 30 May 2022
ByteComp: Revisiting Gradient Compression in Distributed Training Zhuang Wang Yanghua Peng Yibo Zhu T. Ng 66 2 0 28 May 2022
Stochastic Gradient Methods with Compressed Communication for Decentralized Saddle Point Problems Chhavi Sharma Vishnu Narayanan P. Balamurugan 51 2 0 28 May 2022
QUIC-FL: Quick Unbiased Compression for Federated Learning Ran Ben-Basat S. Vargaftik Amit Portnoy Gil Einziger Y. Ben-Itzhak Michael Mitzenmacher FedML 144 13 0 26 May 2022
BlockDFL: A Blockchain-based Fully Decentralized Peer-to-Peer Federated Learning Framework Zhen Qin Xueqiang Yan Mengchu Zhou Shuiguang Deng 102 16 0 21 May 2022
Federated learning: Applications, challenges and future directions Subrato Bharati Hossain Mondal Prajoy Podder V. B. Surya Prasath FedML 78 59 0 18 May 2022
On Distributed Adaptive Optimization with Gradient Compression Xiaoyun Li Belhal Karimi Ping Li 77 27 0 11 May 2022
Federated Random Reshuffling with Compression and Variance Reduction Grigory Malinovsky Peter Richtárik FedML 100 10 0 08 May 2022
Over-the-Air Federated Multi-Task Learning via Model Sparsification and Turbo Compressed Sensing Haoming Ma Xiaojun Yuan Z. Ding Dian Fan Jun Fang 75 1 0 08 May 2022
Training Mixed-Domain Translation Models via Federated Learning Peyman Passban Tanya Roosta Rahul Gupta Ankit R. Chadha Clement Chung FedML AI4CE 94 18 0 03 May 2022
MiCS: Near-linear Scaling for Training Gigantic Model on Public Cloud Zhen Zhang Shuai Zheng Yida Wang Justin Chiu George Karypis Trishul Chilimbi Mu Li Xin Jin 67 39 0 30 Apr 2022
A review of Federated Learning in Intrusion Detection Systems for IoT Aitor Belenguer J. Navaridas J. A. Pascual 90 16 0 26 Apr 2022
Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications Han Cai Ji Lin Chengyue Wu Zhijian Liu Haotian Tang Hanrui Wang Ligeng Zhu Song Han 116 115 0 25 Apr 2022
FedVQCS: Federated Learning via Vector Quantized Compressed Sensing Yong-Nam Oh Yo-Seb Jeon Mingzhe Chen Walid Saad FedML 74 12 0 16 Apr 2022
Adaptive Differential Filters for Fast and Communication-Efficient Federated Learning Daniel Becking H. Kirchhoffer G. Tech Paul Haase Karsten Müller H. Schwarz Wojciech Samek FedML 64 4 0 09 Apr 2022
Accelerating Federated Edge Learning via Topology Optimization Shanfeng Huang Zezhong Zhang Shuai Wang Rui Wang Kaibin Huang FedML 90 13 0 01 Apr 2022
Auditing Privacy Defenses in Federated Learning via Generative Gradient Leakage Zhuohang Li Jiaxin Zhang Lu Liu Jian-Dong Liu FedML 83 120 0 29 Mar 2022
Local Stochastic Factored Gradient Descent for Distributed Quantum State Tomography Junhyung Lyle Kim Taha Toghani César A. Uribe Anastasios Kyrillidis 47 3 0 22 Mar 2022
Similarity-based Label Inference Attack against Training and Inference of Split Learning Junlin Liu Xinchen Lyu Qimei Cui Xiaofeng Tao FedML 88 27 0 10 Mar 2022
Correlated quantization for distributed mean estimation and optimization A. Suresh Ziteng Sun Jae Hun Ro Felix X. Yu 101 12 0 09 Mar 2022
The Fundamental Price of Secure Aggregation in Differentially Private Federated Learning Wei-Ning Chen Christopher A. Choquette-Choo Peter Kairouz A. Suresh FedML 118 65 0 07 Mar 2022
Differentially Private Federated Learning with Local Regularization and Sparsification Anda Cheng Peisong Wang Xi Sheryl Zhang Jian Cheng FedML 73 77 0 07 Mar 2022
Linear Stochastic Bandits over a Bit-Constrained Channel A. Mitra Hamed Hassani George J. Pappas 88 8 0 02 Mar 2022
DropIT: Dropping Intermediate Tensors for Memory-Efficient DNN Training Joya Chen Kai Xu Yuhui Wang Yifei Cheng Angela Yao 52 8 0 28 Feb 2022
No One Left Behind: Inclusive Federated Learning over Heterogeneous Devices Ruixuan Liu Fangzhao Wu Chuhan Wu Yanlin Wang Lingjuan Lyu Hong Chen Xing Xie FedML 89 72 0 16 Feb 2022
OLIVE: Oblivious Federated Learning on Trusted Execution Environment against the risk of sparsification Fumiyuki Kato Yang Cao Masatoshi Yoshikawa FedML 72 5 0 15 Feb 2022
BEAS: Blockchain Enabled Asynchronous & Secure Federated Machine Learning A. Mondal Harpreet Virk Debayan Gupta 81 16 0 06 Feb 2022
Distributed Learning With Sparsified Gradient Differences Yicheng Chen Rick S. Blum Martin Takáč Brian M. Sadler 77 15 0 05 Feb 2022
Improved Information Theoretic Generalization Bounds for Distributed and Federated Learning L. P. Barnes Alex Dytso H. V. Poor FedML 57 19 0 04 Feb 2022
3PC: Three Point Compressors for Communication-Efficient Distributed Training and a Better Theory for Lazy Aggregation Peter Richtárik Igor Sokolov Ilyas Fatkhullin Elnur Gasanov Zhize Li Eduard A. Gorbunov 75 32 0 02 Feb 2022