v1v2v3 (latest)

Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training

5 December 2017

Song Han

ArXiv (abs)PDF HTML Github (222★)

Papers citing "Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training"

50 / 625 papers shown

Title
Daydream: Accurately Estimating the Efficacy of Optimizations for DNN Training Hongyu Zhu Amar Phanishayee Gennady Pekhimenko 145 50 0 05 Jun 2020
UVeQFed: Universal Vector Quantization for Federated Learning Nir Shlezinger Mingzhe Chen Yonina C. Eldar H. Vincent Poor Shuguang Cui FedML MQ 65 231 0 05 Jun 2020
Federated Learning for 6G Communications: Challenges, Methods, and Future Directions Yi Liu Lizhen Qu Zehui Xiong Jiawen Kang Xiaofei Wang Dusit Niyato FedML AI4CE 70 283 0 04 Jun 2020
Local SGD With a Communication Overhead Depending Only on the Number of Workers Artin Spiridonoff Alexander Olshevsky I. Paschalidis FedML 61 19 0 03 Jun 2020
DaSGD: Squeezing SGD Parallelization Performance in Distributed Training Using Delayed Averaging Q. Zhou Yawen Zhang Pengcheng Li Xiaoyong Liu Jun Yang Runsheng Wang Ru Huang FedML 45 2 0 31 May 2020
rTop-k: A Statistical Estimation Approach to Distributed SGD L. P. Barnes Huseyin A. Inan Berivan Isik Ayfer Özgür 75 65 0 21 May 2020
OD-SGD: One-step Delay Stochastic Gradient Descent for Distributed Training Yemao Xu Dezun Dong Weixia Xu Xiangke Liao 47 7 0 14 May 2020
SQuARM-SGD: Communication-Efficient Momentum SGD for Decentralized Optimization Navjot Singh Deepesh Data Jemin George Suhas Diggavi 91 55 0 13 May 2020
Breaking (Global) Barriers in Parallel Stochastic Optimization with Wait-Avoiding Group Averaging Shigang Li Tal Ben-Nun Giorgi Nadiradze Salvatore Di Girolamo Nikoli Dryden Dan Alistarh Torsten Hoefler 75 15 0 30 Apr 2020
Memory-efficient training with streaming dimensionality reduction Siyuan Huang Brian D. Hoskins M. Daniels M. D. Stiles G. Adam 37 3 0 25 Apr 2020
A Review of Privacy-preserving Federated Learning for the Internet-of-Things Christopher Briggs Zhong Fan Péter András 133 15 0 24 Apr 2020
A Framework for Evaluating Gradient Leakage Attacks in Federated Learning Wenqi Wei Ling Liu Margaret Loper Ka-Ho Chow Mehmet Emre Gursoy Stacey Truex Yanzhao Wu FedML 111 150 0 22 Apr 2020
How to Train your DNN: The Network Operator Edition M. Chang D. Bottini Lisa Jian Pranay Kumar Aurojit Panda S. Shenker 21 1 0 21 Apr 2020
Efficient Synthesis of Compact Deep Neural Networks Wenhan Xia Hongxu Yin N. Jha 62 3 0 18 Apr 2020
Detached Error Feedback for Distributed SGD with Random Sparsification An Xu Heng-Chiao Huang 71 9 0 11 Apr 2020
Client Selection and Bandwidth Allocation in Wireless Federated Learning Networks: A Long-Term Perspective Jie Xu Heqiang Wang 52 362 0 09 Apr 2020
Evaluating the Communication Efficiency in Federated Learning Algorithms Muhammad Asad Ahmed Moustafa Takayuki Ito M. Aslam FedML 65 55 0 06 Apr 2020
Reducing Data Motion to Accelerate the Training of Deep Neural Networks Sicong Zhuang Cristiano Malossi Marc Casas 34 0 0 05 Apr 2020
Scheduling for Cellular Federated Edge Learning with Importance and Channel Awareness Jinke Ren Yinghui He Dingzhu Wen Guanding Yu Kaibin Huang Dongning Guo 108 197 0 01 Apr 2020
Edge Intelligence: Architectures, Challenges, and Applications Dianlei Xu Tong Li Yong Li Xiang Su Sasu Tarkoma Tao Jiang Jon Crowcroft Pan Hui 116 29 0 26 Mar 2020
FedSel: Federated SGD under Local Differential Privacy with Top-k Dimension Selection Ruixuan Liu Yang Cao Masatoshi Yoshikawa Hong Chen FedML 67 108 0 24 Mar 2020
A Compressive Sensing Approach for Federated Learning over Massive MIMO Communication Systems Yo-Seb Jeon M. Amiri Jun Li H. Vincent Poor 70 9 0 18 Mar 2020
Communication-Efficient Distributed Deep Learning: A Comprehensive Survey Zhenheng Tang Shaoshuai Shi Wei Wang Yue Liu Xiaowen Chu 83 49 0 10 Mar 2020
Trends and Advancements in Deep Neural Network Communication Felix Sattler Thomas Wiegand Wojciech Samek GNN 72 9 0 06 Mar 2020
Decentralized SGD with Over-the-Air Computation Emre Ozfatura Stefano Rini Deniz Gunduz 70 38 0 06 Mar 2020
Communication optimization strategies for distributed deep neural network training: A survey Shuo Ouyang Dezun Dong Yemao Xu Liquan Xiao 123 12 0 06 Mar 2020
Distributed Momentum for Byzantine-resilient Learning El-Mahdi El-Mhamdi R. Guerraoui Sébastien Rouault FedML 58 22 0 28 Feb 2020
On Biased Compression for Distributed Learning Aleksandr Beznosikov Samuel Horváth Peter Richtárik M. Safaryan 81 189 0 27 Feb 2020
An On-Device Federated Learning Approach for Cooperative Model Update between Edge Devices Rei Ito Mineto Tsukada Hiroki Matsutani FedML 61 7 0 27 Feb 2020
LASG: Lazily Aggregated Stochastic Gradients for Communication-Efficient Distributed Learning Tianyi Chen Yuejiao Sun W. Yin FedML 47 14 0 26 Feb 2020
Optimal Gradient Quantization Condition for Communication-Efficient Distributed Training An Xu Zhouyuan Huo Heng-Chiao Huang MQ 40 6 0 25 Feb 2020
Stochastic-Sign SGD for Federated Learning with Theoretical Guarantees Richeng Jin Yufan Huang Xiaofan He H. Dai Tianfu Wu FedML 94 64 0 25 Feb 2020
Communication Contention Aware Scheduling of Multiple Deep Learning Training Jobs Qiang-qiang Wang Shaoshuai Shi Canhui Wang Xiaowen Chu 70 13 0 24 Feb 2020
Communication-Efficient Decentralized Learning with Sparsification and Adaptive Peer Selection Zhenheng Tang Shaoshuai Shi Xiaowen Chu FedML 62 58 0 22 Feb 2020
Communication-Efficient Edge AI: Algorithms and Systems Yuanming Shi Kai Yang Tao Jiang Jun Zhang Khaled B. Letaief GNN 99 334 0 22 Feb 2020
Uncertainty Principle for Communication Compression in Distributed and Federated Learning and the Search for an Optimal Compressor M. Safaryan Egor Shulgin Peter Richtárik 110 61 0 20 Feb 2020
MDLdroid: a ChainSGD-reduce Approach to Mobile Deep Learning for Personal Mobile Sensing Yu Zhang Tao Gu Xi Zhang FedML 56 21 0 07 Feb 2020
Differentially Quantized Gradient Methods Chung-Yi Lin V. Kostina B. Hassibi MQ 66 8 0 06 Feb 2020
Communication Efficient Federated Learning over Multiple Access Channels Wei-Ting Chang Ravi Tandon FedML 84 44 0 23 Jan 2020
Intermittent Pulling with Local Compensation for Communication-Efficient Federated Learning Yining Qi Zhihao Qu Song Guo Xin Gao Ruixuan Li Baoliu Ye FedML 45 9 0 22 Jan 2020
A Federated Deep Learning Framework for Privacy Preservation and Communication Efficiency Tien-Dung Cao Tram Truong-Huu H. Tran K. Tran FedML 34 28 0 22 Jan 2020
Elastic Consistency: A General Consistency Model for Distributed Stochastic Gradient Descent Giorgi Nadiradze Ilia Markov Bapi Chatterjee Vyacheslav Kungurtsev Dan Alistarh FedML 121 14 0 16 Jan 2020
One-Bit Over-the-Air Aggregation for Communication-Efficient Federated Edge Learning: Design and Convergence Analysis Guangxu Zhu Yuqing Du Deniz Gunduz Kaibin Huang 107 317 0 16 Jan 2020
Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning Approach Pengchao Han Shiqiang Wang K. Leung FedML 86 182 0 14 Jan 2020
Sparse Weight Activation Training Md Aamir Raihan Tor M. Aamodt 142 73 0 07 Jan 2020
Think Locally, Act Globally: Federated Learning with Local and Global Representations Paul Pu Liang Terrance Liu Liu Ziyin Nicholas B. Allen Randy P. Auerbach David Brent Ruslan Salakhutdinov Louis-Philippe Morency FedML 124 569 0 06 Jan 2020
Variance Reduced Local SGD with Lower Communication Complexity Xian-Feng Liang Shuheng Shen Jingchang Liu Zhen Pan Enhong Chen Yifei Cheng FedML 95 154 0 30 Dec 2019
MG-WFBP: Merging Gradients Wisely for Efficient Communication in Distributed Deep Learning Shaoshuai Shi Xiaowen Chu Bo Li FedML 61 25 0 18 Dec 2019
Advances and Open Problems in Federated Learning Peter Kairouz H. B. McMahan Brendan Avent A. Bellet M. Bennis ... Zheng Xu Qiang Yang Felix X. Yu Han Yu Sen Zhao FedML AI4CE 296 6,335 0 10 Dec 2019
Privacy-Preserving Blockchain Based Federated Learning with Differential Data Sharing Anudit Nagar 51 21 0 10 Dec 2019