Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training

5 December 2017

Song Han

Papers citing "Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training"

50 / 616 papers shown

Title
Memory-efficient training with streaming dimensionality reduction Siyuan Huang Brian D. Hoskins M. Daniels M. D. Stiles G. Adam 11 3 0 25 Apr 2020
A Review of Privacy-preserving Federated Learning for the Internet-of-Things Christopher Briggs Zhong Fan Péter András 31 15 0 24 Apr 2020
A Framework for Evaluating Gradient Leakage Attacks in Federated Learning Wenqi Wei Ling Liu Margaret Loper Ka-Ho Chow Mehmet Emre Gursoy Stacey Truex Yanzhao Wu FedML 26 147 0 22 Apr 2020
How to Train your DNN: The Network Operator Edition M. Chang D. Bottini Lisa Jian Pranay Kumar Aurojit Panda S. Shenker 11 1 0 21 Apr 2020
Efficient Synthesis of Compact Deep Neural Networks Wenhan Xia Hongxu Yin N. Jha 29 3 0 18 Apr 2020
Detached Error Feedback for Distributed SGD with Random Sparsification An Xu Heng-Chiao Huang 41 9 0 11 Apr 2020
Client Selection and Bandwidth Allocation in Wireless Federated Learning Networks: A Long-Term Perspective Jie Xu Heqiang Wang 22 349 0 09 Apr 2020
Evaluating the Communication Efficiency in Federated Learning Algorithms Muhammad Asad Ahmed Moustafa Takayuki Ito M. Aslam FedML 22 51 0 06 Apr 2020
Reducing Data Motion to Accelerate the Training of Deep Neural Networks Sicong Zhuang Cristiano Malossi Marc Casas 27 0 0 05 Apr 2020
Scheduling for Cellular Federated Edge Learning with Importance and Channel Awareness Jinke Ren Yinghui He Dingzhu Wen Guanding Yu Kaibin Huang Dongning Guo 49 194 0 01 Apr 2020
Edge Intelligence: Architectures, Challenges, and Applications Dianlei Xu Tong Li Yong Li Xiang Su Sasu Tarkoma Tao Jiang Jon Crowcroft Pan Hui 53 29 0 26 Mar 2020
FedSel: Federated SGD under Local Differential Privacy with Top-k Dimension Selection Ruixuan Liu Yang Cao Masatoshi Yoshikawa Hong Chen FedML 14 106 0 24 Mar 2020
A Compressive Sensing Approach for Federated Learning over Massive MIMO Communication Systems Yo-Seb Jeon M. Amiri Jun Li H. Vincent Poor 30 9 0 18 Mar 2020
Communication-Efficient Distributed Deep Learning: A Comprehensive Survey Zhenheng Tang Shaoshuai Shi Wei Wang Bo Li Xuming Hu 31 48 0 10 Mar 2020
Trends and Advancements in Deep Neural Network Communication Felix Sattler Thomas Wiegand Wojciech Samek GNN 33 9 0 06 Mar 2020
Decentralized SGD with Over-the-Air Computation Emre Ozfatura Stefano Rini Deniz Gunduz 28 38 0 06 Mar 2020
Communication optimization strategies for distributed deep neural network training: A survey Shuo Ouyang Dezun Dong Yemao Xu Liquan Xiao 30 12 0 06 Mar 2020
Distributed Momentum for Byzantine-resilient Learning El-Mahdi El-Mhamdi R. Guerraoui Sébastien Rouault FedML 22 22 0 28 Feb 2020
On Biased Compression for Distributed Learning Aleksandr Beznosikov Samuel Horváth Peter Richtárik M. Safaryan 10 186 0 27 Feb 2020
An On-Device Federated Learning Approach for Cooperative Model Update between Edge Devices Rei Ito Mineto Tsukada Hiroki Matsutani FedML 26 7 0 27 Feb 2020
LASG: Lazily Aggregated Stochastic Gradients for Communication-Efficient Distributed Learning Tianyi Chen Yuejiao Sun W. Yin FedML 22 14 0 26 Feb 2020
Optimal Gradient Quantization Condition for Communication-Efficient Distributed Training An Xu Zhouyuan Huo Heng-Chiao Huang MQ 16 6 0 25 Feb 2020
Stochastic-Sign SGD for Federated Learning with Theoretical Guarantees Richeng Jin Yufan Huang Xiaofan He H. Dai Tianfu Wu FedML 27 62 0 25 Feb 2020
Communication Contention Aware Scheduling of Multiple Deep Learning Training Jobs Qiang-qiang Wang Shaoshuai Shi Canhui Wang Xiaowen Chu 29 13 0 24 Feb 2020
Communication-Efficient Decentralized Learning with Sparsification and Adaptive Peer Selection Zhenheng Tang Shaoshuai Shi Xiaowen Chu FedML 21 57 0 22 Feb 2020
Communication-Efficient Edge AI: Algorithms and Systems Yuanming Shi Kai Yang Tao Jiang Jun Zhang Khaled B. Letaief GNN 31 327 0 22 Feb 2020
Uncertainty Principle for Communication Compression in Distributed and Federated Learning and the Search for an Optimal Compressor M. Safaryan Egor Shulgin Peter Richtárik 32 61 0 20 Feb 2020
MDLdroid: a ChainSGD-reduce Approach to Mobile Deep Learning for Personal Mobile Sensing Yu Zhang Tao Gu Xi Zhang FedML 22 21 0 07 Feb 2020
Differentially Quantized Gradient Methods Chung-Yi Lin V. Kostina B. Hassibi MQ 30 7 0 06 Feb 2020
Communication Efficient Federated Learning over Multiple Access Channels Wei-Ting Chang Ravi Tandon FedML 23 44 0 23 Jan 2020
Intermittent Pulling with Local Compensation for Communication-Efficient Federated Learning Yining Qi Zhihao Qu Song Guo Xin Gao Ruixuan Li Baoliu Ye FedML 18 8 0 22 Jan 2020
A Federated Deep Learning Framework for Privacy Preservation and Communication Efficiency Tien-Dung Cao Tram Truong-Huu H. Tran K. Tran FedML 17 27 0 22 Jan 2020
Elastic Consistency: A General Consistency Model for Distributed Stochastic Gradient Descent Giorgi Nadiradze Ilia Markov Bapi Chatterjee Vyacheslav Kungurtsev Dan Alistarh FedML 22 14 0 16 Jan 2020
One-Bit Over-the-Air Aggregation for Communication-Efficient Federated Edge Learning: Design and Convergence Analysis Guangxu Zhu Yuqing Du Deniz Gunduz Kaibin Huang 44 308 0 16 Jan 2020
Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning Approach Pengchao Han Shiqiang Wang K. Leung FedML 35 175 0 14 Jan 2020
Sparse Weight Activation Training Md Aamir Raihan Tor M. Aamodt 34 73 0 07 Jan 2020
Think Locally, Act Globally: Federated Learning with Local and Global Representations Paul Pu Liang Terrance Liu Liu Ziyin Nicholas B. Allen Randy P. Auerbach David Brent Ruslan Salakhutdinov Louis-Philippe Morency FedML 32 552 0 06 Jan 2020
Variance Reduced Local SGD with Lower Communication Complexity Xian-Feng Liang Shuheng Shen Jingchang Liu Zhen Pan Enhong Chen Yifei Cheng FedML 44 152 0 30 Dec 2019
MG-WFBP: Merging Gradients Wisely for Efficient Communication in Distributed Deep Learning Shaoshuai Shi Xiaowen Chu Bo Li FedML 28 25 0 18 Dec 2019
Advances and Open Problems in Federated Learning Peter Kairouz H. B. McMahan Brendan Avent A. Bellet M. Bennis ... Zheng Xu Qiang Yang Felix X. Yu Han Yu Sen Zhao FedML AI4CE 81 6,103 0 10 Dec 2019
Privacy-Preserving Blockchain Based Federated Learning with Differential Data Sharing Anudit Nagar 16 21 0 10 Dec 2019
Auto-Precision Scaling for Distributed Deep Learning Ruobing Han J. Demmel Yang You 21 5 0 20 Nov 2019
Understanding Top-k Sparsification in Distributed Deep Learning Shaoshuai Shi Xiaowen Chu Ka Chun Cheung Simon See 30 95 0 20 Nov 2019
Layer-wise Adaptive Gradient Sparsification for Distributed Deep Learning with Convergence Guarantees Shaoshuai Shi Zhenheng Tang Qiang-qiang Wang Kaiyong Zhao Xiaowen Chu 19 22 0 20 Nov 2019
Distributed Machine Learning through Heterogeneous Edge Systems Han Hu Dan Wang Chuan Wu 33 43 0 16 Nov 2019
Hyper-Sphere Quantization: Communication-Efficient SGD for Federated Learning XINYAN DAI Xiao Yan Kaiwen Zhou Han Yang K. K. Ng James Cheng Yu Fan FedML 27 47 0 12 Nov 2019
An Overview of Data-Importance Aware Radio Resource Management for Edge Machine Learning Dingzhu Wen Xiaoyang Li Qunsong Zeng Jinke Ren Kaibin Huang 24 24 0 10 Nov 2019
A Crowdsourcing Framework for On-Device Federated Learning Shashi Raj Pandey N. H. Tran M. Bennis Y. Tun Aunas Manzoor Choong Seon Hong FedML 22 250 0 04 Nov 2019
On-Device Machine Learning: An Algorithms and Learning Theory Perspective Sauptik Dhar Junyao Guo Jiayi Liu S. Tripathi Unmesh Kurup Mohak Shah 33 141 0 02 Nov 2019
Progressive Compressed Records: Taking a Byte out of Deep Learning Data Michael Kuchnik George Amvrosiadis Virginia Smith 22 9 0 01 Nov 2019