v1v2v3 (latest)

Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training

5 December 2017

Song Han

ArXiv (abs)PDF HTML Github (222★)

Papers citing "Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training"

50 / 625 papers shown

Title
Auto-Precision Scaling for Distributed Deep Learning Ruobing Han J. Demmel Yang You 43 5 0 20 Nov 2019
Understanding Top-k Sparsification in Distributed Deep Learning Shaoshuai Shi Xiaowen Chu Ka Chun Cheung Simon See 233 101 0 20 Nov 2019
Layer-wise Adaptive Gradient Sparsification for Distributed Deep Learning with Convergence Guarantees Shaoshuai Shi Zhenheng Tang Qiang-qiang Wang Kaiyong Zhao Xiaowen Chu 65 22 0 20 Nov 2019
Distributed Machine Learning through Heterogeneous Edge Systems Han Hu Dan Wang Chuan Wu 71 43 0 16 Nov 2019
Hyper-Sphere Quantization: Communication-Efficient SGD for Federated Learning XINYAN DAI Xiao Yan Kaiwen Zhou Han Yang K. K. Ng James Cheng Yu Fan FedML 68 47 0 12 Nov 2019
An Overview of Data-Importance Aware Radio Resource Management for Edge Machine Learning Dingzhu Wen Xiaoyang Li Qunsong Zeng Jinke Ren Kaibin Huang 68 25 0 10 Nov 2019
A Crowdsourcing Framework for On-Device Federated Learning Shashi Raj Pandey N. H. Tran M. Bennis Y. Tun Aunas Manzoor Choong Seon Hong FedML 73 253 0 04 Nov 2019
On-Device Machine Learning: An Algorithms and Learning Theory Perspective Sauptik Dhar Junyao Guo Jiayi Liu S. Tripathi Unmesh Kurup Mohak Shah 129 144 0 02 Nov 2019
Progressive Compressed Records: Taking a Byte out of Deep Learning Data Michael Kuchnik George Amvrosiadis Virginia Smith 86 9 0 01 Nov 2019
Robust Federated Learning with Noisy Communication F. Ang Li Chen Senior Member Ieee Nan Zhao Senior Member Ieee Yunfei Chen Weidong Wang Feng Yu FedML 58 117 0 01 Nov 2019
SPARQ-SGD: Event-Triggered and Compressed Communication in Decentralized Stochastic Optimization Navjot Singh Deepesh Data Jemin George Suhas Diggavi 88 23 0 31 Oct 2019
Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronization Farzin Haddadpour Mohammad Mahdi Kamani M. Mahdavi V. Cadambe FedML 87 202 0 30 Oct 2019
E2-Train: Training State-of-the-art CNNs with Over 80% Energy Savings Yue Wang Ziyu Jiang Xiaohan Chen Pengfei Xu Yang Zhao Yingyan Lin Zhangyang Wang MQ 114 83 0 29 Oct 2019
Shielding Collaborative Learning: Mitigating Poisoning Attacks through Client-Side Detection Lingchen Zhao Shengshan Hu Qian Wang Jianlin Jiang Chao Shen Xiangyang Luo Pengfei Hu AAML 72 96 0 29 Oct 2019
Gradient Sparification for Asynchronous Distributed Training Zijie Yan FedML 26 1 0 24 Oct 2019
Communication-Efficient Local Decentralized SGD Methods Xiang Li Wenhao Yang Shusen Wang Zhihua Zhang 97 53 0 21 Oct 2019
Machine Learning Systems for Highly-Distributed and Rapidly-Growing Data Kevin Hsieh SyDa OOD 45 4 0 18 Oct 2019
Election Coding for Distributed Learning: Protecting SignSGD against Byzantine Attacks Jy-yong Sohn Dong-Jun Han Beongjun Choi Jaekyun Moon FedML 98 37 0 14 Oct 2019
Eavesdrop the Composition Proportion of Training Labels in Federated Learning Lixu Wang Shichao Xu Tianlin Li Qi Zhu FedML 83 63 0 14 Oct 2019
JSDoop and TensorFlow.js: Volunteer Distributed Web Browser-Based Neural Network Training José Á. Morell Andrés Camero Enrique Alba 58 9 0 12 Oct 2019
Straggler-Agnostic and Communication-Efficient Distributed Primal-Dual Algorithm for High-Dimensional Data Mining Zhouyuan Huo Heng-Chiao Huang FedML 47 5 0 09 Oct 2019
Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints Felix Sattler K. Müller Wojciech Samek FedML 200 1,015 0 04 Oct 2019
Minimax Bounds for Distributed Logistic Regression Florent Chiaroni N. Hueber FedML 32 5 0 03 Oct 2019
The Non-IID Data Quagmire of Decentralized Machine Learning Kevin Hsieh Amar Phanishayee O. Mutlu Phillip B. Gibbons 192 575 0 01 Oct 2019
Model Pruning Enables Efficient Federated Learning on Edge Devices Yuang Jiang Shiqiang Wang Victor Valls Bongjun Ko Wei-Han Lee Kin K. Leung Leandros Tassiulas 160 470 0 26 Sep 2019
Communication-Efficient Distributed Learning via Lazily Aggregated Quantized Gradients Jun Sun Tianyi Chen G. Giannakis Zaiyue Yang 88 95 0 17 Sep 2019
Neural Machine Translation with 4-Bit Precision and Beyond Alham Fikri Aji Kenneth Heafield MQ 22 7 0 13 Sep 2019
Accelerating Training using Tensor Decomposition Mostafa Elhoushi Ye Tian Zihao Chen F. Shafiq Joey Yiwei Li 39 3 0 10 Sep 2019
Gradient Descent with Compressed Iterates Ahmed Khaled Peter Richtárik 41 22 0 10 Sep 2019
Hierarchical Federated Learning Across Heterogeneous Cellular Networks Mehdi Salehi Heydar Abad Emre Ozfatura Deniz Gunduz Ozgur Ercetin FedML 143 314 0 05 Sep 2019
Beyond Human-Level Accuracy: Computational Challenges in Deep Learning Joel Hestness Newsha Ardalani G. Diamos 64 68 0 03 Sep 2019
Big Data Analytics for Large Scale Wireless Networks: Challenges and Opportunities Hongning Dai Raymond Chi-Wing Wong Hao Wang Zibin Zheng A. Vasilakos AI4CE GNN 63 65 0 02 Sep 2019
Big Data Analytics for Manufacturing Internet of Things: Opportunities, Challenges and Enabling Technologies Hongning Dai Hao Wang Guangquan Xu J. Wan Muhammad Imran 66 251 0 01 Sep 2019
Recent Advances in Deep Learning for Object Detection Xiongwei Wu Doyen Sahoo Guosheng Lin VLM ObjD 131 824 0 10 Aug 2019
Machine Learning at the Network Edge: A Survey M. G. Sarwar Murshed Chris Murphy Daqing Hou Nazar Khan Ganesh Ananthanarayanan Faraz Hussain 92 392 0 31 Jul 2019
Taming Momentum in a Distributed Asynchronous Environment Ido Hakimi Saar Barkai Moshe Gabel Assaf Schuster 93 23 0 26 Jul 2019
Benchmarking TPU, GPU, and CPU Platforms for Deep Learning Y. Wang Gu-Yeon Wei David Brooks ELM VLM 97 278 0 24 Jul 2019
Federated Learning over Wireless Fading Channels M. Amiri Deniz Gunduz 125 515 0 23 Jul 2019
A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection Yue Liu Zeyi Wen Zhaomin Wu Sixu Hu Naibo Wang Yuan N. Li Xu Liu Bingsheng He FedML 130 1,014 0 23 Jul 2019
Decentralized Deep Learning with Arbitrary Communication Compression Anastasia Koloskova Tao R. Lin Sebastian U. Stich Martin Jaggi FedML 95 235 0 22 Jul 2019
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey Xiaofei Wang Yiwen Han Victor C. M. Leung Dusit Niyato Xueqiang Yan Xu Chen 102 1,002 0 19 Jul 2019
Faster Distributed Deep Net Training: Computation and Communication Decoupled Stochastic Gradient Descent Shuheng Shen Linli Xu Jingchang Liu Xianfeng Liang Yifei Cheng ODL FedML 68 24 0 28 Jun 2019
Database Meets Deep Learning: Challenges and Opportunities Wei Wang Meihui Zhang Gang Chen H. V. Jagadish Beng Chin Ooi K. Tan 89 148 0 21 Jun 2019
Deep Leakage from Gradients Ligeng Zhu Zhijian Liu Song Han FedML 112 2,236 0 21 Jun 2019
Distributed Optimization for Over-Parameterized Learning Chi Zhang Qianxiao Li 80 5 0 14 Jun 2019
Associative Convolutional Layers H. Omidvar Vahideh Akhlaghi M. Franceschetti Rajesh K. Gupta 44 1 0 10 Jun 2019
Making Asynchronous Stochastic Gradient Descent Work for Transformers Alham Fikri Aji Kenneth Heafield 68 13 0 08 Jun 2019
Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computations Debraj Basu Deepesh Data C. Karakuş Suhas Diggavi MQ 74 408 0 06 Jun 2019
PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization Thijs Vogels Sai Praneeth Karimireddy Martin Jaggi 105 322 0 31 May 2019
On the Convergence of Memory-Based Distributed SGD Shen-Yi Zhao Hao Gao Wu-Jun Li 25 1 0 30 May 2019