v1v2v3 (latest)

Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training

5 December 2017

Song Han

ArXiv (abs)PDF HTML Github (222★)

Papers citing "Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training"

50 / 625 papers shown

Title
A Distributed SGD Algorithm with Global Sketching for Deep Learning Training Acceleration Lingfei Dai Boyu Diao Chao Li Yongjun Xu 68 5 0 13 Aug 2021
Communication Optimization in Large Scale Federated Learning using Autoencoder Compressed Weight Updates Srikanth Chandar Pravin Chandran Raghavendra Bhat Avinash Chakravarthi AI4CE 70 3 0 12 Aug 2021
PRECODE - A Generic Model Extension to Prevent Deep Gradient Leakage Daniel Scheliga Patrick Mäder M. Seeland MIACV 45 37 0 10 Aug 2021
Understanding Human Innate Immune System Dependencies using Graph Neural Networks Shagufta Henna GNN AI4CE 57 0 0 05 Aug 2021
Learning a Neural Diff for Speech Models J. Macoskey Grant P. Strimel Ariya Rastrow 59 2 0 03 Aug 2021
Rethinking gradient sparsification as total error minimization Atal Narayan Sahu Aritra Dutta A. Abdelmoniem Trambak Banerjee Marco Canini Panos Kalnis 97 58 0 02 Aug 2021
Communication-Efficient Federated Learning via Predictive Coding Kai Yue Richeng Jin Chau-Wai Wong H. Dai FedML 78 14 0 02 Aug 2021
FedLab: A Flexible Federated Learning Framework Dun Zeng Siqi Liang Xiangjing Hu Hui Wang Zenglin Xu FedML 80 119 0 24 Jul 2021
RingFed: Reducing Communication Costs in Federated Learning on Non-IID Data Guang Yang Ke Mu Chunhe Song Zhijia Yang Tierui Gong FedML 33 16 0 19 Jul 2021
An Experimental Study of Data Heterogeneity in Federated Learning Methods for Medical Imaging Liangqiong Qu N. Balachandar D. Rubin 66 24 0 18 Jul 2021
A Field Guide to Federated Optimization Jianyu Wang Zachary B. Charles Zheng Xu Gauri Joshi H. B. McMahan ... Mi Zhang Tong Zhang Chunxiang Zheng Chen Zhu Wennan Zhu FedML 279 422 0 14 Jul 2021
Fine-Grained AutoAugmentation for Multi-Label Classification Y. Wang Hesen Chen Fangyi Zhang Yaohua Wang Xiuyu Sun Ming Lin Hao Li 97 2 0 12 Jul 2021
Unity Perception: Generate Synthetic Data for Computer Vision S. Borkman A. Crespi S. Dhakad Sujoy Ganguly Jonathan Hogins ... Cesar Romero Wesley Smith Alex Thaman Samuel Warren Nupur Yadav 3DV SyDa VLM 81 102 0 09 Jul 2021
KAISA: An Adaptive Second-Order Optimizer Framework for Deep Neural Networks J. G. Pauloski Qi Huang Lei Huang Shivaram Venkataraman Kyle Chard Ian Foster Zhao-jie Zhang 86 29 0 04 Jul 2021
Gradient-Leakage Resilient Federated Learning Wenqi Wei Ling Liu Yanzhao Wu Gong Su Arun Iyengar FedML 72 91 0 02 Jul 2021
Over-the-Air Federated Multi-Task Learning Haiyan Ma Xiaojun Yuan Dian Fan Z. Ding Xin Wang Jun Fang 25 4 0 27 Jun 2021
AdaptCL: Efficient Collaborative Learning with Dynamic and Adaptive Pruning Guangmeng Zhou Ke Xu Qi Li Yang Liu Yi Zhao 54 8 0 27 Jun 2021
Handling Data Heterogeneity with Generative Replay in Collaborative Learning for Medical Imaging Liangqiong Qu N. Balachandar Miao Zhang D. Rubin MedIm 89 23 0 24 Jun 2021
Secure Distributed Training at Scale Eduard A. Gorbunov Alexander Borzunov Michael Diskin Max Ryabinin FedML 90 15 0 21 Jun 2021
CD-SGD: Distributed Stochastic Gradient Descent with Compression and Delay Compensation Enda Yu Dezun Dong Yemao Xu Shuo Ouyang Xiangke Liao 49 5 0 21 Jun 2021
Fine-Grained Data Selection for Improved Energy Efficiency of Federated Edge Learning A. Albaseer M. Abdallah Ala I. Al-Fuqaha A. Erbad 76 35 0 20 Jun 2021
Distributed Deep Learning in Open Collaborations Michael Diskin Alexey Bukhtiyarov Max Ryabinin Lucile Saulnier Quentin Lhoest ... Denis Mazur Ilia Kobelev Yacine Jernite Thomas Wolf Gennady Pekhimenko FedML 129 59 0 18 Jun 2021
Federated Learning for Internet of Things: A Federated Learning Framework for On-device Anomaly Data Detection Tuo Zhang Chaoyang He Tian-Shya Ma Lei Gao Mark Ma Salman Avestimehr FedML 99 119 0 15 Jun 2021
SynthASR: Unlocking Synthetic Data for Speech Recognition A. Fazel Wei Yang Yulan Liu Roberto Barra-Chicote Yi Meng Roland Maas J. Droppo SyDa 110 51 0 14 Jun 2021
Compressed Gradient Tracking for Decentralized Optimization Over General Directed Networks Zhuoqing Song Lei Shi Shi Pu Ming Yan 95 25 0 14 Jun 2021
CFedAvg: Achieving Efficient Communication and Fast Convergence in Non-IID Federated Learning Haibo Yang Jia Liu Elizabeth S. Bentley FedML 40 20 0 14 Jun 2021
Federated Learning Over Wireless Channels: Dynamic Resource Allocation and Task Scheduling Shunfeng Chu Jun Li Jianxin Wang Zhe Wang Ming Ding Yijin Zang Yuwen Qian Wen Chen 72 16 0 13 Jun 2021
Communication-efficient SGD: From Local SGD to One-Shot Averaging Artin Spiridonoff Alexander Olshevsky I. Paschalidis FedML 110 20 0 09 Jun 2021
Theoretically Better and Numerically Faster Distributed Optimization with Smoothness-Aware Quantization Techniques Bokun Wang M. Safaryan Peter Richtárik MQ 37 10 0 07 Jun 2021
Neural Distributed Source Coding Jay Whang Alliot Nagle Anish Acharya Hyeji Kim A. Dimakis 85 21 0 05 Jun 2021
QLSD: Quantised Langevin stochastic dynamics for Bayesian federated learning Maxime Vono Vincent Plassier Alain Durmus Hadrien Hendrikx Eric Moulines FedML 93 36 0 01 Jun 2021
H-FL: A Hierarchical Communication-Efficient and Privacy-Protected Architecture for Federated Learning He Yang 70 28 0 01 Jun 2021
Fast Federated Learning by Balancing Communication Trade-Offs Milad Khademi Nori Sangseok Yun Il-Min Kim FedML 77 57 0 23 May 2021
User-Level Label Leakage from Gradients in Federated Learning A. Wainakh Fabrizio G. Ventola Till Müßig Jens Keim Carlos Garcia Cordero Ephraim Zimmer Tim Grube Kristian Kersting M. Mühlhäuser FedML AAML 72 55 0 19 May 2021
Compressed Communication for Distributed Training: Adaptive Methods and System Yuchen Zhong Cong Xie Shuai Zheng Yanghua Peng 74 9 0 17 May 2021
LocalNewton: Reducing Communication Bottleneck for Distributed Learning Vipul Gupta Avishek Ghosh Michal Derezinski Rajiv Khanna Kannan Ramchandran Michael W. Mahoney 77 13 0 16 May 2021
Node Selection Toward Faster Convergence for Federated Learning on Non-IID Data Hongda Wu Ping Wang FedML 98 143 0 14 May 2021
Slashing Communication Traffic in Federated Learning by Transmitting Clustered Model Updates Laizhong Cui Xiaoxin Su Yipeng Zhou Yi Pan FedML 75 37 0 10 May 2021
OCTOPUS: Overcoming Performance andPrivatization Bottlenecks in Distributed Learning Shuo Wang Surya Nepal Kristen Moore M. Grobler Carsten Rudolph A. Abuadbba FedML 67 8 0 03 May 2021
Personalized Federated Learning by Structured and Unstructured Pruning under Data Heterogeneity Saeed Vahidian Mahdi Morafah Bill Lin 115 61 0 02 May 2021
GRNN: Generative Regression Neural Network -- A Data Leakage Attack for Federated Learning Hanchi Ren Jingjing Deng Xianghua Xie SILM AAML FedML 109 103 0 02 May 2021
ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training Jianfei Chen Lianmin Zheng Z. Yao Dequan Wang Ion Stoica Michael W. Mahoney Joseph E. Gonzalez MQ 77 75 0 29 Apr 2021
NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization Ali Ramezani-Kebrya Fartash Faghri Ilya Markov V. Aksenov Dan Alistarh Daniel M. Roy MQ 111 33 0 28 Apr 2021
Partitioning sparse deep neural networks for scalable training and inference G. Demirci Hakan Ferhatosmanoglu 35 11 0 23 Apr 2021
ScaleCom: Scalable Sparsified Gradient Compression for Communication-Efficient Distributed Training Chia-Yu Chen Jiamin Ni Songtao Lu Xiaodong Cui Pin-Yu Chen ... Naigang Wang Swagath Venkataramani Vijayalakshmi Srinivasan Wei Zhang K. Gopalakrishnan 79 67 0 21 Apr 2021
Communication Efficient Federated Learning with Adaptive Quantization Yuzhu Mao Zihao Zhao Guangfeng Yan Yang Liu Tian-Shing Lan Linqi Song Wenbo Ding MQ 43 10 0 13 Apr 2021
Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models Dheevatsa Mudigere Y. Hao Jianyu Huang Zhihao Jia Andrew Tulloch ... Ajit Mathews Lin Qiao M. Smelyanskiy Bill Jia Vijay Rao 113 155 0 12 Apr 2021
Federated Learning: A Signal Processing Perspective Tomer Gafni Nir Shlezinger Kobi Cohen Yonina C. Eldar H. Vincent Poor FedML 102 130 0 31 Mar 2021
Threshold-Based Data Exclusion Approach for Energy-Efficient Federated Edge Learning A. Albaseer M. Abdallah Ala I. Al-Fuqaha A. Erbad 74 12 0 30 Mar 2021
1-Bit Compressive Sensing for Efficient Federated Learning Over the Air Xin-Yue Fan Yue Wang Yan Huo Z. Tian FedML 73 34 0 30 Mar 2021