Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training

5 December 2017

Song Han

Papers citing "Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training"

50 / 616 papers shown

Title
An Experimental Study of Data Heterogeneity in Federated Learning Methods for Medical Imaging Liangqiong Qu N. Balachandar D. Rubin 34 24 0 18 Jul 2021
A Field Guide to Federated Optimization Jianyu Wang Zachary B. Charles Zheng Xu Gauri Joshi H. B. McMahan ... Mi Zhang Tong Zhang Chunxiang Zheng Chen Zhu Wennan Zhu FedML 187 412 0 14 Jul 2021
Fine-Grained AutoAugmentation for Multi-Label Classification Y. Wang Hesen Chen Fangyi Zhang Yaohua Wang Xiuyu Sun Ming Lin Hao Li 37 2 0 12 Jul 2021
Unity Perception: Generate Synthetic Data for Computer Vision S. Borkman A. Crespi S. Dhakad Sujoy Ganguly Jonathan Hogins ... Cesar Romero Wesley Smith Alex Thaman Samuel Warren Nupur Yadav 3DV SyDa VLM 16 99 0 09 Jul 2021
KAISA: An Adaptive Second-Order Optimizer Framework for Deep Neural Networks J. G. Pauloski Qi Huang Lei Huang Shivaram Venkataraman Kyle Chard Ian Foster Zhao-jie Zhang 14 28 0 04 Jul 2021
Gradient-Leakage Resilient Federated Learning Wenqi Wei Ling Liu Yanzhao Wu Gong Su Arun Iyengar FedML 19 81 0 02 Jul 2021
Over-the-Air Federated Multi-Task Learning Haiyan Ma Xiaojun Yuan Dian Fan Z. Ding Xin Wang Jun Fang 11 4 0 27 Jun 2021
AdaptCL: Efficient Collaborative Learning with Dynamic and Adaptive Pruning Guangmeng Zhou Ke Xu Qi Li Yang Liu Yi Zhao 24 8 0 27 Jun 2021
Handling Data Heterogeneity with Generative Replay in Collaborative Learning for Medical Imaging Liangqiong Qu N. Balachandar Miao Zhang D. Rubin MedIm 22 19 0 24 Jun 2021
Secure Distributed Training at Scale Eduard A. Gorbunov Alexander Borzunov Michael Diskin Max Ryabinin FedML 26 15 0 21 Jun 2021
CD-SGD: Distributed Stochastic Gradient Descent with Compression and Delay Compensation Enda Yu Dezun Dong Yemao Xu Shuo Ouyang Xiangke Liao 16 5 0 21 Jun 2021
Fine-Grained Data Selection for Improved Energy Efficiency of Federated Edge Learning A. Albaseer M. Abdallah Ala I. Al-Fuqaha A. Erbad 33 35 0 20 Jun 2021
Distributed Deep Learning in Open Collaborations Michael Diskin Alexey Bukhtiyarov Max Ryabinin Lucile Saulnier Quentin Lhoest ... Denis Mazur Ilia Kobelev Yacine Jernite Thomas Wolf Gennady Pekhimenko FedML 41 54 0 18 Jun 2021
Federated Learning for Internet of Things: A Federated Learning Framework for On-device Anomaly Data Detection Tuo Zhang Chaoyang He Tian-Shya Ma Lei Gao Mark Ma Salman Avestimehr FedML 26 112 0 15 Jun 2021
SynthASR: Unlocking Synthetic Data for Speech Recognition A. Fazel Wei Yang Yulan Liu Roberto Barra-Chicote Yi Meng Roland Maas J. Droppo SyDa 21 48 0 14 Jun 2021
Compressed Gradient Tracking for Decentralized Optimization Over General Directed Networks Zhuoqing Song Lei Shi Shi Pu Ming Yan 19 24 0 14 Jun 2021
CFedAvg: Achieving Efficient Communication and Fast Convergence in Non-IID Federated Learning Haibo Yang Jia Liu Elizabeth S. Bentley FedML 21 17 0 14 Jun 2021
Federated Learning Over Wireless Channels: Dynamic Resource Allocation and Task Scheduling Shunfeng Chu Jun Li Jianxin Wang Zhe Wang Ming Ding Yijin Zang Yuwen Qian Wen Chen 33 15 0 13 Jun 2021
Communication-efficient SGD: From Local SGD to One-Shot Averaging Artin Spiridonoff Alexander Olshevsky I. Paschalidis FedML 39 20 0 09 Jun 2021
Theoretically Better and Numerically Faster Distributed Optimization with Smoothness-Aware Quantization Techniques Bokun Wang M. Safaryan Peter Richtárik MQ 11 10 0 07 Jun 2021
Neural Distributed Source Coding Jay Whang Alliot Nagle Anish Acharya Hyeji Kim A. Dimakis 40 20 0 05 Jun 2021
QLSD: Quantised Langevin stochastic dynamics for Bayesian federated learning Maxime Vono Vincent Plassier Alain Durmus Aymeric Dieuleveut Eric Moulines FedML 43 35 0 01 Jun 2021
H-FL: A Hierarchical Communication-Efficient and Privacy-Protected Architecture for Federated Learning He Yang 22 27 0 01 Jun 2021
Fast Federated Learning by Balancing Communication Trade-Offs Milad Khademi Nori Sangseok Yun Il-Min Kim FedML 32 53 0 23 May 2021
User-Level Label Leakage from Gradients in Federated Learning A. Wainakh Fabrizio G. Ventola Till Müßig Jens Keim Carlos Garcia Cordero Ephraim Zimmer Tim Grube Kristian Kersting M. Mühlhäuser FedML AAML 11 52 0 19 May 2021
Compressed Communication for Distributed Training: Adaptive Methods and System Yuchen Zhong Cong Xie Shuai Zheng Yanghua Peng 48 9 0 17 May 2021
LocalNewton: Reducing Communication Bottleneck for Distributed Learning Vipul Gupta Avishek Ghosh Michal Derezinski Rajiv Khanna Kannan Ramchandran Michael W. Mahoney 38 12 0 16 May 2021
Node Selection Toward Faster Convergence for Federated Learning on Non-IID Data Hongda Wu Ping Wang FedML 34 135 0 14 May 2021
Slashing Communication Traffic in Federated Learning by Transmitting Clustered Model Updates Laizhong Cui Xiaoxin Su Yipeng Zhou Yi Pan FedML 38 36 0 10 May 2021
OCTOPUS: Overcoming Performance andPrivatization Bottlenecks in Distributed Learning Shuo Wang Surya Nepal Kristen Moore M. Grobler Carsten Rudolph A. Abuadbba FedML 42 8 0 03 May 2021
Personalized Federated Learning by Structured and Unstructured Pruning under Data Heterogeneity Saeed Vahidian Mahdi Morafah Bill Lin 67 58 0 02 May 2021
GRNN: Generative Regression Neural Network -- A Data Leakage Attack for Federated Learning Hanchi Ren Jingjing Deng Xianghua Xie SILM AAML FedML 63 101 0 02 May 2021
ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training Jianfei Chen Lianmin Zheng Z. Yao Dequan Wang Ion Stoica Michael W. Mahoney Joseph E. Gonzalez MQ 32 74 0 29 Apr 2021
NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization Ali Ramezani-Kebrya Fartash Faghri Ilya Markov V. Aksenov Dan Alistarh Daniel M. Roy MQ 65 31 0 28 Apr 2021
Partitioning sparse deep neural networks for scalable training and inference G. Demirci Hakan Ferhatosmanoglu 25 11 0 23 Apr 2021
ScaleCom: Scalable Sparsified Gradient Compression for Communication-Efficient Distributed Training Chia-Yu Chen Jiamin Ni Songtao Lu Xiaodong Cui Pin-Yu Chen ... Naigang Wang Swagath Venkataramani Vijayalakshmi Srinivasan Wei Zhang K. Gopalakrishnan 31 0 0 21 Apr 2021
Communication Efficient Federated Learning with Adaptive Quantization Yuzhu Mao Zihao Zhao Guangfeng Yan Yang Liu Tian-Shing Lan Linqi Song Wenbo Ding MQ 30 10 0 13 Apr 2021
Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models Dheevatsa Mudigere Y. Hao Jianyu Huang Zhihao Jia Andrew Tulloch ... Ajit Mathews Lin Qiao M. Smelyanskiy Bill Jia Vijay Rao 40 150 0 12 Apr 2021
Federated Learning: A Signal Processing Perspective Tomer Gafni Nir Shlezinger Kobi Cohen Yonina C. Eldar H. Vincent Poor FedML 39 129 0 31 Mar 2021
Threshold-Based Data Exclusion Approach for Energy-Efficient Federated Edge Learning A. Albaseer M. Abdallah Ala I. Al-Fuqaha A. Erbad 40 12 0 30 Mar 2021
1-Bit Compressive Sensing for Efficient Federated Learning Over the Air Xin-Yue Fan Yue Wang Yan Huo Z. Tian FedML 22 33 0 30 Mar 2021
MergeComp: A Compression Scheduler for Scalable Communication-Efficient Distributed Training Zhuang Wang X. Wu T. Ng GNN 14 4 0 28 Mar 2021
Learned Gradient Compression for Distributed Deep Learning L. Abrahamyan Yiming Chen Giannis Bekoulis Nikos Deligiannis 40 46 0 16 Mar 2021
EventGraD: Event-Triggered Communication in Parallel Machine Learning Soumyadip Ghosh B. Aquino V. Gupta FedML 26 8 0 12 Mar 2021
Pufferfish: Communication-efficient Models At No Extra Cost Hongyi Wang Saurabh Agarwal Dimitris Papailiopoulos 19 56 0 05 Mar 2021
Large Graph Convolutional Network Training with GPU-Oriented Data Communication Architecture S. Min Kun Wu Sitao Huang Mert Hidayetouglu Jinjun Xiong Eiman Ebrahimi Deming Chen Wen-mei W. Hwu GNN 10 67 0 04 Mar 2021
Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices Max Ryabinin Eduard A. Gorbunov Vsevolod Plokhotnyuk Gennady Pekhimenko 39 33 0 04 Mar 2021
Temporal-Structure-Assisted Gradient Aggregation for Over-the-Air Federated Edge Learning Dian Fan Xiaojun Yuan Y. Zhang FedML 26 24 0 03 Mar 2021
Adaptive Transmission Scheduling in Wireless Networks for Asynchronous Federated Learning Hyun-Suk Lee Jang-Won Lee 81 53 0 02 Mar 2021
Towards Personalized Federated Learning A. Tan Han Yu Li-zhen Cui Qiang Yang FedML AI4CE 209 847 0 01 Mar 2021