v1v2v3 (latest)

Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training

5 December 2017

Song Han

ArXiv (abs)PDF HTML Github (222★)

Papers citing "Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training"

50 / 625 papers shown

Title
TopoOpt: Co-optimizing Network Topology and Parallelization Strategy for Distributed Training Jobs Weiyang Wang Moein Khazraee Zhizhen Zhong M. Ghobadi Zhihao Jia Dheevatsa Mudigere Ying Zhang A. Kewitsch 131 93 0 01 Feb 2022
Faster Convergence of Local SGD for Over-Parameterized Models Tiancheng Qin S. Rasoul Etesami César A. Uribe FedML 88 6 0 30 Jan 2022
Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges Nuria Rodríguez-Barroso Daniel Jiménez López M. V. Luzón Francisco Herrera Eugenio Martínez-Cámara FedML 85 229 0 20 Jan 2022
DEFER: Distributed Edge Inference for Deep Neural Networks Arjun Parthasarathy Bhaskar Krishnamachari 39 15 0 18 Jan 2022
Egeria: Efficient DNN Training with Knowledge-Guided Layer Freezing Yiding Wang D. Sun Kai Chen Fan Lai Mosharaf Chowdhury 97 47 0 17 Jan 2022
Optimizing the Communication-Accuracy Trade-off in Federated Learning with Rate-Distortion Theory Nicole Mitchell Johannes Ballé Zachary B. Charles Jakub Konecný FedML 128 21 0 07 Jan 2022
Automatic Configuration for Optimal Communication Scheduling in DNN Training Yiqing Ma Hao Wang Yiming Zhang Kai Chen 35 12 0 27 Dec 2021
Sparsified Secure Aggregation for Privacy-Preserving Federated Learning Irem Ergun Hasin Us Sami Başak Güler FedML 85 26 0 23 Dec 2021
FedLGA: Towards System-Heterogeneity of Federated Learning via Local Gradient Approximation Xingyu Li Zhe Qu Bo Tang Zhuo Lu FedML 109 28 0 22 Dec 2021
Efficient Strong Scaling Through Burst Parallel Training S. Park Joshua Fried Sunghyun Kim Mohammad Alizadeh Adam Belay GNN LRM 64 11 0 19 Dec 2021
Efficient Differentially Private Secure Aggregation for Federated Learning via Hardness of Learning with Errors Timothy Stevens Christian Skalka C. Vincent J. Ring Samuel Clark Joseph P. Near FedML 75 73 0 13 Dec 2021
Optimal Rate Adaption in Federated Learning with Compressed Communications Laizhong Cui Xiaoxin Su Yipeng Zhou Jiangchuan Liu FedML 92 43 0 13 Dec 2021
SparseFed: Mitigating Model Poisoning Attacks in Federated Learning with Sparsification Ashwinee Panda Saeed Mahloujifar A. Bhagoji Supriyo Chakraborty Prateek Mittal FedML AAML 76 88 0 12 Dec 2021
Batch Label Inference and Replacement Attacks in Black-Boxed Vertical Federated Learning Yang Liu Tianyuan Zou Yan Kang Wenhan Liu Yuanqin He Zhi-qian Yi Qian Yang FedML AAML 140 20 0 10 Dec 2021
PMFL: Partial Meta-Federated Learning for heterogeneous tasks and its applications on real-world medical records Tianyi Zhang Shirui Zhang Ziwei Chen Dianbo Liu FedML 78 4 0 10 Dec 2021
FastSGD: A Fast Compressed SGD Framework for Distributed Machine Learning Keyu Yang Lu Chen Zhihao Zeng Yunjun Gao 49 9 0 08 Dec 2021
Communication-Efficient Distributed Learning via Sparse and Adaptive Stochastic Gradient Xiaoge Deng Dongsheng Li Tao Sun Xicheng Lu FedML 60 0 0 08 Dec 2021
Collaborative Learning over Wireless Networks: An Introductory Overview Emre Ozfatura Deniz Gunduz H. Vincent Poor 76 12 0 07 Dec 2021
Intrinisic Gradient Compression for Federated Learning Luke Melas-Kyriazi Franklyn Wang FedML 25 3 0 05 Dec 2021
Communication-Efficient Federated Learning via Quantized Compressed Sensing Yong-Nam Oh Namyoon Lee Yo-Seb Jeon H. Vincent Poor FedML MQ 80 36 0 30 Nov 2021
Doing More by Doing Less: How Structured Partial Backpropagation Improves Deep Learning Clusters Adarsh Kumar Kausik Subramanian Shivaram Venkataraman Aditya Akella 33 5 0 20 Nov 2021
An Expectation-Maximization Perspective on Federated Learning Christos Louizos M. Reisser Joseph B. Soriaga Max Welling FedML 83 12 0 19 Nov 2021
COMET: A Novel Memory-Efficient Deep Learning Training Framework by Using Error-Bounded Lossy Compression Sian Jin Chengming Zhang Xintong Jiang Yunhe Feng Hui Guan Guanpeng Li Shuaiwen Leon Song Dingwen Tao 46 25 0 18 Nov 2021
CGX: Adaptive System Support for Communication-Efficient Deep Learning I. Markov Hamidreza Ramezanikebrya Dan Alistarh GNN 82 5 0 16 Nov 2021
CoReS: Compatible Representations via Stationarity Niccoló Biondi F. Pernici Matteo Bruni A. Bimbo OOD 65 9 0 15 Nov 2021
Edge-Native Intelligence for 6G Communications Driven by Federated Learning: A Survey of Trends and Challenges Mohammad M. Al-Quraan Lina S. Mohjazi Lina Bariah A. Centeno A. Zoha Sami Muhaidat Mérouane Debbah Muhammad Ali Imran 102 67 0 14 Nov 2021
SEOFP-NET: Compression and Acceleration of Deep Neural Networks for Speech Enhancement Using Sign-Exponent-Only Floating-Points Yu-Chen Lin Cheng Yu Y. Hsu Szu-Wei Fu Yu Tsao Tei-Wei Kuo 31 6 0 08 Nov 2021
Finite-Time Consensus Learning for Decentralized Optimization with Nonlinear Gossiping Junya Chen Sijia Wang Lawrence Carin Chenyang Tao 39 3 0 04 Nov 2021
Large-Scale Deep Learning Optimizations: A Comprehensive Survey Xiaoxin He Fuzhao Xue Xiaozhe Ren Yang You 90 15 0 01 Nov 2021
Gradient Inversion with Generative Image Prior Jinwoo Jeon Jaechang Kim Kangwook Lee Sewoong Oh Jungseul Ok 77 158 0 28 Oct 2021
Federated Learning over Wireless IoT Networks with Optimized Communication and Resources Student Member Ieee Hao Chen Shaocheng Huang Deyou Zhang Ming Xiao Fellow Ieee Mikael Skoglund L. F. I. H. Vincent Poor 93 96 0 22 Oct 2021
BEV-SGD: Best Effort Voting SGD for Analog Aggregation Based Federated Learning against Byzantine Attackers Xin-Yue Fan Yue Wang Yan Huo Zhi Tian FedML 88 24 0 18 Oct 2021
EmbRace: Accelerating Sparse Communication for Distributed Training of NLP Neural Networks Shengwei Li Zhiquan Lai Dongsheng Li Yiming Zhang Xiangyu Ye Yabo Duan FedML 63 3 0 18 Oct 2021
Federated learning and next generation wireless communications: A survey on bidirectional relationship Debaditya Shome Omer Waqar Wali Ullah Khan 88 32 0 14 Oct 2021
ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Training Hui-Po Wang Sebastian U. Stich Yang He Mario Fritz FedML AI4CE 73 50 0 11 Oct 2021
An Empirical Study on Compressed Decentralized Stochastic Gradient Algorithms with Overparameterized Models A. Rao Hoi-To Wai 28 0 0 09 Oct 2021
Distributed Methods with Compressed Communication for Solving Variational Inequalities, with Theoretical Guarantees Aleksandr Beznosikov Peter Richtárik Michael Diskin Max Ryabinin Alexander Gasnikov FedML 105 22 0 07 Oct 2021
S2 Reducer: High-Performance Sparse Communication to Accelerate Distributed Deep Learning Ke-shi Ge Yongquan Fu Zhiquan Lai Xiaoge Deng Dongsheng Li 48 2 0 05 Oct 2021
Solon: Communication-efficient Byzantine-resilient Distributed Training via Redundant Gradients Lingjiao Chen Leshang Chen Hongyi Wang S. Davidson Yan Sun FedML 66 1 0 04 Oct 2021
Algorithm Fairness in AI for Medicine and Healthcare Richard J. Chen Tiffany Y. Chen Jana Lipkova Judy J. Wang Drew F. K. Williamson Ming Y. Lu S. Sahai Faisal Mahmood FaML 150 47 0 01 Oct 2021
Unbiased Single-scale and Multi-scale Quantizers for Distributed Optimization S. Vineeth MQ 35 0 0 26 Sep 2021
Toward Communication Efficient Adaptive Gradient Method Xiangyi Chen Xiaoyun Li P. Li FedML 80 42 0 10 Sep 2021
On the Convergence of Decentralized Adaptive Gradient Methods Xiangyi Chen Belhal Karimi Weijie Zhao Ping Li 85 21 0 07 Sep 2021
FLASHE: Additively Symmetric Homomorphic Encryption for Cross-Silo Federated Learning Zhifeng Jiang Wen Wang Yang Liu FedML 96 50 0 02 Sep 2021
Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions Yang Wu Dingheng Wang Xiaotong Lu Fan Yang Guoqi Li W. Dong Jianbo Shi 104 18 0 30 Aug 2021
Rate distortion comparison of a few gradient quantizers Tharindu B. Adikari MQ 20 0 0 23 Aug 2021
FedSkel: Efficient Federated Learning on Heterogeneous Systems with Skeleton Gradients Update Junyu Luo Jianlei Yang Xucheng Ye Xin Guo Weisheng Zhao FedML 59 14 0 20 Aug 2021
EDEN: Communication-Efficient and Robust Distributed Mean Estimation for Federated Learning S. Vargaftik Ran Ben-Basat Amit Portnoy Gal Mendelson Y. Ben-Itzhak Michael Mitzenmacher FedML 118 49 0 19 Aug 2021
Compressing gradients by exploiting temporal correlation in momentum-SGD Tharindu B. Adikari S. Draper 20 0 0 17 Aug 2021
FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learning Nam Hyeon-Woo Moon Ye-Bin Tae-Hyun Oh FedML 132 125 0 13 Aug 2021