TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning

22 May 2017

Yiran Chen

Papers citing "TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning"

50 / 467 papers shown

Title
Communication Compression for Byzantine Robust Learning: New Efficient Algorithms and Improved Rates Ahmad Rammal Kaja Gruntkowska Nikita Fedin Eduard A. Gorbunov Peter Richtárik 50 5 0 15 Oct 2023
FLrce: Resource-Efficient Federated Learning with Early-Stopping Strategy Ziru Niu Senior Member Ieee Hai Dong •. A. K. Qin Senior Member Ieee Tao Gu 33 4 0 15 Oct 2023
Fundamental Limits of Distributed Optimization over Multiple Access Channel Shubham K. Jha 36 1 0 05 Oct 2023
CORE: Common Random Reconstruction for Distributed Optimization with Provable Low Communication Complexity Pengyun Yue Hanzheng Zhao Cong Fang Di He Liwei Wang Zhouchen Lin Song-Chun Zhu 42 1 0 23 Sep 2023
FusionAI: Decentralized Training and Deploying LLMs with Massive Consumer-Level GPUs Zhenheng Tang Yuxin Wang Xin He Longteng Zhang Xinglin Pan ... Rongfei Zeng Kaiyong Zhao Shaoshuai Shi Bingsheng He Xuming Hu 46 30 0 03 Sep 2023
TinyProp -- Adaptive Sparse Backpropagation for Efficient TinyML On-device Learning Marcus Rüb Daniel Maier Daniel Mueller-Gritschneder Patrick Selle 36 3 0 17 Aug 2023
Distributed Extra-gradient with Optimal Complexity and Communication Guarantees Ali Ramezani-Kebrya Kimon Antonakopoulos Igor Krawczuk Justin Deschenaux V. Cevher 41 3 0 17 Aug 2023
Stochastic Controlled Averaging for Federated Learning with Communication Compression Xinmeng Huang Ping Li Xiaoyun Li 43 199 0 16 Aug 2023
Private Federated Learning with Autotuned Compression Enayat Ullah Christopher A. Choquette-Choo Peter Kairouz Sewoong Oh FedML 21 6 0 20 Jul 2023
Accelerating Distributed ML Training via Selective Synchronization S. Tyagi Martin Swany FedML 46 3 0 16 Jul 2023
Optimal Compression of Unit Norm Vectors in the High Distortion Regime He Zhu Avishek Ghosh A. Mazumdar 19 3 0 16 Jul 2023
Improved Convergence Analysis and SNR Control Strategies for Federated Learning in the Presence of Noise Antesh Upadhyay Abolfazl Hashemi 49 9 0 14 Jul 2023
Fedward: Flexible Federated Backdoor Defense Framework with Non-IID Data Zekai Chen Fuyi Wang Zhiwei Zheng Ximeng Liu Yujie Lin FedML AAML 47 3 0 01 Jul 2023
Adaptive Compression in Federated Learning via Side Information Berivan Isik Francesco Pase Deniz Gunduz Sanmi Koyejo Tsachy Weissman M. Zorzi FedML 36 9 0 22 Jun 2023
An Efficient Virtual Data Generation Method for Reducing Communication in Federated Learning Cheng Yang Xue Yang Dongxian Wu Xiaohu Tang FedML 29 0 0 21 Jun 2023
Evaluation and Optimization of Gradient Compression for Distributed Deep Learning Lin Zhang Longteng Zhang Shaoshuai Shi Xiaowen Chu Bo Li OffRL 25 7 0 15 Jun 2023
Exact Optimality of Communication-Privacy-Utility Tradeoffs in Distributed Mean Estimation Berivan Isik Wei-Ning Chen Ayfer Özgür Tsachy Weissman Albert No 63 19 0 08 Jun 2023
Global-QSGD: Practical Floatless Quantization for Distributed Learning with Theoretical Guarantees Jihao Xin Marco Canini Peter Richtárik Samuel Horváth 43 2 0 29 May 2023
Reducing Communication for Split Learning by Randomized Top-k Sparsification Fei Zheng Chaochao Chen Lingjuan Lyu Binhui Yao FedML 31 10 0 29 May 2023
Knowledge Distillation Performs Partial Variance Reduction M. Safaryan Alexandra Peste Dan Alistarh 30 6 0 27 May 2023
Unbiased Compression Saves Communication in Distributed Optimization: When and How Much? Yutong He Xinmeng Huang Kun Yuan 31 9 0 25 May 2023
Lower Bounds and Accelerated Algorithms in Distributed Stochastic Optimization with Communication Compression Yutong He Xinmeng Huang Yiming Chen W. Yin Kun Yuan 36 7 0 12 May 2023
Joint Compression and Deadline Optimization for Wireless Federated Learning Maojun Zhang Yongqian Li Dongzhu Liu Richeng Jin Guangxu Zhu Caijun Zhong Tony Q. S. Quek 37 5 0 06 May 2023
A Survey on Approximate Edge AI for Energy Efficient Autonomous Driving Services Dewant Katare Diego Perino J. Nurmi M. Warnier Marijn Janssen Aaron Yi Ding 36 36 0 13 Apr 2023
Approximate Wireless Communication for Federated Learning Xiang Ma Haijian Sun R. Hu Y. Qian 17 3 0 06 Apr 2023
Privacy Amplification via Compression: Achieving the Optimal Privacy-Accuracy-Communication Trade-off in Distributed Mean Estimation Wei-Ning Chen Danni Song Ayfer Özgür Peter Kairouz FedML 34 25 0 04 Apr 2023
SparDL: Distributed Deep Learning Training with Efficient Sparse Communication Minjun Zhao Yichen Yin Yuren Mao Qing Liu Lu Chen Yunjun Gao 26 1 0 03 Apr 2023
Communication-Efficient Design for Quantized Decentralized Federated Learning L. Chen Wei Liu Yunfei Chen Weidong Wang FedML MQ 60 14 0 15 Mar 2023
FedREP: A Byzantine-Robust, Communication-Efficient and Privacy-Preserving Framework for Federated Learning Yi-Rui Yang Kun Wang Wulu Li FedML 52 3 0 09 Mar 2023
Boosting Distributed Full-graph GNN Training with Asynchronous One-bit Communication Mengdie Zhang Qi Hu Peng Sun Yonggang Wen Tianwei Zhang GNN 40 5 0 02 Mar 2023
FederatedTrust: A Solution for Trustworthy Federated Learning Pedro Miguel Sánchez Sánchez Alberto Huertas Celdrán Ning Xie Gérome Bovet Gregorio Martínez Pérez Burkhard Stiller 38 21 0 20 Feb 2023
Magnitude Matters: Fixing SIGNSGD Through Magnitude-Aware Sparsification in the Presence of Data Heterogeneity Richeng Jin Xiaofan He C. Zhong Zhaoyang Zhang Tony Q.S. Quek H. Dai FedML 26 1 0 19 Feb 2023
Breaking the Communication-Privacy-Accuracy Tradeoff with $f$ -Differential Privacy Richeng Jin Z. Su C. Zhong Zhaoyang Zhang Tony Q.S. Quek H. Dai FedML 42 2 0 19 Feb 2023
THC: Accelerating Distributed Deep Learning Using Tensor Homomorphic Compression Minghao Li Ran Ben-Basat S. Vargaftik Chon-In Lao Ke Xu Michael Mitzenmacher Minlan Yu Harvard University 31 15 0 16 Feb 2023
Sparse-SignSGD with Majority Vote for Communication-Efficient Distributed Learning Chanho Park Namyoon Lee FedML 35 3 0 15 Feb 2023
Federated Learning with Regularized Client Participation Grigory Malinovsky Samuel Horváth Konstantin Burlachenko Peter Richtárik FedML 36 13 0 07 Feb 2023
DoCoFL: Downlink Compression for Cross-Device Federated Learning Ron Dorfman S. Vargaftik Y. Ben-Itzhak Kfir Y. Levy FedML 37 19 0 01 Feb 2023
Optimus-CC: Efficient Large NLP Model Training with 3D Parallelism Aware Communication Compression Jaeyong Song Jinkyu Yim Jaewon Jung Hongsun Jang H. Kim Youngsok Kim Jinho Lee GNN 34 25 0 24 Jan 2023
M22: A Communication-Efficient Algorithm for Federated Learning Inspired by Rate-Distortion Yangyi Liu Stefano Rini Sadaf Salehkalaibar Jun Chen FedML 21 4 0 23 Jan 2023
ScaDLES: Scalable Deep Learning over Streaming data at the Edge S. Tyagi Martin Swany 25 6 0 21 Jan 2023
AutoDDL: Automatic Distributed Deep Learning with Near-Optimal Bandwidth Cost Jinfan Chen Shigang Li Ran Guo Jinhui Yuan Torsten Hoefler 31 2 0 17 Jan 2023
Temporal Difference Learning with Compressed Updates: Error-Feedback meets Reinforcement Learning A. Mitra George J. Pappas Hamed Hassani 33 12 0 03 Jan 2023
Deep Hierarchy Quantization Compression algorithm based on Dynamic Sampling W. Jiang Gang Liu Xiaofeng Chen Yipeng Zhou FedML 24 0 0 30 Dec 2022
Adaptive Control of Client Selection and Gradient Compression for Efficient Federated Learning Zhida Jiang Yang Xu Hong-Ze Xu Zhiyuan Wang Chen Qian 28 9 0 19 Dec 2022
Communication-Efficient Federated Learning for Heterogeneous Edge Devices Based on Adaptive Gradient Quantization Heting Liu Fang He Guohong Cao FedML MQ 37 24 0 16 Dec 2022
Distributed Deep Reinforcement Learning: A Survey and A Multi-Player Multi-Agent Learning Toolbox Qiyue Yin Tongtong Yu S. Shen Jun Yang Meijing Zhao Kaiqi Huang Bin Liang Liangsheng Wang OffRL 33 13 0 01 Dec 2022
Linear Convergent Distributed Nash Equilibrium Seeking with Compression Xiaomeng Chen Yuchi Wu Xinlei Yi Minyi Huang Ling Shi 16 4 0 15 Nov 2022
Adaptive Compression for Communication-Efficient Distributed Training Maksim Makarenko Elnur Gasanov Rustem Islamov Abdurakhmon Sadiev Peter Richtárik 52 14 0 31 Oct 2022
L-GreCo: Layerwise-Adaptive Gradient Compression for Efficient and Accurate Deep Learning Mohammadreza Alimohammadi I. Markov Elias Frantar Dan Alistarh 37 5 0 31 Oct 2022
FedGRec: Federated Graph Recommender System with Lazy Update of Latent Embeddings Junyi Li Heng-Chiao Huang FedML 24 6 0 25 Oct 2022