ResearchTrend.AI
TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning
v1v2v3v4v5v6 (latest)

TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning

W. Wen
Cong Xu
Feng Yan
Chunpeng Wu
Yandan Wang
Yiran Chen
Hai Helen Li
ArXiv (abs)PDFHTML

Papers citing "TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning"

50 / 469 papers shown
Title
Memory-Efficient FastText: A Comprehensive Approach Using Double-Array Trie Structures and Mark-Compact Memory Management
Memory-Efficient FastText: A Comprehensive Approach Using Double-Array Trie Structures and Mark-Compact Memory Management
Yimin Du
VLM
61
0
0
02 Jun 2025
PacTrain: Pruning and Adaptive Sparse Gradient Compression for Efficient Collective Communication in Distributed Deep Learning
PacTrain: Pruning and Adaptive Sparse Gradient Compression for Efficient Collective Communication in Distributed Deep Learning
Yisu Wang
Ruilong Wu
Xinjiao Li
Dirk Kutscher
235
0
0
24 May 2025
Striving for Simplicity: Simple Yet Effective Prior-Aware Pseudo-Labeling for Semi-Supervised Ultrasound Image Segmentation
Striving for Simplicity: Simple Yet Effective Prior-Aware Pseudo-Labeling for Semi-Supervised Ultrasound Image Segmentation
Yaxiong Chen
Yujie Wang
Zixuan Zheng
Jingliang Hu
Yilei Shi
Shengwu Xiong
Xiao Xiang Zhu
Lichao Mou
148
1
0
18 Mar 2025
Layer-wise Update Aggregation with Recycling for Communication-Efficient Federated Learning
Jisoo Kim
Sungmin Kang
Sunwoo Lee
FedML
74
0
0
14 Mar 2025
Accelerated Distributed Optimization with Compression and Error Feedback
Accelerated Distributed Optimization with Compression and Error Feedback
Yuan Gao
Anton Rodomanov
Jeremy Rack
Sebastian U. Stich
89
0
0
11 Mar 2025
MARINA-P: Superior Performance in Non-smooth Federated Optimization with Adaptive Stepsizes
Igor Sokolov
Peter Richtárik
149
1
0
22 Dec 2024
Lion Cub: Minimizing Communication Overhead in Distributed Lion
Lion Cub: Minimizing Communication Overhead in Distributed Lion
Satoki Ishikawa
Tal Ben-Nun
B. Van Essen
Rio Yokota
Nikoli Dryden
122
0
0
25 Nov 2024
Problem-dependent convergence bounds for randomized linear gradient compression
Problem-dependent convergence bounds for randomized linear gradient compression
Thomas Flynn
Patrick R. Johnstone
Shinjae Yoo
181
0
0
19 Nov 2024
$r$Age-$k$: Communication-Efficient Federated Learning Using Age Factor
rrAge-kk: Communication-Efficient Federated Learning Using Age Factor
Matin Mortaheb
Priyanka Kaswan
S. Ulukus
FedML
89
1
0
29 Oct 2024
Age-of-Gradient Updates for Federated Learning over Random Access Channels
Age-of-Gradient Updates for Federated Learning over Random Access Channels
Yu Heng Wu
Houman Asgari
Stefano Rini
Andrea Munari
FedML
48
1
0
15 Oct 2024
Asynchronous Stochastic Gradient Descent with Decoupled Backpropagation and Layer-Wise Updates
Asynchronous Stochastic Gradient Descent with Decoupled Backpropagation and Layer-Wise Updates
Cabrel Teguemne Fokam
Khaleelulla Khan Nazeer
Lukas König
David Kappel
Anand Subramoney
90
0
0
08 Oct 2024
Memory-Efficient LLM Training with Online Subspace Descent
Memory-Efficient LLM Training with Online Subspace Descent
Kaizhao Liang
Bo Liu
Lizhang Chen
Qiang Liu
65
15
0
23 Aug 2024
A Joint Approach to Local Updating and Gradient Compression for Efficient Asynchronous Federated Learning
A Joint Approach to Local Updating and Gradient Compression for Efficient Asynchronous Federated Learning
Jiajun Song
Jiajun Luo
Rongwei Lu
Shuzhao Xie
Bin Chen
Zhi Wang
FedML
49
0
0
06 Jul 2024
LoCo: Low-Bit Communication Adaptor for Large-scale Model Training
LoCo: Low-Bit Communication Adaptor for Large-scale Model Training
Xingyu Xie
Zhijie Lin
Kim-Chuan Toh
Pan Zhou
94
3
0
05 Jul 2024
Beyond Throughput and Compression Ratios: Towards High End-to-end Utility of Gradient Compression
Beyond Throughput and Compression Ratios: Towards High End-to-end Utility of Gradient Compression
Wenchen Han
S. Vargaftik
Michael Mitzenmacher
Brad Karp
Ran Ben-Basat
80
3
0
01 Jul 2024
FedEx: Expediting Federated Learning over Heterogeneous Mobile Devices by Overlapping and Participant Selection
FedEx: Expediting Federated Learning over Heterogeneous Mobile Devices by Overlapping and Participant Selection
Jiaxiang Geng
Boyu Li
Xiaoqi Qin
Yixuan Li
Liang Li
Yanzhao Hou
Miao Pan
FedML
106
0
0
01 Jul 2024
FedAQ: Communication-Efficient Federated Edge Learning via Joint Uplink and Downlink Adaptive Quantization
FedAQ: Communication-Efficient Federated Edge Learning via Joint Uplink and Downlink Adaptive Quantization
Linping Qu
Shenghui Song
Chi-Ying Tsui
MQFedML
53
4
0
26 Jun 2024
Grass: Compute Efficient Low-Memory LLM Training with Structured Sparse Gradients
Grass: Compute Efficient Low-Memory LLM Training with Structured Sparse Gradients
Aashiq Muhamed
Oscar Li
David Woodruff
Mona Diab
Virginia Smith
94
13
0
25 Jun 2024
FLUX: Fast Software-based Communication Overlap On GPUs Through Kernel Fusion
FLUX: Fast Software-based Communication Overlap On GPUs Through Kernel Fusion
Li-Wen Chang
Yiyuan Ma
Qi Hou
Chengquan Jiang
Ningxin Zheng
...
Zuquan Song
Ziheng Jiang
Yanghua Peng
Xuanzhe Liu
Xin Liu
88
26
0
11 Jun 2024
VeLoRA: Memory Efficient Training using Rank-1 Sub-Token Projections
VeLoRA: Memory Efficient Training using Rank-1 Sub-Token Projections
Roy Miles
Pradyumna Reddy
Ismail Elezi
Jiankang Deng
VLM
73
7
0
28 May 2024
Full-Stack Allreduce on Multi-Rail Networks
Full-Stack Allreduce on Multi-Rail Networks
Enda Yu
Dezun Dong
Xiangke Liao
GNN
67
0
0
28 May 2024
What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functions
What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functions
Sang Keun Choe
Hwijeen Ahn
Juhan Bae
Kewen Zhao
Minsoo Kang
...
Teruko Mitamura
Jeff Schneider
Eduard Hovy
Roger C. Grosse
Eric Xing
TDI
97
44
0
22 May 2024
Improved Communication-Privacy Trade-offs in $L_2$ Mean Estimation under Streaming Differential Privacy
Improved Communication-Privacy Trade-offs in L2L_2 Mean Estimation under Streaming Differential Privacy
Wei-Ning Chen
Berivan Isik
Peter Kairouz
Albert No
Sewoong Oh
Zheng Xu
97
3
0
02 May 2024
Approximate Wireless Communication for Lossy Gradient Updates in IoT Federated Learning
Approximate Wireless Communication for Lossy Gradient Updates in IoT Federated Learning
Xiang Ma
Haijian Sun
Rose Qingyang Hu
Yi Qian
75
0
0
17 Apr 2024
Hybrid FedGraph: An efficient hybrid federated learning algorithm using graph convolutional neural network
Hybrid FedGraph: An efficient hybrid federated learning algorithm using graph convolutional neural network
Jaeyeon Jang
Diego Klabjan
Veena Mendiratta
Fanfei Meng
FedML
52
1
0
15 Apr 2024
Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey
Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey
Feng Liang
Zhen Zhang
Haifeng Lu
Victor C. M. Leung
Yanyi Guo
Xiping Hu
GNN
103
8
0
09 Apr 2024
A Survey on Error-Bounded Lossy Compression for Scientific Datasets
A Survey on Error-Bounded Lossy Compression for Scientific Datasets
Sheng Di
Jinyang Liu
Kai Zhao
Xin Liang
Robert Underwood
...
Jon C. Calhoun
Guanpeng Li
Kazutomo Yoshii
Khalid Ayed Alharthi
Franck Cappello
AI4CE
88
16
0
03 Apr 2024
Compressed Federated Reinforcement Learning with a Generative Model
Compressed Federated Reinforcement Learning with a Generative Model
Ali Beikmohammadi
Sarit Khirirat
Sindri Magnússon
FedML
125
3
0
26 Mar 2024
SignSGD with Federated Voting
SignSGD with Federated Voting
Chanho Park
H. Vincent Poor
Namyoon Lee
FedML
48
1
0
25 Mar 2024
Quantization Avoids Saddle Points in Distributed Optimization
Quantization Avoids Saddle Points in Distributed Optimization
Yanan Bo
Yongqiang Wang
MQ
57
2
0
15 Mar 2024
Random Aggregate Beamforming for Over-the-Air Federated Learning in Large-Scale Networks
Random Aggregate Beamforming for Over-the-Air Federated Learning in Large-Scale Networks
Chunmei Xu
Shengheng Liu
Yongming Huang
Björn E. Ottersten
Dusist Niyato
FedML
82
3
0
20 Feb 2024
Communication-Efficient Distributed Learning with Local Immediate Error Compensation
Communication-Efficient Distributed Learning with Local Immediate Error Compensation
Yifei Cheng
Li Shen
Linli Xu
Xun Qian
Shiwei Wu
Yiming Zhou
Tie Zhang
Dacheng Tao
Enhong Chen
60
0
0
19 Feb 2024
Smart Information Exchange for Unsupervised Federated Learning via Reinforcement Learning
Smart Information Exchange for Unsupervised Federated Learning via Reinforcement Learning
Seohyun Lee
Anindya Bijoy Das
Satyavrat Wagle
Christopher G. Brinton
FedML
47
0
0
15 Feb 2024
Accelerating Distributed Deep Learning using Lossless Homomorphic Compression
Accelerating Distributed Deep Learning using Lossless Homomorphic Compression
Haoyu Li
Yuchen Xu
Jiayi Chen
Rohit Dwivedula
Wenfei Wu
Keqiang He
Aditya Akella
Daehyeok Kim
FedMLAI4CE
42
4
0
12 Feb 2024
Expediting In-Network Federated Learning by Voting-Based Consensus Model Compression
Expediting In-Network Federated Learning by Voting-Based Consensus Model Compression
Xiaoxin Su
Yipeng Zhou
Laizhong Cui
Song Guo
FedML
53
3
0
06 Feb 2024
Fed-CVLC: Compressing Federated Learning Communications with Variable-Length Codes
Fed-CVLC: Compressing Federated Learning Communications with Variable-Length Codes
Xiaoxin Su
Yipeng Zhou
Laizhong Cui
John C. S. Lui
Jiangchuan Liu
FedML
74
2
0
06 Feb 2024
Breaking MLPerf Training: A Case Study on Optimizing BERT
Breaking MLPerf Training: A Case Study on Optimizing BERT
Yongdeok Kim
Jaehyung Ahn
Myeongwoo Kim
Changin Choi
Heejae Kim
...
Xiongzhan Linghu
Jingkun Ma
Lin Chen
Yuehua Dai
Sungjoo Yoo
52
0
0
04 Feb 2024
Improved Quantization Strategies for Managing Heavy-tailed Gradients in Distributed Learning
Improved Quantization Strategies for Managing Heavy-tailed Gradients in Distributed Learning
Guangfeng Yan
Tan Li
Yuanzhang Xiao
Hanxu Hou
Linqi Song
MQ
53
0
0
02 Feb 2024
Truncated Non-Uniform Quantization for Distributed SGD
Truncated Non-Uniform Quantization for Distributed SGD
Guangfeng Yan
Tan Li
Yuanzhang Xiao
Congduan Li
Linqi Song
MQ
48
0
0
02 Feb 2024
RS-DGC: Exploring Neighborhood Statistics for Dynamic Gradient Compression on Remote Sensing Image Interpretation
RS-DGC: Exploring Neighborhood Statistics for Dynamic Gradient Compression on Remote Sensing Image Interpretation
Weiying Xie
Zixuan Wang
Jitao Ma
Daixun Li
Yunsong Li
63
0
0
29 Dec 2023
FedSZ: Leveraging Error-Bounded Lossy Compression for Federated Learning Communications
FedSZ: Leveraging Error-Bounded Lossy Compression for Federated Learning Communications
Grant Wilkins
Sheng Di
Jon C. Calhoun
Zilinghan Li
Kibaek Kim
Robert Underwood
Richard Mortier
Franck Cappello
FedML
73
4
0
20 Dec 2023
Fed-QSSL: A Framework for Personalized Federated Learning under Bitwidth and Data Heterogeneity
Fed-QSSL: A Framework for Personalized Federated Learning under Bitwidth and Data Heterogeneity
Yiyue Chen
H. Vikalo
C. Wang
FedML
88
5
0
20 Dec 2023
Contractive error feedback for gradient compression
Contractive error feedback for gradient compression
Bingcong Li
Shuai Zheng
Parameswaran Raman
Anshumali Shrivastava
G. Giannakis
66
0
0
13 Dec 2023
Kimad: Adaptive Gradient Compression with Bandwidth Awareness
Kimad: Adaptive Gradient Compression with Bandwidth Awareness
Jihao Xin
Ivan Ilin
Shunkang Zhang
Marco Canini
Peter Richtárik
71
3
0
13 Dec 2023
The Landscape of Modern Machine Learning: A Review of Machine, Distributed and Federated Learning
The Landscape of Modern Machine Learning: A Review of Machine, Distributed and Federated Learning
Omer Subasi
Oceane Bel
Joseph Manzano
Kevin J. Barker
FedMLOODPINN
91
2
0
05 Dec 2023
Flexible Communication for Optimal Distributed Learning over Unpredictable Networks
Flexible Communication for Optimal Distributed Learning over Unpredictable Networks
S. Tyagi
Martin Swany
98
1
0
05 Dec 2023
Edge AI for Internet of Energy: Challenges and Perspectives
Edge AI for Internet of Energy: Challenges and Perspectives
Yassine Himeur
A. Sayed
A. Alsalemi
F. Bensaali
Abbes Amira
135
31
0
28 Nov 2023
ComPEFT: Compression for Communicating Parameter Efficient Updates via Sparsification and Quantization
ComPEFT: Compression for Communicating Parameter Efficient Updates via Sparsification and Quantization
Prateek Yadav
Leshem Choshen
Colin Raffel
Mohit Bansal
89
14
0
22 Nov 2023
EControl: Fast Distributed Optimization with Compression and Error Control
EControl: Fast Distributed Optimization with Compression and Error Control
Yuan Gao
Rustem Islamov
Sebastian U. Stich
84
8
0
06 Nov 2023
Compression with Exact Error Distribution for Federated Learning
Compression with Exact Error Distribution for Federated Learning
Mahmoud Hegazy
Rémi Leluc
Cheuk Ting Li
Aymeric Dieuleveut
FedML
58
11
0
31 Oct 2023

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.