Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales
Search
Open menu
Home
Papers
1802.08021
Cited By
v1
v2
v3 (latest)
SparCML: High-Performance Sparse Communication for Machine Learning
22 February 2018
Cédric Renggli
Saleh Ashkboos
Mehdi Aghagolzadeh
Dan Alistarh
Torsten Hoefler
Re-assign community
ArXiv (abs)
PDF
HTML
Papers citing
"SparCML: High-Performance Sparse Communication for Machine Learning"
50 / 66 papers shown
Lion Cub: Minimizing Communication Overhead in Distributed Lion
Satoki Ishikawa
Tal Ben-Nun
B. Van Essen
Rio Yokota
Nikoli Dryden
466
1
0
25 Nov 2024
DQRM: Deep Quantized Recommendation Models
Yang Zhou
Zhen Dong
Ellick Chan
Dhiraj Kalamkar
Diana Marculescu
Kurt Keutzer
MQ
414
3
0
26 Oct 2024
Parm: Efficient Training of Large Sparsely-Activated Models with Dedicated Schedules
Xinglin Pan
Wenxiang Lin
Shaoshuai Shi
Xiaowen Chu
Weinong Sun
Bo Li
MoE
268
10
0
30 Jun 2024
Grass: Compute Efficient Low-Memory LLM Training with Structured Sparse Gradients
Aashiq Muhamed
Oscar Li
David Woodruff
Mona Diab
Virginia Smith
285
20
0
25 Jun 2024
Full-Stack Allreduce on Multi-Rail Networks
Enda Yu
Dezun Dong
Xiangke Liao
GNN
251
1
0
28 May 2024
Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey
Feng Liang
Zhen Zhang
Haifeng Lu
Victor C. M. Leung
Yanyi Guo
Xiping Hu
GNN
378
27
0
09 Apr 2024
A Survey on Error-Bounded Lossy Compression for Scientific Datasets
ACM Computing Surveys (ACM CSUR), 2024
Sheng Di
Jinyang Liu
Kai Zhao
Xin Liang
Robert Underwood
...
Jon C. Calhoun
Guanpeng Li
Kazutomo Yoshii
Khalid Ayed Alharthi
Franck Cappello
AI4CE
299
54
0
03 Apr 2024
Accelerating Distributed Deep Learning using Lossless Homomorphic Compression
Haoyu Li
Yuchen Xu
Jiayi Chen
Rohit Dwivedula
Wenfei Wu
Keqiang He
Aditya Akella
Daehyeok Kim
FedML
AI4CE
210
8
0
12 Feb 2024
FedSZ: Leveraging Error-Bounded Lossy Compression for Federated Learning Communications
Grant Wilkins
Sheng Di
Jon C. Calhoun
Zilinghan Li
Kibaek Kim
Robert Underwood
Richard Mortier
Franck Cappello
FedML
304
9
0
20 Dec 2023
Compressed and Sparse Models for Non-Convex Decentralized Learning
Andrew Campbell
Hang Liu
Leah Woldemariam
Anna Scaglione
247
0
0
09 Nov 2023
Ultima: Robust and Tail-Optimal AllReduce for Distributed Deep Learning in the Cloud
Symposium on Networked Systems Design and Implementation (NSDI), 2023
Ertza Warraich
Omer Shabtai
Khalid Manaa
S. Vargaftik
Y. Piasetzky
Matty Kadosh
Lalith Suresh
Muhammad Shahbaz
196
1
0
10 Oct 2023
Canary: Congestion-Aware In-Network Allreduce Using Dynamic Trees
Future generations computer systems (FGCS), 2023
Daniele De Sensi
Edgar Costa Molero
Salvatore Di Girolamo
Laurent Vanbever
Torsten Hoefler
223
8
0
28 Sep 2023
Evaluation and Optimization of Gradient Compression for Distributed Deep Learning
IEEE International Conference on Distributed Computing Systems (ICDCS), 2023
Lin Zhang
Longteng Zhang
Shaoshuai Shi
Xiaowen Chu
Yue Liu
OffRL
196
15
0
15 Jun 2023
FMI: Fast and Cheap Message Passing for Serverless Functions
International Conference on Supercomputing (ICS), 2023
Marcin Copik
Roman Böhringer
A. Calotoiu
Torsten Hoefler
232
28
0
15 May 2023
STen: Productive and Efficient Sparsity in PyTorch
Andrei Ivanov
Nikoli Dryden
Tal Ben-Nun
Saleh Ashkboos
Torsten Hoefler
271
7
0
15 Apr 2023
SparDL: Distributed Deep Learning Training with Efficient Sparse Communication
IEEE International Conference on Data Engineering (ICDE), 2023
Minjun Zhao
Yichen Yin
Yuren Mao
Qing Liu
Lu Chen
Yunjun Gao
240
3
0
03 Apr 2023
Sparse-SignSGD with Majority Vote for Communication-Efficient Distributed Learning
International Symposium on Information Theory (ISIT), 2023
Chanho Park
Namyoon Lee
FedML
213
6
0
15 Feb 2023
A Theory of I/O-Efficient Sparse Neural Network Inference
Niels Gleinig
Tal Ben-Nun
Torsten Hoefler
191
0
0
03 Jan 2023
Resource-Constrained Decentralized Federated Learning via Personalized Event-Triggering
Shahryar Zehtabi
Seyyedali Hosseinalipour
Christopher G. Brinton
FedML
321
7
0
23 Nov 2022
L-GreCo: Layerwise-Adaptive Gradient Compression for Efficient and Accurate Deep Learning
Mohammadreza Alimohammadi
I. Markov
Elias Frantar
Dan Alistarh
255
4
0
31 Oct 2022
HammingMesh: A Network Topology for Large-Scale Deep Learning
International Conference for High Performance Computing, Networking, Storage and Analysis (SC), 2022
Torsten Hoefler
Tommaso Bonato
Daniele De Sensi
Salvatore Di Girolamo
Shigang Li
Marco Heddes
Jon Belk
Deepak Goel
Miguel Castro
Steve Scott
3DH
GNN
AI4CE
298
37
0
03 Sep 2022
Reconciling Security and Communication Efficiency in Federated Learning
IEEE Data Engineering Bulletin (DEB), 2022
Karthik Prasad
Sayan Ghosh
Graham Cormode
Ilya Mironov
Ashkan Yousefpour
Pierre Stock
FedML
212
11
0
26 Jul 2022
ByteComp: Revisiting Gradient Compression in Distributed Training
Zhuang Wang
Yanghua Peng
Yibo Zhu
T. Ng
254
2
0
28 May 2022
Parallel Successive Learning for Dynamic Distributed Model Training over Heterogeneous Wireless Networks
IEEE/ACM Transactions on Networking (TON), 2022
Seyyedali Hosseinalipour
Su Wang
Nicolò Michelusi
Vaneet Aggarwal
Christopher G. Brinton
David J. Love
M. Chiang
406
31
0
07 Feb 2022
Near-Optimal Sparse Allreduce for Distributed Deep Learning
ACM SIGPLAN Symposium on Principles & Practice of Parallel Programming (PPoPP), 2022
Shigang Li
Torsten Hoefler
308
66
0
19 Jan 2022
Parallel Algorithms for Adding a Collection of Sparse Matrices
IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum (IPDPS), 2021
Md Taufique Hussain
Guttu Sai Abhishek
A. Buluç
A. Azad
226
5
0
19 Dec 2021
Efficient Strong Scaling Through Burst Parallel Training
Conference on Machine Learning and Systems (MLSys), 2021
S. Park
Joshua Fried
Sunghyun Kim
Mohammad Alizadeh
Adam Belay
GNN
LRM
299
12
0
19 Dec 2021
CGX: Adaptive System Support for Communication-Efficient Deep Learning
I. Markov
Hamidreza Ramezanikebrya
Dan Alistarh
GNN
432
6
0
16 Nov 2021
S2 Reducer: High-Performance Sparse Communication to Accelerate Distributed Deep Learning
Ke-shi Ge
Yongquan Fu
Zhiquan Lai
Xiaoge Deng
Dongsheng Li
134
3
0
05 Oct 2021
Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines
Shigang Li
Torsten Hoefler
AI4CE
LRM
GNN
620
176
0
14 Jul 2021
Flare: Flexible In-Network Allreduce
International Conference for High Performance Computing, Networking, Storage and Analysis (SC), 2021
Daniele De Sensi
Salvatore Di Girolamo
Saleh Ashkboos
Shigang Li
Torsten Hoefler
260
55
0
29 Jun 2021
CD-SGD: Distributed Stochastic Gradient Descent with Compression and Delay Compensation
International Conference on Parallel Processing (ICPP), 2021
Enda Yu
Dezun Dong
Yemao Xu
Shuo Ouyang
Xiangke Liao
178
6
0
21 Jun 2021
An Oracle for Guiding Large-Scale Model/Hybrid Parallel Training of Convolutional Neural Networks
IEEE International Symposium on High-Performance Parallel Distributed Computing (HPDC), 2020
A. Kahira
Truong Thao Nguyen
L. Bautista-Gomez
Ryousei Takano
Rosa M. Badia
Mohamed Wahib
219
14
0
19 Apr 2021
MergeComp: A Compression Scheduler for Scalable Communication-Efficient Distributed Training
Zhuang Wang
X. Wu
T. Ng
GNN
151
4
0
28 Mar 2021
EventGraD: Event-Triggered Communication in Parallel Machine Learning
Neurocomputing (Neurocomputing), 2021
Soumyadip Ghosh
B. Aquino
V. Gupta
FedML
316
9
0
12 Mar 2021
Pufferfish: Communication-efficient Models At No Extra Cost
Conference on Machine Learning and Systems (MLSys), 2021
Hongyi Wang
Saurabh Agarwal
Dimitris Papailiopoulos
181
74
0
05 Mar 2021
Noisy Truncated SGD: Optimization and Generalization
SDM (SDM), 2021
Yingxue Zhou
Xinyan Li
A. Banerjee
265
4
0
26 Feb 2021
DeepReduce: A Sparse-tensor Communication Framework for Distributed Deep Learning
Kelly Kostopoulou
Hang Xu
Aritra Dutta
Xin Li
A. Ntoulas
Panos Kalnis
186
7
0
05 Feb 2021
Towards Scalable Distributed Training of Deep Learning on Public Cloud Clusters
Conference on Machine Learning and Systems (MLSys), 2020
Shaoshuai Shi
Xianhao Zhou
Shutao Song
Xingyao Wang
Zilin Zhu
...
Chenyang Guo
Bo Yang
Zhibo Chen
Yongjian Wu
Xiaowen Chu
GNN
236
59
0
20 Oct 2020
Sparse Communication for Training Deep Networks
Negar Foroutan
Martin Jaggi
FedML
176
19
0
19 Sep 2020
Multi-Stage Hybrid Federated Learning over Large-Scale D2D-Enabled Fog Networks
IEEE/ACM Transactions on Networking (TON), 2020
Seyyedali Hosseinalipour
Sheikh Shams Azam
Christopher G. Brinton
Nicolò Michelusi
Vaneet Aggarwal
David J. Love
H. Dai
383
114
0
18 Jul 2020
Neural Parameter Allocation Search
Bryan A. Plummer
Nikoli Dryden
Julius Frost
Torsten Hoefler
Kate Saenko
582
18
0
18 Jun 2020
O(1) Communication for Distributed SGD through Two-Level Gradient Averaging
IEEE International Conference on Cluster Computing (Cluster), 2020
Subhadeep Bhattacharya
Weikuan Yu
Fahim Chowdhury
FedML
193
3
0
12 Jun 2020
From Federated to Fog Learning: Distributed Machine Learning over Heterogeneous Wireless Networks
Seyyedali Hosseinalipour
Christopher G. Brinton
Vaneet Aggarwal
H. Dai
M. Chiang
FedML
196
12
0
07 Jun 2020
DaSGD: Squeezing SGD Parallelization Performance in Distributed Training Using Delayed Averaging
Q. Zhou
Yawen Zhang
Pengcheng Li
Xiaoyong Liu
Jun Yang
Runsheng Wang
Ru Huang
FedML
177
2
0
31 May 2020
Reducing Communication in Graph Neural Network Training
Alok Tripathy
Katherine Yelick
A. Buluç
GNN
368
120
0
07 May 2020
Breaking (Global) Barriers in Parallel Stochastic Optimization with Wait-Avoiding Group Averaging
IEEE Transactions on Parallel and Distributed Systems (TPDS), 2020
Shigang Li
Tal Ben-Nun
Giorgi Nadiradze
Salvatore Di Girolamo
Nikoli Dryden
Dan Alistarh
Torsten Hoefler
464
15
0
30 Apr 2020
Communication optimization strategies for distributed deep neural network training: A survey
Shuo Ouyang
Dezun Dong
Yemao Xu
Liquan Xiao
362
12
0
06 Mar 2020
Communication-Efficient Decentralized Learning with Sparsification and Adaptive Peer Selection
IEEE International Conference on Distributed Computing Systems (ICDCS), 2020
Zhenheng Tang
Shaoshuai Shi
Xiaowen Chu
FedML
215
71
0
22 Feb 2020
Layer-wise Adaptive Gradient Sparsification for Distributed Deep Learning with Convergence Guarantees
European Conference on Artificial Intelligence (ECAI), 2019
Shaoshuai Shi
Zhenheng Tang
Qiang-qiang Wang
Kaiyong Zhao
Xiaowen Chu
347
28
0
20 Nov 2019
1
2
Next
Page 1 of 2
