Speeding Up Distributed Machine Learning Using Codes

8 December 2015

Kangwook Lee

Maximilian Lam

Ramtin Pedarsani

Dimitris Papailiopoulos

Kannan Ramchandran

ArXiv PDF HTML

Papers citing "Speeding Up Distributed Machine Learning Using Codes"

50 / 266 papers shown

Title
ErasureHead: Distributed Gradient Descent without Delays Using Approximate Gradient Coding Hongyi Wang Zachary B. Charles Dimitris Papailiopoulos 11 54 0 28 Jan 2019
Heterogeneity-aware Gradient Coding for Straggler Tolerance Yining Qi Song Guo Bin Tang Ruixuan Li Chengjie Li 16 20 0 27 Jan 2019
Distributed Matrix-Vector Multiplication: A Convolutional Coding Approach A. Das A. Ramamoorthy 16 30 0 25 Jan 2019
Fundamental Limits of Approximate Gradient Coding Sinong Wang Jiashang Liu Ness B. Shroff 67 43 0 23 Jan 2019
Straggler Resilient Serverless Computing Based on Polar Codes Burak Bartan Mert Pilanci 13 13 0 21 Jan 2019
Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air Mohammad Mohammadi Amiri Deniz Gunduz 30 53 0 03 Jan 2019
CodedSketch: A Coding Scheme for Distributed Computation of Approximated Matrix Multiplication Tayyebeh Jahani-Nezhad M. Maddah-ali 24 41 0 26 Dec 2018
An Application of Storage-Optimal MatDot Codes for Coded Matrix Multiplication: Fast k-Nearest Neighbors Estimation Utsav Sheth Sanghamitra Dutta Malhar Chaudhari Haewon Jeong Yaoqing Yang J. Kohonen Teemu Roos P. Grover 6 39 0 28 Nov 2018
A Unified Coded Deep Neural Network Training Strategy Based on Generalized PolyDot Codes for Matrix Multiplication Sanghamitra Dutta Ziqian Bai Haewon Jeong Tze Meng Low P. Grover 21 107 0 27 Nov 2018
Distributed Gradient Descent with Coded Partial Gradient Computations Emre Ozfatura S. Ulukus Deniz Gunduz 19 40 0 22 Nov 2018
OverSketch: Approximate Matrix Multiplication for the Cloud Vipul Gupta Ryan Sherman Thomas Courtade Kannan Ramchandran 14 49 0 06 Nov 2018
Erasure coding for distributed matrix multiplication for matrices with bounded entries Li Tang Konstantinos Konstantinidis Tianyi Zhou 18 20 0 06 Nov 2018
Computation Scheduling for Distributed Machine Learning with Straggling Workers Mohammad Mohammadi Amiri Deniz Gunduz FedML 16 2 0 23 Oct 2018
Fault Tolerance in Iterative-Convergent Machine Learning Aurick Qiao Bryon Aragam Bingjing Zhang Eric Xing 26 41 0 17 Oct 2018
A Droplet Approach Based on Raptor Codes for Distributed Computing With Straggling Servers A. Severinson Alexandre Graell i Amat E. Rosnes Francisco Lázaro G. Liva 19 10 0 08 Oct 2018
Anytime Stochastic Gradient Descent: A Time to Hear from all the Workers Nuwan S. Ferdinand S. Draper 13 19 0 06 Oct 2018
PolyShard: Coded Sharding Achieves Linearly Scaling Efficiency and Security Simultaneously Songze Li Mingchao Yu Chien-Sheng Yang A. Avestimehr Sreeram Kannan Pramod Viswanath 24 110 0 27 Sep 2018
Speeding Up Distributed Gradient Descent by Utilizing Non-persistent Stragglers Emre Ozfatura Deniz Gunduz S. Ulukus 21 76 0 07 Aug 2018
On the Fundamental Limits of Coded Data Shuffling for Distributed Machine Learning Adel M. Elmahdy S. Mohajer FedML 19 15 0 11 Jul 2018
Faster Data-access in Large-scale Systems: Network-scale Latency Analysis under General Service-time Distributions Avishek Ghosh Kannan Ramchandran 19 2 0 06 Jul 2018
An Exact Quantized Decentralized Gradient Descent Algorithm Amirhossein Reisizadeh Aryan Mokhtari Hamed Hassani Ramtin Pedarsani 11 123 0 29 Jun 2018
Learning a Code: Machine Learning for Approximate Non-Linear Coded Computation J. Kosaian K. V. Rashmi Shivaram Venkataraman 13 49 0 04 Jun 2018
Lagrange Coded Computing: Optimal Design for Resiliency, Security and Privacy Qian-long Yu Songze Li Netanel Raviv Seyed Mohammadreza Mousavi Kalan Jinhyun So A. Avestimehr 36 382 0 04 Jun 2018
Gradient Coding via the Stochastic Block Model Zachary B. Charles Dimitris Papailiopoulos 16 16 0 25 May 2018
Polynomially Coded Regression: Optimal Straggler Mitigation via Data Encoding Songze Li Seyed Mohammadreza Mousavi Kalan Qian-long Yu Mahdi Soltanolkotabi A. Avestimehr 19 48 0 24 May 2018
Coded FFT and Its Communication Overhead Haewon Jeong Tze Meng Low P. Grover 18 13 0 24 May 2018
Robust Gradient Descent via Moment Encoding with LDPC Codes R. Maity A. S. Rawat A. Mazumdar 11 41 0 22 May 2018
Compressed Coded Distributed Computing Songze Li M. Maddah-ali A. Avestimehr 16 35 0 05 May 2018
Rateless Codes for Near-Perfect Load Balancing in Distributed Matrix-Vector Multiplication Ankur Mallick Malhar Chaudhari Utsav Sheth Ganesh Palanikumar Gauri Joshi 11 141 0 27 Apr 2018
Fundamental Limits of Coded Linear Transform Sinong Wang Jiashang Liu Ness B. Shroff Pengyu Yang 24 38 0 25 Apr 2018
Communication-Aware Scheduling of Serial Tasks for Dispersed Computing Chien-Sheng Yang Ramtin Pedarsani A. Avestimehr 11 40 0 17 Apr 2018
Fundamental Resource Trade-offs for Encoded Distributed Optimization A. Avestimehr Seyed Mohammadreza Mousavi Kalan Mahdi Soltanolkotabi 16 2 0 31 Mar 2018
DRACO: Byzantine-resilient Distributed Training via Redundant Gradients Lingjiao Chen Hongyi Wang Zachary B. Charles Dimitris Papailiopoulos 16 243 0 27 Mar 2018
Stochastic Learning under Random Reshuffling with Constant Step-sizes Bicheng Ying Kun Yuan Stefan Vlaski Ali H. Sayed 18 36 0 21 Mar 2018
Redundancy Techniques for Straggler Mitigation in Distributed Optimization and Learning C. Karakuş Yifan Sun Suhas Diggavi W. Yin 16 52 0 14 Mar 2018
Slow and Stale Gradients Can Win the Race: Error-Runtime Trade-offs in Distributed SGD Sanghamitra Dutta Gauri Joshi Soumyadip Ghosh Parijat Dube P. Nagpurkar 31 193 0 03 Mar 2018
Communication-Computation Efficient Gradient Coding Min Ye Emmanuel Abbe 18 158 0 09 Feb 2018
Coded Sparse Matrix Multiplication Sinong Wang Jiashang Liu Ness B. Shroff 21 123 0 09 Feb 2018
Straggler Mitigation in Distributed Matrix Multiplication: Fundamental Limits and Optimal Coding Qian-long Yu M. Maddah-ali A. Avestimehr 23 39 0 23 Jan 2018
Coded Computing for Distributed Graph Analytics Saurav Prakash Amirhossein Reisizadeh Ramtin Pedarsani A. Avestimehr 12 50 0 17 Jan 2018
Hierarchical Coding for Distributed Computing Hyegyeong Park Kangwook Lee Jy-yong Sohn Changho Suh Jaekyun Moon 18 74 0 15 Jan 2018
Dynamic Heterogeneity-Aware Coded Cooperative Computation at the Edge Yasaman Keshtkarjahromi Yuxuan Xing H. Seferoglu 26 42 0 13 Jan 2018
Near Optimal Coded Data Shuffling for Distributed Learning Mohamed Adel Attia Ravi Tandon FedML 23 30 0 05 Jan 2018
Block-Diagonal and LT Codes for Distributed Computing With Straggling Servers A. Severinson A. Graell i Amat E. Rosnes 16 92 0 21 Dec 2017
Combating Computational Heterogeneity in Large-Scale Distributed Computing via Work Exchange Mohamed Adel Attia Ravi Tandon 19 11 0 22 Nov 2017
Approximate Gradient Coding via Sparse Random Graphs Zachary B. Charles Dimitris Papailiopoulos J. Ellenberg 27 77 0 17 Nov 2017
Straggler Mitigation in Distributed Optimization Through Data Encoding C. Karakuş Yifan Sun Suhas Diggavi W. Yin 18 142 0 14 Nov 2017
Near-Optimal Straggler Mitigation for Distributed Gradient Methods Songze Li Seyed Mohammadreza Mousavi Kalan A. Avestimehr Mahdi Soltanolkotabi 25 97 0 27 Oct 2017
A Sequential Approximation Framework for Coded Distributed Optimization Jingge Zhu Ye Pu Vipul Gupta Claire Tomlin Kannan Ramchandran 17 22 0 24 Oct 2017
Coded Fourier Transform Qian-long Yu M. Maddah-ali A. Avestimehr 27 63 0 17 Oct 2017