Distributed Deep Learning Using Synchronous Stochastic Gradient Descent

22 February 2016

Papers citing "Distributed Deep Learning Using Synchronous Stochastic Gradient Descent"

33 / 33 papers shown

Title
Optimus-CC: Efficient Large NLP Model Training with 3D Parallelism Aware Communication Compression Jaeyong Song Jinkyu Yim Jaewon Jung Hongsun Jang H. Kim Youngsok Kim Jinho Lee GNN 34 25 0 24 Jan 2023
Byzantine Fault Tolerance in Distributed Machine Learning : a Survey Djamila Bouhata Hamouma Moumen Moumen Hamouma Ahcène Bounceur AI4CE 31 7 0 05 May 2022
Partitioning sparse deep neural networks for scalable training and inference G. Demirci Hakan Ferhatosmanoglu 25 11 0 23 Apr 2021
Privacy and Trust Redefined in Federated Machine Learning Pavlos Papadopoulos Will Abramson A. Hall Nikolaos Pitropakis William J. Buchanan 33 42 0 29 Mar 2021
GradPIM: A Practical Processing-in-DRAM Architecture for Gradient Descent Heesu Kim Hanmin Park Taehyun Kim Kwanheum Cho Eojin Lee Soojung Ryu Hyuk-Jae Lee Kiyoung Choi Jinho Lee 24 36 0 15 Feb 2021
Sparse Communication for Training Deep Networks Negar Foroutan Martin Jaggi FedML 30 16 0 19 Sep 2020
Review: Deep Learning in Electron Microscopy Jeffrey M. Ede 44 79 0 17 Sep 2020
Enabling Compute-Communication Overlap in Distributed Deep Learning Training Platforms Saeed Rashidi Matthew Denton Srinivas Sridharan Sudarshan Srinivasan Amoghavarsha Suresh Jade Nie T. Krishna 34 45 0 30 Jun 2020
A Distributed Trust Framework for Privacy-Preserving Machine Learning Will Abramson A. Hall Pavlos Papadopoulos Nikolaos Pitropakis William J. Buchanan 14 20 0 03 Jun 2020
Array Languages Make Neural Networks Fast Artjoms Šinkarovs Hans-Nikolai Vießmann S. Scholz 25 5 0 11 Dec 2019
Optimal checkpointing for heterogeneous chains: how to train deep neural networks with limited memory Julien Herrmann Olivier Beaumont Lionel Eyraud-Dubois J. Herrmann Alexis Joly Alena Shilova BDL 31 29 0 27 Nov 2019
Optimizing Multi-GPU Parallelization Strategies for Deep Learning Training Saptadeep Pal Eiman Ebrahimi A. Zulfiqar Yaosheng Fu Victor Zhang Szymon Migacz D. Nellans Puneet Gupta 34 55 0 30 Jul 2019
Quasi-Newton Methods for Machine Learning: Forget the Past, Just Sample A. Berahas Majid Jahani Peter Richtárik Martin Takávc 24 40 0 28 Jan 2019
A Distributed Synchronous SGD Algorithm with Global Top- $k$ Sparsification for Low Bandwidth Networks S. Shi Qiang-qiang Wang Kaiyong Zhao Zhenheng Tang Yuxin Wang Xiang Huang Xiaowen Chu 40 135 0 14 Jan 2019
Bandwidth Reduction using Importance Weighted Pruning on Ring AllReduce Zehua Cheng Zhenghua Xu 22 8 0 06 Jan 2019
No Peek: A Survey of private distributed deep learning Praneeth Vepakomma Tristan Swedish Ramesh Raskar O. Gupta Abhimanyu Dubey SyDa FedML 30 100 0 08 Dec 2018
Mini-batch Serialization: CNN Training with Inter-layer Data Reuse Sangkug Lym Armand Behroozi W. Wen Ge Li Yongkee Kwon M. Erez 17 25 0 30 Sep 2018
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design Ningning Ma Xiangyu Zhang Haitao Zheng Jian Sun 51 4,931 0 30 Jul 2018
Backdrop: Stochastic Backpropagation Siavash Golkar Kyle Cranmer 41 2 0 04 Jun 2018
Revisiting Small Batch Training for Deep Neural Networks Dominic Masters Carlo Luschi ODL 37 662 0 20 Apr 2018
On Scale-out Deep Learning Training for Cloud and HPC Srinivas Sridharan K. Vaidyanathan Dhiraj D. Kalamkar Dipankar Das Mikhail E. Smorkalov ... Dheevatsa Mudigere Naveen Mellempudi Sasikanth Avancha Bharat Kaul Pradeep Dubey BDL 26 30 0 24 Jan 2018
Integrated Model, Batch and Domain Parallelism in Training Neural Networks A. Gholami A. Azad Peter H. Jin Kurt Keutzer A. Buluç 26 82 0 12 Dec 2017
AdaBatch: Adaptive Batch Sizes for Training Deep Neural Networks Aditya Devarakonda Maxim Naumov M. Garland ODL 24 136 0 06 Dec 2017
Performance Modeling and Evaluation of Distributed Deep Learning Frameworks on GPUs S. Shi Qiang-qiang Wang Xiaowen Chu 37 110 0 16 Nov 2017
SparCE: Sparsity aware General Purpose Core Extensions to Accelerate Deep Neural Networks Sanchari Sen Shubham Jain Swagath Venkataramani A. Raghunathan 24 30 0 07 Nov 2017
What does fault tolerant Deep Learning need from MPI? Vinay C. Amatya Abhinav Vishnu Charles Siegel J. Daily 28 19 0 11 Sep 2017
Spectral Norm Regularization for Improving the Generalizability of Deep Learning Yuichi Yoshida Takeru Miyato 46 325 0 31 May 2017
Train longer, generalize better: closing the generalization gap in large batch training of neural networks Elad Hoffer Itay Hubara Daniel Soudry ODL 44 795 0 24 May 2017
Exploring the Design Space of Deep Convolutional Neural Networks at Large Scale F. Iandola 3DV 26 18 0 20 Dec 2016
How to scale distributed deep learning? Peter H. Jin Qiaochu Yuan F. Iandola Kurt Keutzer 3DH 27 136 0 14 Nov 2016
Asynchronous Stochastic Gradient Descent with Delay Compensation Shuxin Zheng Qi Meng Taifeng Wang Wei Chen Nenghai Yu Zhiming Ma Tie-Yan Liu 32 312 0 27 Sep 2016
Benchmarking State-of-the-Art Deep Learning Software Tools S. Shi Qiang-qiang Wang Pengfei Xu Xiaowen Chu BDL 19 327 0 25 Aug 2016
Boda-RTC: Productive Generation of Portable, Efficient Code for Convolutional Neural Networks on Mobile Computing Platforms Matthew W. Moskewicz F. Iandola Kurt Keutzer 14 8 0 01 Jun 2016