ADA-GP: Accelerating DNN Training By Adaptive Gradient Prediction

v1v2 (latest)

ADA-GP: Accelerating DNN Training By Adaptive Gradient Prediction

22 May 2023

Shantanu Mandal

ArXiv (abs)PDF HTML

Papers citing "ADA-GP: Accelerating DNN Training By Adaptive Gradient Prediction"

18 / 18 papers shown

Title
MERCURY: Accelerating DNN Training By Exploiting Input Similarity Vahid Janfaza Kevin Weston Moein Razavi Shantanu Mandal Farabi Mahmud Alex Hilty A. Muzahid 55 5 0 28 Oct 2021
DAPPLE: A Pipelined Data Parallel Approach for Training Large Models Shiqing Fan Yi Rong Chen Meng Zongyan Cao Siyu Wang ... Jun Yang Lixue Xia Lansong Diao Xiaoyong Liu Wei Lin 92 239 0 02 Jul 2020
Taming Unbalanced Training Workloads in Deep Learning with Partial Collective Operations Shigang Li Tal Ben-Nun Salvatore Di Girolamo Dan Alistarh Torsten Hoefler 109 58 0 12 Aug 2019
Fully Decoupled Neural Network Learning Using Delayed Gradients Huiping Zhuang Yi Wang Qinglai Liu Shuai Zhang Zhiping Lin FedML 73 31 0 21 Jun 2019
Decoupled Greedy Learning of CNNs Eugene Belilovsky Michael Eickenberg Edouard Oyallon 57 117 0 23 Jan 2019
Training Neural Networks Using Features Replay Zhouyuan Huo Bin Gu Heng-Chiao Huang 69 70 0 12 Jul 2018
YOLOv3: An Incremental Improvement Joseph Redmon Ali Farhadi ObjD 128 21,482 0 08 Apr 2018
Horovod: fast and easy distributed deep learning in TensorFlow Alexander Sergeev Mike Del Balso 100 1,221 0 15 Feb 2018
Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm David Silver Thomas Hubert Julian Schrittwieser Ioannis Antonoglou Matthew Lai ... D. Kumaran T. Graepel Timothy Lillicrap Karen Simonyan Demis Hassabis 153 1,782 0 05 Dec 2017
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour Priya Goyal Piotr Dollár Ross B. Girshick P. Noordhuis Lukasz Wesolowski Aapo Kyrola Andrew Tulloch Yangqing Jia Kaiming He 3DH 128 3,685 0 08 Jun 2017
Efficient Processing of Deep Neural Networks: A Tutorial and Survey Vivienne Sze Yu-hsin Chen Tien-Ju Yang J. Emer AAML 3DV 120 3,026 0 27 Mar 2017
Understanding Synthetic Gradients and Decoupled Neural Interfaces Wojciech M. Czarnecki G. Swirszcz Max Jaderberg Simon Osindero Oriol Vinyals Koray Kavukcuoglu 71 82 0 01 Mar 2017
Direct Feedback Alignment Provides Learning in Deep Neural Networks Arild Nøkland ODL 97 459 0 06 Sep 2016
Densely Connected Convolutional Networks Gao Huang Zhuang Liu Laurens van der Maaten Kilian Q. Weinberger PINN 3DV 786 36,881 0 25 Aug 2016
Decoupled Neural Interfaces using Synthetic Gradients Max Jaderberg Wojciech M. Czarnecki Simon Osindero Oriol Vinyals Alex Graves David Silver Koray Kavukcuoglu 85 358 0 18 Aug 2016
Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning Christian Szegedy Sergey Ioffe Vincent Vanhoucke Alexander A. Alemi 381 14,263 0 23 Feb 2016
Rethinking the Inception Architecture for Computer Vision Christian Szegedy Vincent Vanhoucke Sergey Ioffe Jonathon Shlens Z. Wojna 3DV BDL 886 27,416 0 02 Dec 2015
One weird trick for parallelizing convolutional neural networks A. Krizhevsky GNN 93 1,303 0 23 Apr 2014

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.