Reversible designs for extreme memory cost reduction of CNN training

24 October 2019

Papers citing "Reversible designs for extreme memory cost reduction of CNN training"

20 / 20 papers shown

Title
An Empirical Model of Large-Batch Training Sam McCandlish Jared Kaplan Dario Amodei OpenAI Dota Team 74 280 0 14 Dec 2018
Measuring the Effects of Data Parallelism on Neural Network Training Christopher J. Shallue Jaehoon Lee J. Antognini J. Mamou J. Ketterling Yao Wang 100 409 0 08 Nov 2018
Invertible Residual Networks Jens Behrmann Will Grathwohl Ricky T. Q. Chen David Duvenaud J. Jacobsen UQCV TPM 159 624 0 02 Nov 2018
Excessive Invariance Causes Adversarial Vulnerability J. Jacobsen Jens Behrmann R. Zemel Matthias Bethge AAML 81 167 0 01 Nov 2018
Glow: Generative Flow with Invertible 1x1 Convolutions Diederik P. Kingma Prafulla Dhariwal BDL DRL 305 3,144 0 09 Jul 2018
Neural Ordinary Differential Equations T. Chen Yulia Rubanova J. Bettencourt David Duvenaud AI4CE 452 5,168 0 19 Jun 2018
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks Jonathan Frankle Michael Carbin 274 3,488 0 09 Mar 2018
i-RevNet: Deep Invertible Networks J. Jacobsen A. Smeulders Edouard Oyallon 85 333 0 20 Feb 2018
Training and Inference with Integers in Deep Neural Networks Shuang Wu Guoqi Li F. Chen Luping Shi MQ 69 391 0 13 Feb 2018
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference Benoit Jacob S. Kligys Bo Chen Menglong Zhu Matthew Tang Andrew G. Howard Hartwig Adam Dmitry Kalenichenko MQ 167 3,143 0 15 Dec 2017
In-Place Activated BatchNorm for Memory-Optimized Training of DNNs Samuel Rota Buló Lorenzo Porzi Peter Kontschieder 77 357 0 07 Dec 2017
Mixed Precision Training Paulius Micikevicius Sharan Narang Jonah Alben G. Diamos Erich Elsen ... Boris Ginsburg Michael Houston Oleksii Kuchaiev Ganesh Venkatesh Hao Wu 178 1,805 0 10 Oct 2017
The Reversible Residual Network: Backpropagation Without Storing Activations Aidan Gomez Mengye Ren R. Urtasun Roger C. Grosse 79 551 0 14 Jul 2017
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 1.2K 20,900 0 17 Apr 2017
Faster Eigenvector Computation via Shift-and-Invert Preconditioning Dan Garber Laurent Dinh Chi Jin Jascha Narain Sohl-Dickstein Samy Bengio Praneeth Netrapalli Aaron Sidford 277 3,722 0 26 May 2016
Training Deep Nets with Sublinear Memory Cost Tianqi Chen Bing Xu Chiyuan Zhang Carlos Guestrin 108 1,174 0 21 Apr 2016
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size F. Iandola Song Han Matthew W. Moskewicz Khalid Ashraf W. Dally Kurt Keutzer 163 7,501 0 24 Feb 2016
Variational Inference with Normalizing Flows Danilo Jimenez Rezende S. Mohamed DRL BDL 322 4,197 0 21 May 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 2.1K 150,364 0 22 Dec 2014
NICE: Non-linear Independent Components Estimation Laurent Dinh David M. Krueger Yoshua Bengio DRL BDL 131 2,269 0 30 Oct 2014