Don't Decay the Learning Rate, Increase the Batch Size

v1v2 (latest)

Don't Decay the Learning Rate, Increase the Batch Size

1 November 2017

Samuel L. Smith

Pieter-Jan Kindermans

ArXiv (abs)PDF HTML

Papers citing "Don't Decay the Learning Rate, Increase the Batch Size"

4 / 454 papers shown

Title
Scale out for large minibatch SGD: Residual network training on ImageNet-1K with improved accuracy and reduced time to train V. Codreanu Damian Podareanu V. Saletore 72 55 0 12 Nov 2017
Efficient Training of Convolutional Neural Nets on Large Distributed Systems Sameer Kumar D. Sreedhar Vaibhav Saxena Yogish Sabharwal Ashish Verma 67 4 0 02 Nov 2017
A Bayesian Perspective on Generalization and Stochastic Gradient Descent Samuel L. Smith Quoc V. Le BDL 126 253 0 17 Oct 2017
Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates L. Smith Nicholay Topin AI4CE 139 518 0 23 Aug 2017