v1v2v3 (latest)

To Raise or Not To Raise: The Autonomous Learning Rate Question

16 June 2021

ArXiv (abs)PDF HTML Github (2★)

Papers citing "To Raise or Not To Raise: The Autonomous Learning Rate Question"

31 / 31 papers shown

Title
Scaled-YOLOv4: Scaling Cross Stage Partial Network Chien-Yao Wang Alexey Bochkovskiy H. Liao ObjD 61 1,148 0 16 Nov 2020
YOLOv4: Optimal Speed and Accuracy of Object Detection Alexey Bochkovskiy Chien-Yao Wang H. Liao VLM ObjD 164 12,299 0 23 Apr 2020
CSPNet: A New Backbone that can Enhance Learning Capability of CNN Chien-Yao Wang H. Liao I-Hau Yeh Yueh-hua Wu Ping-Yang Chen J. Hsieh 90 3,101 0 27 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 459 20,298 0 23 Oct 2019
FastEstimator: A Deep Learning Library for Fast Prototyping and Productization Xiaomeng Dong Junpyo Hong Hsi-Ming Chang Michael Potter Aritra Chowdhury ... Rajesh Tamada Gaurav Kumar Caroline Favart V. R. Saripalli Gopal Avinash 23 2 0 07 Oct 2019
Verified Uncertainty Calibration Ananya Kumar Percy Liang Tengyu Ma 170 357 0 23 Sep 2019
Learning an Adaptive Learning Rate Schedule Zhen Xu Andrew M. Dai Jonas Kemp Luke Metz 62 62 0 20 Sep 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 674 24,541 0 26 Jul 2019
Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates Sharan Vaswani Aaron Mishkin I. Laradji Mark Schmidt Gauthier Gidel Simon Lacoste-Julien ODL 86 210 0 24 May 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,114 0 11 Oct 2018
DARTS: Differentiable Architecture Search Hanxiao Liu Karen Simonyan Yiming Yang 204 4,366 0 24 Jun 2018
Understanding Short-Horizon Bias in Stochastic Meta-Optimization Yuhuai Wu Mengye Ren Renjie Liao Roger C. Grosse 99 138 0 06 Mar 2018
Efficient Neural Architecture Search via Parameter Sharing Hieu H. Pham M. Guan Barret Zoph Quoc V. Le J. Dean 115 2,766 0 09 Feb 2018
Neural Optimizer Search with Reinforcement Learning Irwan Bello Barret Zoph Vijay Vasudevan Quoc V. Le ODL 62 386 0 21 Sep 2017
Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates L. Smith Nicholay Topin AI4CE 86 520 0 23 Aug 2017
Focal Loss for Dense Object Detection Nayeon Lee Priya Goyal Ross B. Girshick Kaiming He Piotr Dollár ObjD 124 2,998 0 07 Aug 2017
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour Priya Goyal Piotr Dollár Ross B. Girshick P. Noordhuis Lukasz Wesolowski Aapo Kyrola Andrew Tulloch Yangqing Jia Kaiming He 3DH 128 3,685 0 08 Jun 2017
Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics Alex Kendall Y. Gal R. Cipolla 3DH 272 3,135 0 19 May 2017
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 475 5,378 0 05 Nov 2016
Densely Connected Convolutional Networks Gao Huang Zhuang Liu Laurens van der Maaten Kilian Q. Weinberger PINN 3DV 775 36,861 0 25 Aug 2016
SGDR: Stochastic Gradient Descent with Warm Restarts I. Loshchilov Frank Hutter ODL 347 8,169 0 13 Aug 2016
Wide Residual Networks Sergey Zagoruyko N. Komodakis 351 7,995 0 23 May 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 194,322 0 10 Dec 2015
Rethinking the Inception Architecture for Computer Vision Christian Szegedy Vincent Vanhoucke Sergey Ioffe Jonathon Shlens Z. Wojna 3DV BDL 886 27,412 0 02 Dec 2015
U-Net: Convolutional Networks for Biomedical Image Segmentation Olaf Ronneberger Philipp Fischer Thomas Brox SSeg 3DV 1.9K 77,341 0 18 May 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 465 43,328 0 11 Feb 2015
Probabilistic Line Searches for Stochastic Optimization Maren Mahsereci Philipp Hennig ODL 68 126 0 10 Feb 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 2.0K 150,260 0 22 Dec 2014
Explaining and Harnessing Adversarial Examples Ian Goodfellow Jonathon Shlens Christian Szegedy AAML GAN 282 19,107 0 20 Dec 2014
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 1.7K 100,479 0 04 Sep 2014
On the Properties of Neural Machine Translation: Encoder-Decoder Approaches Kyunghyun Cho B. V. Merrienboer Dzmitry Bahdanau Yoshua Bengio AI4CE AIMat 257 6,784 0 03 Sep 2014