Adaptive Hierarchical Hyper-gradient Descent

v1v2v3 (latest)

Adaptive Hierarchical Hyper-gradient Descent

17 August 2020

Junbin Gao

ArXiv (abs)PDF HTML

Papers citing "Adaptive Hierarchical Hyper-gradient Descent"

13 / 13 papers shown

Title
Optimization for deep learning: theory and algorithms Ruoyu Sun ODL 124 169 0 19 Dec 2019
Lookahead Optimizer: k steps forward, 1 step back Michael Ruogu Zhang James Lucas Geoffrey E. Hinton Jimmy Ba ODL 157 732 0 19 Jul 2019
The Step Decay Schedule: A Near Optimal, Geometrically Decaying Learning Rate Procedure For Least Squares Rong Ge Sham Kakade Rahul Kidambi Praneeth Netrapalli 100 155 0 29 Apr 2019
Adaptive Gradient Methods with Dynamic Bound of Learning Rate Liangchen Luo Yuanhao Xiong Yan Liu Xu Sun ODL 80 602 0 26 Feb 2019
Improved Regularization of Convolutional Neural Networks with Cutout Terrance Devries Graham W. Taylor 129 3,774 0 15 Aug 2017
Learned Optimizers that Scale and Generalize Olga Wichrowska Niru Maheswaranathan Matthew W. Hoffman Sergio Gomez Colmenarejo Misha Denil Nando de Freitas Jascha Narain Sohl-Dickstein AI4CE 62 284 0 14 Mar 2017
Online Learning Rate Adaptation with Hypergradient Descent A. G. Baydin R. Cornish David Martínez-Rubio Mark Schmidt Frank Wood ODL 83 250 0 14 Mar 2017
Learning Gradient Descent: Better Generalization and Longer Horizons Kaifeng Lyu Shunhua Jiang Jian Li 79 114 0 10 Mar 2017
Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition Hamed Karimi J. Nutini Mark Schmidt 280 1,221 0 16 Aug 2016
Learning to learn by gradient descent by gradient descent Marcin Andrychowicz Misha Denil Sergio Gomez Colmenarejo Matthew W. Hoffman David Pfau Tom Schaul Brendan Shillingford Nando de Freitas 124 2,008 0 14 Jun 2016
Automatic differentiation in machine learning: a survey A. G. Baydin Barak A. Pearlmutter Alexey Radul J. Siskind PINN AI4CE ODL 168 2,816 0 20 Feb 2015
Gradient-based Hyperparameter Optimization through Reversible Learning D. Maclaurin David Duvenaud Ryan P. Adams DD 227 946 0 11 Feb 2015
Convolutional Neural Networks at Constrained Time Cost Kaiming He Jian Sun 3DV 88 1,292 0 04 Dec 2014

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.