Learning to Optimize Quasi-Newton Methods

v1v2 (latest)

Learning to Optimize Quasi-Newton Methods

11 October 2022

Rumen Dangovski

Jakob N. Foerster

ArXiv (abs)PDF HTML

Papers citing "Learning to Optimize Quasi-Newton Methods"

19 / 19 papers shown

Title
From Learning to Optimize to Learning Optimization Algorithms Camille Castera Peter Ochs 176 1 0 28 May 2024
Amortized Proximal Optimization Juhan Bae Paul Vicol Jeff Z. HaoChen Roger C. Grosse ODL 100 14 0 28 Feb 2022
Step-size Adaptation Using Exponentiated Gradient Updates Ehsan Amid Rohan Anil Christopher Fifty Manfred K. Warmuth 63 9 0 31 Jan 2022
Learning to Optimize: A Primer and A Benchmark Tianlong Chen Xiaohan Chen Wuyang Chen Howard Heaton Jialin Liu Zhangyang Wang W. Yin 252 236 0 23 Mar 2021
Tasks, stability, architecture, and compute: Training more effective learned optimizers, and using them to train themselves Luke Metz Niru Maheswaranathan C. Freeman Ben Poole Jascha Narain Sohl-Dickstein 144 61 0 23 Sep 2020
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning Z. Yao A. Gholami Sheng Shen Mustafa Mustafa Kurt Keutzer Michael W. Mahoney ODL 118 287 0 01 Jun 2020
On the distance between two neural networks and the stability of learning Jeremy Bernstein Arash Vahdat Yisong Yue Xuan Li ODL 252 59 0 09 Feb 2020
First-Order Preconditioning via Hypergradient Descent Theodore H. Moskovitz Rui Wang Janice Lan Sanyam Kapoor Thomas Miconi J. Yosinski Aditya Rawal AI4CE 59 8 0 18 Oct 2019
A Survey of Optimization Methods from a Machine Learning Perspective Shiliang Sun Zehui Cao Han Zhu Jing Zhao 58 559 0 17 Jun 2019
Meta-Curvature Eunbyung Park Junier B. Oliva BDL 73 124 0 09 Feb 2019
Understanding and correcting pathologies in the training of learned optimizers Luke Metz Niru Maheswaranathan Jeremy Nixon C. Freeman Jascha Narain Sohl-Dickstein ODL 89 148 0 24 Oct 2018
Non-convex Optimization for Machine Learning Prateek Jain Purushottam Kar 174 486 0 21 Dec 2017
Large Batch Training of Convolutional Networks Yang You Igor Gitman Boris Ginsburg ODL 141 852 0 13 Aug 2017
Self-Normalizing Neural Networks Günter Klambauer Thomas Unterthiner Andreas Mayr Sepp Hochreiter 489 2,520 0 08 Jun 2017
Online Learning Rate Adaptation with Hypergradient Descent A. G. Baydin R. Cornish David Martínez-Rubio Mark Schmidt Frank Wood ODL 89 250 0 14 Mar 2017
Learning Gradient Descent: Better Generalization and Longer Horizons Kaifeng Lyu Shunhua Jiang Jian Li 107 114 0 10 Mar 2017
Conditional Image Generation with PixelCNN Decoders Aaron van den Oord Nal Kalchbrenner Oriol Vinyals L. Espeholt Alex Graves Koray Kavukcuoglu VLM 223 2,519 0 16 Jun 2016
Learning to learn by gradient descent by gradient descent Marcin Andrychowicz Misha Denil Sergio Gomez Colmenarejo Matthew W. Hoffman David Pfau Tom Schaul Brendan Shillingford Nando de Freitas 132 2,008 0 14 Jun 2016
Optimizing Neural Networks with Kronecker-factored Approximate Curvature James Martens Roger C. Grosse ODL 109 1,024 0 19 Mar 2015

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.