Reverse engineering learned optimizers reveals known and novel
mechanisms

Reverse engineering learned optimizers reveals known and novel mechanisms

4 November 2020

Niru Maheswaranathan

Jascha Narain Sohl-Dickstein

Papers citing "Reverse engineering learned optimizers reveals known and novel mechanisms"

18 / 18 papers shown

Title
Improving Learning to Optimize Using Parameter Symmetries Guy Zamir Aryan Dokania B. Zhao Rose Yu 22 0 0 21 Apr 2025
Investigation into the Training Dynamics of Learned Optimizers Jan Sobotka Petr Simánek Daniel Vasata 28 0 0 12 Dec 2023
Improving physics-informed neural networks with meta-learned optimization Alexander Bihlo PINN 36 18 0 13 Mar 2023
Learning to Optimize for Reinforcement Learning Qingfeng Lan Rupam Mahmood Shuicheng Yan Zhongwen Xu OffRL 26 6 0 03 Feb 2023
Transformer-Based Learned Optimization Erik Gartner Luke Metz Mykhaylo Andriluka C. Freeman C. Sminchisescu 18 11 0 02 Dec 2022
VeLO: Training Versatile Learned Optimizers by Scaling Up Luke Metz James Harrison C. Freeman Amil Merchant Lucas Beyer ... Naman Agrawal Ben Poole Igor Mordatch Adam Roberts Jascha Narain Sohl-Dickstein 29 60 0 17 Nov 2022
A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases James Harrison Luke Metz Jascha Narain Sohl-Dickstein 47 22 0 22 Sep 2022
Beyond accuracy: generalization properties of bio-plausible temporal credit assignment rules Yuhan Helena Liu Arna Ghosh Blake A. Richards E. Shea-Brown Guillaume Lajoie 26 9 0 02 Jun 2022
Practical tradeoffs between memory, compute, and performance in learned optimizers Luke Metz C. Freeman James Harrison Niru Maheswaranathan Jascha Narain Sohl-Dickstein 33 32 0 22 Mar 2022
Symbolic Learning to Optimize: Towards Interpretability and Scalability Wenqing Zheng Tianlong Chen Ting-Kuei Hu Zhangyang Wang 45 19 0 13 Mar 2022
Graph-based Neural Acceleration for Nonnegative Matrix Factorization Jens Sjölund Maria Bånkestad GNN 40 9 0 01 Feb 2022
Tutorial on amortized optimization Brandon Amos OffRL 75 43 0 01 Feb 2022
A Simple Guard for Learned Optimizers Isabeau Prémont-Schwarz Jaroslav Vítkru Jan Feyereisl 49 7 0 28 Jan 2022
Learn2Hop: Learned Optimization on Rough Landscapes Amil Merchant Luke Metz S. Schoenholz E. D. Cubuk 23 16 0 20 Jul 2021
Meta-Learning Bidirectional Update Rules Mark Sandler Max Vladymyrov A. Zhmoginov Nolan Miller Andrew Jackson T. Madams Blaise Agüera y Arcas 19 15 0 10 Apr 2021
Learning to Optimize: A Primer and A Benchmark Tianlong Chen Xiaohan Chen Wuyang Chen Howard Heaton Jialin Liu Zhangyang Wang W. Yin 40 225 0 23 Mar 2021
An Operator Theoretic Approach for Analyzing Sequence Neural Networks Ilana D Naiman Omri Azencot 19 10 0 15 Feb 2021
A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights Weijie Su Stephen P. Boyd Emmanuel J. Candes 108 1,154 0 04 Mar 2015