Understanding Short-Horizon Bias in Stochastic Meta-Optimization

6 March 2018

Papers citing "Understanding Short-Horizon Bias in Stochastic Meta-Optimization"

42 / 42 papers shown

Title
Reinforcement Teaching Alex Lewandowski Calarina Muslimani Dale Schuurmans Matthew E. Taylor Jun Luo 81 1 0 28 Jan 2025
A New First-Order Meta-Learning Algorithm with Convergence Guarantees El Mahdi Chayti Martin Jaggi 30 1 0 05 Sep 2024
Variance-Reduced Gradient Estimation via Noise-Reuse in Online Evolution Strategies Oscar Li James Harrison Jascha Narain Sohl-Dickstein Virginia Smith Luke Metz 49 5 0 21 Apr 2023
Learning To Optimize Quantum Neural Network Without Gradients Ankit Kulshrestha Xiaoyuan Liu Hayato Ushijima-Mwesigwa Ilya Safro 20 5 0 15 Apr 2023
Achieving Hierarchy-Free Approximation for Bilevel Programs With Equilibrium Constraints Jiayang Li Jiahao Yu Boyi Liu Zhaoran Wang Y. Nie 35 6 0 20 Feb 2023
Online Loss Function Learning Christian Raymond Qi Chen Bing Xue Mengjie Zhang 35 5 0 30 Jan 2023
A Survey of Meta-Reinforcement Learning Jacob Beck Risto Vuorio E. Liu Zheng Xiong L. Zintgraf Chelsea Finn Shimon Whiteson OOD OffRL 37 122 0 19 Jan 2023
Federated Automatic Differentiation Keith Rush Zachary B. Charles Zachary Garrett FedML 34 1 0 18 Jan 2023
Data Distillation: A Survey Noveen Sachdeva Julian McAuley DD 45 73 0 11 Jan 2023
Learning to Optimize with Dynamic Mode Decomposition Petr Simánek Daniel Vasata Pavel Kordík 31 5 0 29 Nov 2022
An Investigation of the Bias-Variance Tradeoff in Meta-Gradients Risto Vuorio Jacob Beck Shimon Whiteson Jakob N. Foerster Gregory Farquhar 24 8 0 22 Sep 2022
A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases James Harrison Luke Metz Jascha Narain Sohl-Dickstein 47 22 0 22 Sep 2022
Learning Symbolic Model-Agnostic Loss Functions via Meta-Learning Christian Raymond Qi Chen Bing Xue Mengjie Zhang FedML 29 11 0 19 Sep 2022
Theseus: A Library for Differentiable Nonlinear Optimization Luis Pineda Taosha Fan Maurizio Monge S. Venkataraman Paloma Sodhi ... Austin S. Wang Stuart Anderson Jing Dong Brandon Amos Mustafa Mukadam 29 76 0 19 Jul 2022
Dataset Distillation using Neural Feature Regression Yongchao Zhou E. Nezhadarya Jimmy Ba DD FedML 44 149 0 01 Jun 2022
Symbolic Learning to Optimize: Towards Interpretability and Scalability Wenqing Zheng Tianlong Chen Ting-Kuei Hu Zhangyang Wang 45 19 0 13 Mar 2022
Amortized Proximal Optimization Juhan Bae Paul Vicol Jeff Z. HaoChen Roger C. Grosse ODL 25 14 0 28 Feb 2022
Tutorial on amortized optimization Brandon Amos OffRL 75 43 0 01 Feb 2022
A Simple Guard for Learned Optimizers Isabeau Prémont-Schwarz Jaroslav Vítkru Jan Feyereisl 49 7 0 28 Jan 2022
Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies Paul Vicol Luke Metz Jascha Narain Sohl-Dickstein 27 67 0 27 Dec 2021
Linear Speedup in Personalized Collaborative Learning El Mahdi Chayti Sai Praneeth Karimireddy Sebastian U. Stich Nicolas Flammarion Martin Jaggi FedML 18 13 0 10 Nov 2021
Gradients are Not All You Need Luke Metz C. Freeman S. Schoenholz Tal Kachman 28 93 0 10 Nov 2021
Scalable One-Pass Optimisation of High-Dimensional Weight-Update Hyperparameters by Implicit Differentiation Ross M. Clarke E. T. Oldewage José Miguel Hernández-Lobato 28 9 0 20 Oct 2021
Online Hyperparameter Meta-Learning with Hypergradient Distillation Haebeom Lee Hayeon Lee Jaewoong Shin Eunho Yang Timothy M. Hospedales Sung Ju Hwang DD 20 2 0 06 Oct 2021
Data Summarization via Bilevel Optimization Zalan Borsos Mojmír Mutný Marco Tagliasacchi Andreas Krause 30 8 0 26 Sep 2021
Bootstrapped Meta-Learning Sebastian Flennerhag Yannick Schroecker Tom Zahavy Hado van Hasselt David Silver Satinder Singh 38 58 0 09 Sep 2021
On Accelerating Distributed Convex Optimizations Kushal Chakrabarti Nirupam Gupta Nikhil Chopra 26 7 0 19 Aug 2021
Analyzing Monotonic Linear Interpolation in Neural Network Loss Landscapes James Lucas Juhan Bae Michael Ruogu Zhang Stanislav Fort R. Zemel Roger C. Grosse MoMe 164 28 0 22 Apr 2021
Self-Tuning Stochastic Optimization with Curvature-Aware Gradient Filtering Ricky T. Q. Chen Dami Choi Lukas Balles David Duvenaud Philipp Hennig ODL 44 6 0 09 Nov 2020
Reverse engineering learned optimizers reveals known and novel mechanisms Niru Maheswaranathan David Sussillo Luke Metz Ruoxi Sun Jascha Narain Sohl-Dickstein 22 21 0 04 Nov 2020
Tasks, stability, architecture, and compute: Training more effective learned optimizers, and using them to train themselves Luke Metz Niru Maheswaranathan C. Freeman Ben Poole Jascha Narain Sohl-Dickstein 33 62 0 23 Sep 2020
Offline Meta-Reinforcement Learning with Advantage Weighting E. Mitchell Rafael Rafailov Xue Bin Peng Sergey Levine Chelsea Finn OffRL 33 103 0 13 Aug 2020
MaxVA: Fast Adaptation of Step Sizes by Maximizing Observed Variance of Gradients Chenfei Zhu Yu Cheng Zhe Gan Furong Huang Jingjing Liu Tom Goldstein ODL 32 2 0 21 Jun 2020
Meta-Learning in Neural Networks: A Survey Timothy M. Hospedales Antreas Antoniou P. Micaelli Amos Storkey OOD 52 1,928 0 11 Apr 2020
Modular Meta-Learning with Shrinkage Yutian Chen A. Friesen Feryal M. P. Behbahani Arnaud Doucet David Budden Matthew W. Hoffman Nando de Freitas KELM OffRL 17 35 0 12 Sep 2019
Lookahead Optimizer: k steps forward, 1 step back Michael Ruogu Zhang James Lucas Geoffrey E. Hinton Jimmy Ba ODL 36 719 0 19 Jul 2019
An Empirical Study of Large-Batch Stochastic Gradient Descent with Structured Covariance Noise Yeming Wen Kevin Luk Maxime Gazeau Guodong Zhang Harris Chan Jimmy Ba ODL 20 22 0 21 Feb 2019
Incremental Few-Shot Learning with Attention Attractor Networks Mengye Ren Renjie Liao Ethan Fetaya R. Zemel CLL 30 181 0 16 Oct 2018
Closing the Generalization Gap of Adaptive Gradient Methods in Training Deep Neural Networks Jinghui Chen Dongruo Zhou Yiqi Tang Ziyan Yang Yuan Cao Quanquan Gu ODL 19 193 0 18 Jun 2018
Meta-Learning for Stochastic Gradient MCMC Wenbo Gong Yingzhen Li José Miguel Hernández-Lobato BDL 24 44 0 12 Jun 2018
Learning to Reweight Examples for Robust Deep Learning Mengye Ren Wenyuan Zeng Binh Yang R. Urtasun OOD NoLa 48 1,410 0 24 Mar 2018
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 359 11,684 0 09 Mar 2017