v1v2 (latest)

UFO-BLO: Unbiased First-Order Bilevel Optimization

5 June 2020

Valerii Likhosherstov

Papers citing "UFO-BLO: Unbiased First-Order Bilevel Optimization"

38 / 38 papers shown

Title
Theoretical Convergence of Multi-Step Model-Agnostic Meta-Learning Kaiyi Ji Junjie Yang Yingbin Liang 83 50 0 18 Feb 2020
Reformer: The Efficient Transformer Nikita Kitaev Lukasz Kaiser Anselm Levskaya VLM 189 2,320 0 13 Jan 2020
Optimizing Millions of Hyperparameters by Implicit Differentiation Jonathan Lorraine Paul Vicol David Duvenaud DD 114 415 0 06 Nov 2019
Generalized Inner Loop Meta-Learning Jaya Kumar Alageshan Brandon Amos A. Verma Phu Mon Htut Artem Molchanov Franziska Meier Douwe Kiela Kyunghyun Cho Soumith Chintala AI4CE 95 159 0 03 Oct 2019
ES-MAML: Simple Hessian-Free Meta Learning Xingyou Song Wenbo Gao Yuxiang Yang K. Choromanski Aldo Pacchiano Yunhao Tang 114 118 0 25 Sep 2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism Mohammad Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 329 1,904 0 17 Sep 2019
Meta-Learning with Implicit Gradients Aravind Rajeswaran Chelsea Finn Sham Kakade Sergey Levine 110 855 0 10 Sep 2019
Introduction to Online Convex Optimization Elad Hazan OffRL 175 1,932 0 07 Sep 2019
On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithms Alireza Fallah Aryan Mokhtari Asuman Ozdaglar 85 225 0 27 Aug 2019
XLNet: Generalized Autoregressive Pretraining for Language Understanding Zhilin Yang Zihang Dai Yiming Yang J. Carbonell Ruslan Salakhutdinov Quoc V. Le AI4CE 232 8,433 0 19 Jun 2019
Energy and Policy Considerations for Deep Learning in NLP Emma Strubell Ananya Ganesh Andrew McCallum 69 2,657 0 05 Jun 2019
Graph Normalizing Flows Jenny Liu Aviral Kumar Jimmy Ba J. Kiros Kevin Swersky BDL GNN AI4CE 72 165 0 30 May 2019
Provable Guarantees for Gradient-Based Meta-Learning M. Khodak Maria-Florina Balcan Ameet Talwalkar FedML 122 150 0 27 Feb 2019
Adversarial Attacks on Graph Neural Networks via Meta Learning Daniel Zügner Stephan Günnemann OOD AAML GNN 126 572 0 22 Feb 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 241 3,730 0 09 Jan 2019
Truncated Back-propagation for Bilevel Optimization Amirreza Shaban Ching-An Cheng Nathan Hatch Byron Boots 101 266 0 25 Oct 2018
Meta-Learning: A Survey Joaquin Vanschoren FedML OOD 66 762 0 08 Oct 2018
DARTS: Differentiable Architecture Search Hanxiao Liu Karen Simonyan Yiming Yang 199 4,355 0 24 Jun 2018
Neural Ordinary Differential Equations T. Chen Yulia Rubanova J. Bettencourt David Duvenaud AI4CE 417 5,111 0 19 Jun 2018
Bilevel Programming for Hyperparameter Optimization and Meta-Learning Luca Franceschi P. Frasconi Saverio Salzo Riccardo Grazzi Massimiliano Pontil 176 728 0 13 Jun 2018
On First-Order Meta-Learning Algorithms Alex Nichol Joshua Achiam John Schulman 230 2,232 0 08 Mar 2018
The Reversible Residual Network: Backpropagation Without Storing Activations Aidan Gomez Mengye Ren R. Urtasun Roger C. Grosse 74 550 0 14 Jul 2017
Unbiasing Truncated Backpropagation Through Time Corentin Tallec Yann Ollivier 56 76 0 23 May 2017
Concrete Dropout Y. Gal Jiri Hron Alex Kendall BDL UQCV 179 592 0 22 May 2017
REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models George Tucker A. Mnih Chris J. Maddison John Lawson Jascha Narain Sohl-Dickstein BDL 216 282 0 21 Mar 2017
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 823 11,909 0 09 Mar 2017
Forward and Reverse Gradient-Based Hyperparameter Optimization Luca Franceschi Michele Donini P. Frasconi Massimiliano Pontil 213 417 0 06 Mar 2017
Categorical Reparameterization with Gumbel-Softmax Eric Jang S. Gu Ben Poole BDL 334 5,364 0 03 Nov 2016
On Differentiating Parameterized Argmin and Argmax Problems with Application to Bi-level Optimization Stephen Gould Basura Fernando A. Cherian Peter Anderson Rodrigo Santa Cruz Edison Guo 51 225 0 19 Jul 2016
Optimization Methods for Large-Scale Machine Learning Léon Bottou Frank E. Curtis J. Nocedal 246 3,216 0 15 Jun 2016
Matching Networks for One Shot Learning Oriol Vinyals Charles Blundell Timothy Lillicrap Koray Kavukcuoglu Daan Wierstra VLM 373 7,323 0 13 Jun 2016
Training Deep Nets with Sublinear Memory Cost Tianqi Chen Bing Xu Chiyuan Zhang Carlos Guestrin 103 1,167 0 21 Apr 2016
Hyperparameter optimization with approximate gradient Fabian Pedregosa 106 449 0 07 Feb 2016
MuProp: Unbiased Backpropagation for Stochastic Neural Networks S. Gu Sergey Levine Ilya Sutskever A. Mnih BDL 49 143 0 16 Nov 2015
Variational Dropout and the Local Reparameterization Trick Diederik P. Kingma Tim Salimans Max Welling BDL 226 1,514 0 08 Jun 2015
Gradient-based Hyperparameter Optimization through Reversible Learning D. Maclaurin David Duvenaud Ryan P. Adams DD 227 945 0 11 Feb 2015
Auto-Encoding Variational Bayes Diederik P. Kingma Max Welling BDL 452 16,929 0 20 Dec 2013
Estimating the Hessian by Back-propagating Curvature James Martens Ilya Sutskever Kevin Swersky 86 80 0 27 Jun 2012