Optimizing Millions of Hyperparameters by Implicit Differentiation

6 November 2019

Paul Vicol

Papers citing "Optimizing Millions of Hyperparameters by Implicit Differentiation"

50 / 271 papers shown

Title
Value Function Based Difference-of-Convex Algorithm for Bilevel Hyperparameter Selection Problems Lucy L. Gao J. Ye Haian Yin Shangzhi Zeng Jin Zhang 32 24 0 13 Jun 2022
Dataset Distillation using Neural Feature Regression Yongchao Zhou E. Nezhadarya Jimmy Ba DD FedML 55 151 0 01 Jun 2022
On the complexity of nonsmooth automatic differentiation Jérôme Bolte Ryan Boustany Edouard Pauwels B. Pesquet-Popescu 18 2 0 01 Jun 2022
Automatic differentiation of nonsmooth iterative algorithms Jérôme Bolte Edouard Pauwels Samuel Vaiter 23 22 0 31 May 2022
Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learning Jiahui Gao Renjie Pi Yong Lin Hang Xu Jiacheng Ye Zhiyong Wu Weizhong Zhang Xiaodan Liang Zhenguo Li Lingpeng Kong SyDa VLM 75 45 0 25 May 2022
Revisiting GANs by Best-Response Constraint: Perspective, Methodology, and Application Risheng Liu Jiaxin Gao Xuan Liu Xin-Yue Fan 37 4 0 20 May 2022
Towards Extremely Fast Bilevel Optimization with Self-governed Convergence Guarantees Risheng Liu Xuan Liu Wei-Ting Yao Shangzhi Zeng Jin Zhang 28 3 0 20 May 2022
Set-based Meta-Interpolation for Few-Task Meta-Learning Seanie Lee Bruno Andreis Kenji Kawaguchi Juho Lee Sung Ju Hwang 43 9 0 20 May 2022
Beyond backpropagation: bilevel optimization through implicit differentiation and equilibrium propagation Nicolas Zucchet João Sacramento 36 15 0 06 May 2022
Meta-learning Adaptive Deep Kernel Gaussian Processes for Molecular Property Prediction Wenlin Chen Austin Tripp José Miguel Hernández-Lobato 22 23 0 05 May 2022
Model-Based Deep Learning: On the Intersection of Deep Learning and Optimization Nir Shlezinger Yonina C. Eldar Stephen P. Boyd 30 130 0 05 May 2022
Second-Order Sensitivity Analysis for Bilevel Optimization Robert Dyro Edward Schmerling Nikos Arechiga Marco Pavone 35 3 0 04 May 2022
FedNest: Federated Bilevel, Minimax, and Compositional Optimization Davoud Ataee Tarzanagh Mingchen Li Christos Thrampoulidis Samet Oymak FedML 44 73 0 04 May 2022
Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning Antonio Emanuele Cinà Kathrin Grosse Ambra Demontis Sebastiano Vascon Werner Zellinger Bernhard A. Moser Alina Oprea Battista Biggio Marcello Pelillo Fabio Roli AAML 25 119 0 04 May 2022
Open-Set Semi-Supervised Learning for 3D Point Cloud Understanding Xian Shi Xun Xu Wanyue Zhang Xiatian Zhu Chuan-Sheng Foo Kui Jia 3DPC 33 5 0 02 May 2022
Machines of finite depth: towards a formalization of neural networks Pietro Vertechi M. Bergomi PINN 24 2 0 27 Apr 2022
Relaxing Equivariance Constraints with Non-stationary Continuous Filters Tycho F. A. van der Ouderaa David W. Romero Mark van der Wilk 29 33 0 14 Apr 2022
Data Augmentation for Electrocardiograms Aniruddh Raghu Divya Shanmugam E. Pomerantsev John Guttag Collin M. Stultz 23 18 0 09 Apr 2022
Equivariance Discovery by Learned Parameter-Sharing Raymond A. Yeh Yuan-Ting Hu M. Hasegawa-Johnson Alex Schwing FedML 37 15 0 07 Apr 2022
A Fast and Convergent Proximal Algorithm for Regularized Nonconvex and Nonsmooth Bi-level Optimization Ziyi Chen B. Kailkhura Yi Zhou 26 8 0 30 Mar 2022
Predicting Multi-Antenna Frequency-Selective Channels via Meta-Learned Linear Filters based on Long-Short Term Channel Decomposition Sangwoo Park Osvaldo Simeone 39 4 0 23 Mar 2022
Learning to Generate Synthetic Training Data using Gradient Matching and Implicit Differentiation Dmitry Medvedev A. Dýakonov DD 16 9 0 16 Mar 2022
Training Protocol Matters: Towards Accurate Scene Text Recognition via Training Protocol Searching Xiaojie Chu Yongtao Wang Chunhua Shen Jingdong Chen Wei Chu 24 0 0 13 Mar 2022
A Primal-Dual Approach to Bilevel Optimization with Multiple Inner Minima Daouda Sow Kaiyi Ji Ziwei Guan Yingbin Liang 32 9 0 01 Mar 2022
Amortized Proximal Optimization Juhan Bae Paul Vicol Jeff Z. HaoChen Roger C. Grosse ODL 29 14 0 28 Feb 2022
Learning Invariant Weights in Neural Networks Tycho F. A. van der Ouderaa Mark van der Wilk 29 21 0 25 Feb 2022
Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximations Alexander Immer Tycho F. A. van der Ouderaa Gunnar Rätsch Vincent Fortuin Mark van der Wilk BDL 41 44 0 22 Feb 2022
Supervising the Multi-Fidelity Race of Hyperparameter Configurations Martin Wistuba Arlind Kadra Josif Grabocka 36 14 0 20 Feb 2022
ACORT: A Compact Object Relation Transformer for Parameter Efficient Image Captioning J. Tan Y. Tan C. Chan Joon Huang Chuah VLM ViT 31 15 0 11 Feb 2022
Trust in AI: Interpretability is not necessary or sufficient, while black-box interaction is necessary and sufficient Max W. Shen 27 18 0 10 Feb 2022
Bilevel Optimization with a Lower-level Contraction: Optimal Sample Complexity without Warm-start Riccardo Grazzi Massimiliano Pontil Saverio Salzo 28 12 0 07 Feb 2022
Tutorial on amortized optimization Brandon Amos OffRL 78 43 0 01 Feb 2022
Efficient Hyperparameter Tuning for Large Scale Kernel Ridge Regression Giacomo Meanti Luigi Carratino Ernesto De Vito Lorenzo Rosasco 24 12 0 17 Jan 2022
AutoBalance: Optimized Loss Functions for Imbalanced Data Mingchen Li Xuechen Zhang Christos Thrampoulidis Jiasi Chen Samet Oymak 19 67 0 04 Jan 2022
Efficient Automatic Differentiation of Implicit Functions C. Margossian M. Betancourt 30 2 0 28 Dec 2021
Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies Paul Vicol Luke Metz Jascha Narain Sohl-Dickstein 27 68 0 27 Dec 2021
Lyapunov Exponents for Diversity in Differentiable Games Jonathan Lorraine Paul Vicol Jack Parker-Holder Tal Kachman Luke Metz Jakob N. Foerster 35 7 0 24 Dec 2021
DARTS without a Validation Set: Optimizing the Marginal Likelihood M. Fil Binxin Ru Clare Lyle Y. Gal 29 2 0 24 Dec 2021
Adaptive Incentive Design with Multi-Agent Meta-Gradient Reinforcement Learning Jiachen Yang Ethan Wang Rakshit S. Trivedi T. Zhao H. Zha 32 20 0 20 Dec 2021
Automated Deep Learning: Neural Architecture Search Is Not the End Xuanyi Dong D. Kedziora Katarzyna Musial Bogdan Gabrys 31 26 0 16 Dec 2021
Noether Networks: Meta-Learning Useful Conserved Quantities Ferran Alet Dylan D. Doblar Allan Zhou J. Tenenbaum Kenji Kawaguchi Chelsea Finn 75 27 0 06 Dec 2021
Amortized Implicit Differentiation for Stochastic Bilevel Optimization Michael Arbel Julien Mairal 105 58 0 29 Nov 2021
Learning Augmentation Distributions using Transformed Risk Minimization Evangelos Chatzipantazis Stefanos Pertigkiozoglou Kostas Daniilidis Yan Sun 51 15 0 16 Nov 2021
On Training Implicit Models Zhengyang Geng Xinyu Zhang Shaojie Bai Yisen Wang Zhouchen Lin 72 69 0 09 Nov 2021
Meta-Learning to Improve Pre-Training Aniruddh Raghu Jonathan Lorraine Simon Kornblith Matthew B. A. McDermott David Duvenaud 27 30 0 02 Nov 2021
Scalable One-Pass Optimisation of High-Dimensional Weight-Update Hyperparameters by Implicit Differentiation Ross M. Clarke E. T. Oldewage José Miguel Hernández-Lobato 31 9 0 20 Oct 2021
On the Convergence Theory for Hessian-Free Bilevel Algorithms Daouda Sow Kaiyi Ji Yingbin Liang 28 28 0 13 Oct 2021
Value-Function-based Sequential Minimization for Bi-level Optimization Risheng Liu Xuan Liu Shangzhi Zeng Jin Zhang Yixuan Zhang 40 30 0 11 Oct 2021
Online Hyperparameter Meta-Learning with Hypergradient Distillation Haebeom Lee Hayeon Lee Jaewoong Shin Eunho Yang Timothy M. Hospedales Sung Ju Hwang DD 42 2 0 06 Oct 2021
Inexact bilevel stochastic gradient methods for constrained and unconstrained lower-level problems Tommaso Giovannelli G. Kent Luis Nunes Vicente 33 12 0 01 Oct 2021