Optimizing Millions of Hyperparameters by Implicit Differentiation

6 November 2019

Paul Vicol

Papers citing "Optimizing Millions of Hyperparameters by Implicit Differentiation"

50 / 271 papers shown

Title
Towards Gradient-based Bilevel Optimization with Non-convex Followers and Beyond Risheng Liu Yaohua Liu Shangzhi Zeng Jin Zhang 30 80 0 01 Oct 2021
Predicting Flat-Fading Channels via Meta-Learned Closed-Form Linear Filters and Equilibrium Propagation Sangwoo Park Osvaldo Simeone 47 8 0 01 Oct 2021
Data Summarization via Bilevel Optimization Zalan Borsos Mojmír Mutný Marco Tagliasacchi Andreas Krause 30 8 0 26 Sep 2021
Should We Be Pre-training? An Argument for End-task Aware Training as an Alternative Lucio Dery Paul Michel Ameet Talwalkar Graham Neubig CLL 36 35 0 15 Sep 2021
DHA: End-to-End Joint Optimization of Data Augmentation Policy, Hyper-parameter and Architecture Kaichen Zhou Lanqing Hong Shuailiang Hu Fengwei Zhou Binxin Ru Jiashi Feng Zhenguo Li 62 10 0 13 Sep 2021
Is Attention Better Than Matrix Decomposition? Zhengyang Geng Meng-Hao Guo Hongxu Chen Xia Li Ke Wei Zhouchen Lin 62 138 0 09 Sep 2021
Can we globally optimize cross-validation loss? Quasiconvexity in ridge regression William T. Stephenson Zachary Frangella Madeleine Udell Tamara Broderick 27 12 0 19 Jul 2021
Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges B. Bischl Martin Binder Michel Lang Tobias Pielok Jakob Richter ... Theresa Ullmann Marc Becker A. Boulesteix Difan Deng Marius Lindauer 85 455 0 13 Jul 2021
Meta-learning PINN loss functions Apostolos F. Psaros Kenji Kawaguchi George Karniadakis PINN 49 97 0 12 Jul 2021
iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients Miao Zhang Steven W. Su Shirui Pan Xiaojun Chang Ehsan Abbasnejad Reza Haffari 26 68 0 21 Jun 2021
EvoGrad: Efficient Gradient-Based Meta-Learning and Hyperparameter Optimization Ondrej Bohdal Yongxin Yang Timothy M. Hospedales 31 20 0 19 Jun 2021
Meta-Calibration: Learning of Model Calibration Using Differentiable Expected Calibration Error Ondrej Bohdal Yongxin Yang Timothy M. Hospedales UQCV OOD 45 21 0 17 Jun 2021
A Value-Function-based Interior-point Method for Non-convex Bi-level Optimization Risheng Liu Xuan Liu Xiaoming Yuan Shangzhi Zeng Jin Zhang 15 79 0 15 Jun 2021
Last Layer Marginal Likelihood for Invariance Learning Pola Schwobel Martin Jørgensen Sebastian W. Ober Mark van der Wilk BDL UQCV 26 28 0 14 Jun 2021
Proximal Optimal Transport Modeling of Population Dynamics Charlotte Bunne Laetitia Meng-Papaxanthos Andreas Krause Marco Cuturi OT 51 82 0 11 Jun 2021
Stability and Generalization of Bilevel Programming in Hyperparameter Optimization Fan Bao Guoqiang Wu Chongxuan Li Jun Zhu Bo Zhang 30 30 0 08 Jun 2021
Deep Proxy Causal Learning and its Application to Confounded Bandit Policy Evaluation Liyuan Xu Heishiro Kanagawa Arthur Gretton CML 23 35 0 07 Jun 2021
Control-Oriented Model-Based Reinforcement Learning with Implicit Differentiation Evgenii Nikishin Romina Abachi Rishabh Agarwal Pierre-Luc Bacon OffRL 52 35 0 06 Jun 2021
Neural Auto-Curricula Xidong Feng Oliver Slumbers Bo Liu Bo Liu Stephen Marcus McAleer Ying Wen Jun Wang Yaodong Yang 35 1 0 04 Jun 2021
Debiasing a First-order Heuristic for Approximate Bi-level Optimization Valerii Likhosherstov Xingyou Song K. Choromanski Jared Davis Adrian Weller AI4CE 19 5 0 04 Jun 2021
Fair Machine Learning under Limited Demographically Labeled Data Mustafa Safa Ozdayi Murat Kantarcioglu Rishabh K. Iyer FaML 23 3 0 03 Jun 2021
SHINE: SHaring the INverse Estimate from the forward pass for bi-level optimization and implicit models Zaccharie Ramzi Florian Mannel Shaojie Bai Jean-Luc Starck P. Ciuciu Thomas Moreau 37 28 0 01 Jun 2021
Efficient and Modular Implicit Differentiation Mathieu Blondel Quentin Berthet Marco Cuturi Roy Frostig Stephan Hoyer Felipe Llinares-López Fabian Pedregosa Jean-Philippe Vert 27 220 0 31 May 2021
A Gradient Method for Multilevel Optimization Ryo Sato Mirai Tanaka Akiko Takeda 20 18 0 28 May 2021
AutoSampling: Search for Effective Data Sampling Schedules Ming Sun Hao Dou Baopu Li Lei Cui Junjie Yan Wanli Ouyang 42 6 0 28 May 2021
Optimization Induced Equilibrium Networks Xingyu Xie Qiuhao Wang Zenan Ling Xia Li Yisen Wang Guangcan Liu Zhouchen Lin 13 9 0 27 May 2021
Bilevel Programs Meet Deep Learning: A Unifying View on Inference Learning Methods Christopher Zach FedML 16 5 0 15 May 2021
Implicit differentiation for fast hyperparameter selection in non-smooth convex learning Quentin Bertrand Quentin Klopfenstein Mathurin Massias Mathieu Blondel Samuel Vaiter Alexandre Gramfort Joseph Salmon 53 26 0 04 May 2021
Generalization Guarantees for Neural Architecture Search with Train-Validation Split Samet Oymak Mingchen Li Mahdi Soltanolkotabi AI4CE OOD 36 13 0 29 Apr 2021
A contrastive rule for meta-learning Nicolas Zucchet Simon Schug J. Oswald Dominic Zhao João Sacramento MLT 33 19 0 04 Apr 2021
JFB: Jacobian-Free Backpropagation for Implicit Networks Samy Wu Fung Howard Heaton Qiuwei Li Daniel McKenzie Stanley Osher W. Yin FedML 35 84 0 23 Mar 2021
AutoDO: Robust AutoAugment for Biased Data with Label Noise via Scalable Probabilistic Implicit Differentiation Denis A. Gudovskiy Luca Rigazio Shun Ishizaka Kazuki Kozuka Sotaro Tsukizawa NoLa 38 21 0 10 Mar 2021
Label-Imbalanced and Group-Sensitive Classification under Overparameterization Ganesh Ramachandra Kini Orestis Paraskevas Samet Oymak Christos Thrampoulidis 31 93 0 02 Mar 2021
Complex Momentum for Optimization in Games Jonathan Lorraine David Acuna Paul Vicol David Duvenaud 20 9 0 16 Feb 2021
A General Descent Aggregation Framework for Gradient-based Bi-level Optimization Risheng Liu Pan Mu Xiaoming Yuan Shangzhi Zeng Jin Zhang AI4CE 33 36 0 16 Feb 2021
HYDRA: Hypergradient Data Relevance Analysis for Interpreting Deep Neural Networks Yuanyuan Chen Boyang Albert Li Han Yu Pengcheng Wu Chunyan Miao TDI 27 39 0 04 Feb 2021
Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond Risheng Liu Jiaxin Gao Jin Zhang Deyu Meng Zhouchen Lin AI4CE 62 223 0 27 Jan 2021
Cost-Efficient Online Hyperparameter Optimization Jingkang Wang Mengye Ren Ilija Bogunovic Yuwen Xiong R. Urtasun 47 2 0 17 Jan 2021
AutonoML: Towards an Integrated Framework for Autonomous Machine Learning D. Kedziora Katarzyna Musial Bogdan Gabrys 30 16 0 23 Dec 2020
Convergence Properties of Stochastic Hypergradients Riccardo Grazzi Massimiliano Pontil Saverio Salzo 24 26 0 13 Nov 2020
Teaching with Commentaries Aniruddh Raghu M. Raghu Simon Kornblith David Duvenaud Geoffrey E. Hinton 27 24 0 05 Nov 2020
Dataset Meta-Learning from Kernel Ridge-Regression Timothy Nguyen Zhourung Chen Jaehoon Lee DD 36 240 0 30 Oct 2020
Delta-STN: Efficient Bilevel Optimization for Neural Networks using Structured Response Jacobians Juhan Bae Roger C. Grosse 27 24 0 26 Oct 2020
Bilevel Optimization: Convergence Analysis and Enhanced Design Kaiyi Ji Junjie Yang Yingbin Liang 30 249 0 15 Oct 2020
How Out-of-Distribution Data Hurts Semi-Supervised Learning Xujiang Zhao Killamsetty Krishnateja Rishabh K. Iyer Feng Chen 14 3 0 07 Oct 2020
Gradient Descent-Ascent Provably Converges to Strict Local Minmax Equilibria with a Finite Timescale Separation Tanner Fiez Lillian J. Ratliff 22 16 0 30 Sep 2020
SecDD: Efficient and Secure Method for Remotely Training Neural Networks Ilia Sucholutsky Matthias Schonlau 16 15 0 19 Sep 2020
From Federated Learning to Federated Neural Architecture Search: A Survey Hangyu Zhu Haoyu Zhang Yaochu Jin FedML OOD AI4CE 14 148 0 12 Sep 2020
Approximate Cross-Validation with Low-Rank Data in High Dimensions William T. Stephenson Madeleine Udell Tamara Broderick 15 2 0 24 Aug 2020
Efficient hyperparameter optimization by way of PAC-Bayes bound minimization John J. Cherian Andrew G. Taube R. McGibbon Panagiotis Angelikopoulos Guy Blanc M. Snarski D. D. Richman J. L. Klepeis D. Shaw 12 6 0 14 Aug 2020