Optimizing Millions of Hyperparameters by Implicit Differentiation

6 November 2019

Paul Vicol

Papers citing "Optimizing Millions of Hyperparameters by Implicit Differentiation"

50 / 271 papers shown

Title
Generalizable and Stable Finetuning of Pretrained Language Models on Low-Resource Texts Sai Ashish Somayajula Youwei Liang Abhishek Singh Li Zhang Pengtao Xie 32 2 0 19 Mar 2024
BiLoRA: A Bi-level Optimization Framework for Overfitting-Resilient Low-Rank Adaptation of Large Pre-trained Models Rushi Qiang Ruiyi Zhang Pengtao Xie AI4CE 30 8 0 19 Mar 2024
Nonsmooth Implicit Differentiation: Deterministic and Stochastic Convergence Rates Riccardo Grazzi Massimiliano Pontil Saverio Salzo 49 1 0 18 Mar 2024
One Category One Prompt: Dataset Distillation using Diffusion Models Ali Abbasi Ashkan Shahbazi Hamed Pirsiavash Soheil Kolouri DiffM DD 40 3 0 11 Mar 2024
Better than classical? The subtle art of benchmarking quantum machine learning models Joseph Bowles Shahnawaz Ahmed Maria Schuld 47 67 0 11 Mar 2024
Tune without Validation: Searching for Learning Rate and Weight Decay on Training Sets Lorenzo Brigato Stavroula Mougiakakou 45 0 0 08 Mar 2024
Fast and Efficient Local Search for Genetic Programming Based Loss Function Learning Christian Raymond Qi Chen Bing Xue Mengjie Zhang 49 2 0 01 Mar 2024
A Framework for Bilevel Optimization on Riemannian Manifolds Andi Han Bamdev Mishra Pratik Jawanpuria Akiko Takeda 21 5 0 06 Feb 2024
Glocal Hypergradient Estimation with Koopman Operator Ryuichiro Hataya Yoshinobu Kawahara 44 2 0 05 Feb 2024
Enhancing Molecular Property Prediction with Auxiliary Learning and Task-Specific Adaptation Vishal Dey Xia Ning AAML AI4CE 29 0 0 29 Jan 2024
Constrained Bi-Level Optimization: Proximal Lagrangian Value function Approach and Hessian-free Algorithm Wei-Ting Yao Chengming Yu Shangzhi Zeng Jin Zhang 23 13 0 29 Jan 2024
Importance-Aware Adaptive Dataset Distillation Guang Li Ren Togo Takahiro Ogawa Miki Haseyama DD 30 6 0 29 Jan 2024
Crafter: Facial Feature Crafting against Inversion-based Identity Theft on Deep Models Shiming Wang Zhe Ji Liyao Xiang Hao Zhang Xinbing Wang Cheng Zhou Bo-wen Li 24 4 0 14 Jan 2024
TaskMet: Task-Driven Metric Learning for Model Learning Dishank Bansal Ricky T. Q. Chen Mustafa Mukadam Brandon Amos FedML 28 10 0 08 Dec 2023
Using Large Language Models for Hyperparameter Optimization Michael Ruogu Zhang Nishkrit Desai Juhan Bae Jonathan Lorraine Jimmy Ba 36 51 0 07 Dec 2023
Adaptive Instrument Design for Indirect Experiments Yash Chandak Shiv Shankar Vasilis Syrgkanis Emma Brunskill 40 4 0 05 Dec 2023
IMMA: Immunizing text-to-image Models against Malicious Adaptation Yijia Zheng Raymond A. Yeh 53 8 0 30 Nov 2023
Gradient-based bilevel optimization for multi-penalty Ridge regression through matrix differential calculus Gabriele Maroni Loris Cannelli Dario Piga 22 0 0 23 Nov 2023
Embarassingly Simple Dataset Distillation Yunzhen Feng Ramakrishna Vedantam Julia Kempe DD 36 5 0 13 Nov 2023
Behavior Alignment via Reward Function Optimization Dhawal Gupta Yash Chandak Scott M. Jordan Philip S. Thomas Bruno Castro da Silva 31 10 0 29 Oct 2023
Studying K-FAC Heuristics by Viewing Adam through a Second-Order Lens Ross M. Clarke José Miguel Hernández-Lobato 54 2 0 23 Oct 2023
Series of Hessian-Vector Products for Tractable Saddle-Free Newton Optimisation of Neural Networks E. T. Oldewage Ross M. Clarke José Miguel Hernández-Lobato ODL 25 1 0 23 Oct 2023
You Only Condense Once: Two Rules for Pruning Condensed Datasets Yang He Lingao Xiao Qiufeng Wang 37 14 0 21 Oct 2023
Farzi Data: Autoregressive Data Distillation Noveen Sachdeva Zexue He Wang-Cheng Kang Jianmo Ni D. Cheng Julian McAuley DD 25 3 0 15 Oct 2023
PICProp: Physics-Informed Confidence Propagation for Uncertainty Quantification Qianli Shen Wai Hoh Tang Zhun Deng Apostolos F. Psaros Kenji Kawaguchi 102 1 0 10 Oct 2023
Learning Layer-wise Equivariances Automatically using Gradients Tycho F. A. van der Ouderaa Alexander Immer Mark van der Wilk MLT 52 12 0 09 Oct 2023
Deep Concept Removal Yegor Klochkov Jean-François Ton Ruocheng Guo Yang Liu Hang Li 23 0 0 09 Oct 2023
Making Scalable Meta Learning Practical Sang Keun Choe Sanket Vaibhav Mehta Hwijeen Ahn Willie Neiswanger Pengtao Xie Emma Strubell Eric Xing 58 15 0 09 Oct 2023
FedL2P: Federated Learning to Personalize Royson Lee Minyoung Kim Da Li Xinchi Qiu Timothy M. Hospedales Ferenc Huszár Nicholas D. Lane FedML 18 0 0 03 Oct 2023
Unveiling Invariances via Neural Network Pruning Derek Xu Yizhou Sun Wei Wang 41 0 0 15 Sep 2023
Differentiable Robust Model Predictive Control Alex Oshin Hassan Almubarak Evangelos A. Theodorou 17 5 0 16 Aug 2023
Vision-Language Dataset Distillation Xindi Wu Byron Zhang Zhiwei Deng Olga Russakovsky DD VLM 33 9 0 15 Aug 2023
Influence Function Based Second-Order Channel Pruning-Evaluating True Loss Changes For Pruning Is Possible Without Retraining Hongrong Cheng Miao Zhang Javen Qinfeng Shi AAML 33 1 0 13 Aug 2023
Non-Convex Bilevel Optimization with Time-Varying Objective Functions Sen-Fon Lin Daouda Sow Kaiyi Ji Yitao Liang Ness B. Shroff 36 2 0 07 Aug 2023
Doubly Robust Instance-Reweighted Adversarial Training Daouda Sow Sen-Fon Lin Zhangyang Wang Yitao Liang AAML OOD 33 2 0 01 Aug 2023
Learning with Constraint Learning: New Perspective, Solution Strategy and Various Applications Risheng Liu Jiaxin Gao Xuan Liu Xin-Yue Fan 40 9 0 28 Jul 2023
Sharpness-Aware Graph Collaborative Filtering Huiyuan Chen Chin-Chia Michael Yeh Yujie Fan Yan Zheng Junpeng Wang Vivian Lai Mahashweta Das Hao Yang 36 5 0 18 Jul 2023
Automated Polynomial Filter Learning for Graph Neural Networks Wendi Yu Zhichao Hou Xiaorui Liu 24 0 0 16 Jul 2023
Near-Optimal Nonconvex-Strongly-Convex Bilevel Optimization with Fully First-Order Oracles Le‐Yu Chen Yaohua Ma Jiaming Zhang 86 2 0 26 Jun 2023
Taming Small-sample Bias in Low-budget Active Learning Linxin Song Jieyu Zhang Xiaotian Lu Dinesh Manocha AI4CE 22 0 0 19 Jun 2023
Differentiable Instruction Optimization for Cross-Task Generalization Masaru Isonuma Junichiro Mori Ichiro Sakata 29 0 0 16 Jun 2023
Unbiased Learning of Deep Generative Models with Structured Discrete Representations H. Bendekgey Gabriel Hope Erik B. Sudderth OCL BDL DRL 30 1 0 14 Jun 2023
AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Risks Alexander Tornede Difan Deng Theresa Eimer Joseph Giovanelli Aditya Mohan ... Sarah Segel Daphne Theodorakopoulos Tanja Tornede Henning Wachsmuth Marius Lindauer 41 23 0 13 Jun 2023
Stepsize Learning for Policy Gradient Methods in Contextual Markov Decision Processes Luca Sabbioni Francesco Corda Marcello Restelli 29 0 0 13 Jun 2023
Stochastic Marginal Likelihood Gradients using Neural Tangent Kernels Alexander Immer Tycho F. A. van der Ouderaa Mark van der Wilk Gunnar Rätsch Bernhard Schölkopf BDL 36 11 0 06 Jun 2023
BiSLS/SPS: Auto-tune Step Sizes for Stable Bi-level Optimization Chen Fan Gaspard Choné-Ducasse Mark Schmidt Christos Thrampoulidis 30 3 0 30 May 2023
One-step differentiation of iterative algorithms Jérôme Bolte Edouard Pauwels Samuel Vaiter 72 13 0 23 May 2023
Effective Bilevel Optimization via Minimax Reformulation Xiaoyu Wang Rui Pan Renjie Pi Tong Zhang 42 1 0 22 May 2023
Hyperparameter Optimization through Neural Network Partitioning Bruno Mlodozeniec M. Reisser Christos Louizos 42 6 0 28 Apr 2023
Low-Variance Gradient Estimation in Unrolled Computation Graphs with ES-Single Paul Vicol Zico Kolter Kevin Swersky 21 6 0 21 Apr 2023