Gradient-based Hyperparameter Optimization through Reversible Learning

11 February 2015

Papers citing "Gradient-based Hyperparameter Optimization through Reversible Learning"

50 / 187 papers shown

Title
Efficient Curvature-Aware Hypergradient Approximation for Bilevel Optimization Youran Dong Junfeng Yang Wei-Ting Yao Jin Zhang 135 0 0 04 May 2025
Scalable Meta-Learning via Mixed-Mode Differentiation Iurii Kemaev Dan A Calian Luisa M Zintgraf Gregory Farquhar H. V. Hasselt 57 0 0 01 May 2025
MAGIC: Near-Optimal Data Attribution for Deep Learning Andrew Ilyas Logan Engstrom TDI 39 0 0 23 Apr 2025
An experimental survey and Perspective View on Meta-Learning for Automated Algorithms Selection and Parametrization Moncef Garouani 28 0 0 08 Apr 2025
Reinforcement Teaching Alex Lewandowski Calarina Muslimani Dale Schuurmans Matthew E. Taylor Jun Luo 81 1 0 28 Jan 2025
Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomy Chen Wang Kaiyi Ji Junyi Geng Zhongqiang Ren Taimeng Fu ... Yi Du Qihang Li Yi Yang Xiao Lin Zhipeng Zhao SSL 89 9 0 28 Jan 2025
Multi-Objective Hyperparameter Selection via Hypothesis Testing on Reliability Graphs Amirmohammad Farzaneh Osvaldo Simeone 86 0 0 22 Jan 2025
A Learn-to-Optimize Approach for Coordinate-Wise Step Sizes for Quasi-Newton Methods Wei Lin Qingyu Song Hong Xu 94 1 0 25 Nov 2024
Influence functions and regularity tangents for efficient active learning Frederik Eaton TDI 94 0 0 22 Nov 2024
Fully First-Order Methods for Decentralized Bilevel Optimization Xiaoyu Wang Xuxing Chen Shiqian Ma Tong Zhang 38 0 0 25 Oct 2024
Differentially Private Bilevel Optimization Guy Kornowski 142 0 0 29 Sep 2024
Adaptive Learn-then-Test: Statistically Valid and Efficient Hyperparameter Selection Matteo Zecchin Sangwoo Park Osvaldo Simeone LM&MA 77 3 0 24 Sep 2024
Online Nonconvex Bilevel Optimization with Bregman Divergences Jason Bohne David Rosenberg Gary Kazantsev Pawel Polak 27 0 0 16 Sep 2024
A New First-Order Meta-Learning Algorithm with Convergence Guarantees El Mahdi Chayti Martin Jaggi 30 1 0 05 Sep 2024
Learning to (Learn at Test Time): RNNs with Expressive Hidden States Yu Sun Xinhao Li Karan Dalal Jiarui Xu Arjun Vikram ... Xinlei Chen Xiaolong Wang Sanmi Koyejo Tatsunori Hashimoto Carlos Guestrin 63 92 0 05 Jul 2024
First-Order Methods for Linearly Constrained Bilevel Optimization Guy Kornowski Swati Padmanabhan Kai Wang Zhe Zhang S. Sra 78 5 0 18 Jun 2024
LancBiO: dynamic Lanczos-aided bilevel optimization via Krylov subspace Bin Gao Yan Yang Ya-xiang Yuan 39 2 0 04 Apr 2024
DiLM: Distilling Dataset into Language Model for Text-level Dataset Distillation Aru Maekawa Satoshi Kosugi Kotaro Funakoshi Manabu Okumura DD 43 10 0 30 Mar 2024
AutoLoRA: Automatically Tuning Matrix Ranks in Low-Rank Adaptation Based on Meta Learning Ruiyi Zhang Rushi Qiang Sai Ashish Somayajula Pengtao Xie 42 13 0 14 Mar 2024
Bilevel Optimization under Unbounded Smoothness: A New Algorithm and Convergence Analysis Jie Hao Xiaochuan Gong Mingrui Liu 30 7 0 17 Jan 2024
A First-Order Multi-Gradient Algorithm for Multi-Objective Bi-Level Optimization Feiyang Ye Baijiong Lin Xiao-Qun Cao Yu Zhang Ivor Tsang 50 6 0 17 Jan 2024
MADA: Meta-Adaptive Optimizers through hyper-gradient Descent Kaan Ozkara Can Karakus Parameswaran Raman Mingyi Hong Shoham Sabach B. Kveton V. Cevher 27 2 0 17 Jan 2024
TorchDEQ: A Library for Deep Equilibrium Models Zhengyang Geng J. Zico Kolter VLM 56 12 0 28 Oct 2023
Farzi Data: Autoregressive Data Distillation Noveen Sachdeva Zexue He Wang-Cheng Kang Jianmo Ni D. Cheng Julian McAuley DD 23 3 0 15 Oct 2023
Learning How to Propagate Messages in Graph Neural Networks Teng Xiao Zhengyu Chen Donglin Wang Suhang Wang GNN 34 76 0 01 Oct 2023
Relax and penalize: a new bilevel approach to mixed-binary hyperparameter optimization M. D. Santis Jordan Frécon Francesco Rinaldi Saverio Salzo Martin Schmidt Martin Schmidt 53 0 0 21 Aug 2023
An adaptively inexact first-order method for bilevel optimization with application to hyperparameter learning Mohammad Salehi Subhadip Mukherjee Lindon Roberts Matthias Joachim Ehrhardt 26 5 0 19 Aug 2023
Doubly Robust Instance-Reweighted Adversarial Training Daouda Sow Sen-Fon Lin Zhangyang Wang Yitao Liang AAML OOD 33 2 0 01 Aug 2023
Probabilistic Unrolling: Scalable, Inverse-Free Maximum Likelihood Estimation for Latent Gaussian Models Alexander Lin Bahareh Tolooshams Yves Atchadé Demba E. Ba 36 1 0 05 Jun 2023
Federated Multi-Sequence Stochastic Approximation with Local Hypergradient Estimation Davoud Ataee Tarzanagh Mingchen Li Pranay Sharma Samet Oymak 26 0 0 02 Jun 2023
FedAVO: Improving Communication Efficiency in Federated Learning with African Vultures Optimizer Md Zarif Hossain Ahmed Imteaj FedML 29 5 0 02 May 2023
Variance-Reduced Gradient Estimation via Noise-Reuse in Online Evolution Strategies Oscar Li James Harrison Jascha Narain Sohl-Dickstein Virginia Smith Luke Metz 49 5 0 21 Apr 2023
Achieving Hierarchy-Free Approximation for Bilevel Programs With Equilibrium Constraints Jiayang Li Jiahao Yu Boyi Liu Zhaoran Wang Y. Nie 35 6 0 20 Feb 2023
Dataset Distillation with Convexified Implicit Gradients Noel Loo Ramin Hasani Mathias Lechner Daniela Rus DD 31 41 0 13 Feb 2023
Online Loss Function Learning Christian Raymond Qi Chen Bing Xue Mengjie Zhang 35 5 0 30 Jan 2023
Probabilistic Bilevel Coreset Selection Xiao Zhou Renjie Pi Weizhong Zhang Yong Lin Tong Zhang NoLa 31 27 0 24 Jan 2023
Model Agnostic Sample Reweighting for Out-of-Distribution Learning Xiao Zhou Yong Lin Renjie Pi Weizhong Zhang Renzhe Xu Peng Cui Tong Zhang OODD 39 60 0 24 Jan 2023
Federated Automatic Differentiation Keith Rush Zachary B. Charles Zachary Garrett FedML 34 1 0 18 Jan 2023
Dataset Distillation: A Comprehensive Review Ruonan Yu Songhua Liu Xinchao Wang DD 53 121 0 17 Jan 2023
A Comprehensive Survey of Dataset Distillation Shiye Lei Dacheng Tao DD 31 87 0 13 Jan 2023
Analyzing Inexact Hypergradients for Bilevel Learning Matthias Joachim Ehrhardt Lindon Roberts 24 8 0 11 Jan 2023
Data Distillation: A Survey Noveen Sachdeva Julian McAuley DD 45 73 0 11 Jan 2023
First-order penalty methods for bilevel optimization Zhaosong Lu Sanyou Mei 66 31 0 04 Jan 2023
Accelerating Dataset Distillation via Model Augmentation Lei Zhang Jie M. Zhang Bowen Lei Subhabrata Mukherjee Xiang Pan Bo-Lu Zhao Caiwen Ding Heng Chang Dongkuan Xu DD 43 62 0 12 Dec 2022
Task Discovery: Finding the Tasks that Neural Networks Generalize on Andrei Atanov Andrei Filatov Teresa Yeo Ajay Sohmshetty Amir Zamir OOD 45 10 0 01 Dec 2022
VeLO: Training Versatile Learned Optimizers by Scaling Up Luke Metz James Harrison C. Freeman Amil Merchant Lucas Beyer ... Naman Agrawal Ben Poole Igor Mordatch Adam Roberts Jascha Narain Sohl-Dickstein 35 60 0 17 Nov 2022
A Penalty-Based Method for Communication-Efficient Decentralized Bilevel Programming Parvin Nazari Ahmad Mousavi Davoud Ataee Tarzanagh George Michailidis 28 4 0 08 Nov 2022
Learning to Learn with Generative Models of Neural Network Checkpoints William S. Peebles Ilija Radosavovic Tim Brooks Alexei A. Efros Jitendra Malik UQCV 75 64 0 26 Sep 2022
One-Shot Learning of Stochastic Differential Equations with Data Adapted Kernels Matthieu Darcy B. Hamzi Giulia Livieri H. Owhadi P. Tavallali 36 25 0 24 Sep 2022
Tradeoffs between convergence rate and noise amplification for momentum-based accelerated optimization algorithms Hesameddin Mohammadi Meisam Razaviyayn Mihailo R. Jovanović 18 7 0 24 Sep 2022