Optimizing Millions of Hyperparameters by Implicit Differentiation

6 November 2019

Paul Vicol

Papers citing "Optimizing Millions of Hyperparameters by Implicit Differentiation"

50 / 271 papers shown

Title
Bi-Level Policy Optimization with Nyström Hypergradients Arjun Prakash Naicheng He Denizalp Goktas Amy Greenwald 4 0 0 16 May 2025
Argus: Federated Non-convex Bilevel Learning over 6G Space-Air-Ground Integrated Network Ya Liu Kai Yang Yu Zhu Keying Yang Haibo Zhao 31 0 0 14 May 2025
You Only Look One Step: Accelerating Backpropagation in Diffusion Sampling with Gradient Shortcuts Hongkun Dou Zeyu Li Xingyu Jiang Yiming Li Lijun Yang Wen Yao Yue Deng DiffM 38 0 0 12 May 2025
Frozen Layers: Memory-efficient Many-fidelity Hyperparameter Optimization Timur Carstensen Neeratyoy Mallik Frank Hutter Martin Rapp AI4CE 32 0 0 14 Apr 2025
Data Poisoning in Deep Learning: A Survey Pinlong Zhao Weiyao Zhu Pengfei Jiao Di Gao Ou Wu AAML 39 0 0 27 Mar 2025
Architecture-Aware Minimization (A $^2$ M): How to Find Flat Minima in Neural Architecture Search Matteo Gambella Fabrizio Pittorino Manuel Roveri 61 0 0 13 Mar 2025
Group-robust Sample Reweighting for Subpopulation Shifts via Influence Functions Rui Qiao Zhaoxuan Wu Jingtan Wang Pang Wei Koh Bryan Kian Hsiang Low OOD 53 2 0 10 Mar 2025
Variational Bayesian Pseudo-Coreset Hyungi Lee Seanie Lee Juho Lee BDL 38 0 0 28 Feb 2025
PRDP: Progressively Refined Differentiable Physics Kanishk Bhatia Felix Koehler Nils Thuerey AI4CE 66 0 0 26 Feb 2025
Spectral-factorized Positive-definite Curvature Learning for NN Training Wu Lin Felix Dangel Runa Eschenhagen Juhan Bae Richard E. Turner Roger B. Grosse 47 0 0 10 Feb 2025
Position: Curvature Matrices Should Be Democratized via Linear Operators Felix Dangel Runa Eschenhagen Weronika Ormaniec Andres Fernandez Lukas Tatzel Agustinus Kristiadi 61 3 0 31 Jan 2025
Think Smarter not Harder: Adaptive Reasoning with Inference Aware Optimization Zishun Yu Tengyu Xu Di Jin Karthik Abinav Sankararaman Yun He ... Eryk Helenowski Chen Zhu Sinong Wang Hao Ma Han Fang LRM 54 5 0 29 Jan 2025
Functional Risk Minimization Ferran Alet Clement Gehring Tomás Lozano-Pérez Kenji Kawaguchi Joshua B. Tenenbaum Leslie Pack Kaelbling OffRL 60 0 0 31 Dec 2024
Consistency of Compositional Generalization across Multiple Levels Chuanhao Li Zhen Li Chenchen Jing Xiaomeng Fan Wenbo Ye Yuwei Wu Yunde Jia CoGe 84 0 0 18 Dec 2024
Memory-Reduced Meta-Learning with Guaranteed Convergence Honglin Yang Ji Ma Xiao Yu 74 0 0 16 Dec 2024
Learning Set Functions with Implicit Differentiation Gözde Özcan Chengzhi Shi Stratis Ioannidis 69 0 0 15 Dec 2024
Exploring the Generalization Capabilities of AID-based Bi-level Optimization C. L. Philip Chen Li Shen Zhiqiang Xu Wei Liu Zhi-Quan Luo Peilin Zhao 73 1 0 25 Nov 2024
Zero-Shot Coreset Selection: Efficient Pruning for Unlabeled Data Brent A. Griffin Jacob Marks Jason J. Corso VLM 79 2 0 22 Nov 2024
Influence functions and regularity tangents for efficient active learning Frederik Eaton TDI 98 0 0 22 Nov 2024
Understanding Representation of Deep Equilibrium Models from Neural Collapse Perspective Haixiang Sun Ye Shi 42 0 0 30 Oct 2024
Theoretical Investigations and Practical Enhancements on Tail Task Risk Minimization in Meta Learning Yiqin Lv Qi Wang Dong Liang Zheng Xie 44 3 0 30 Oct 2024
Fully First-Order Methods for Decentralized Bilevel Optimization Xiaoyu Wang Xuxing Chen Shiqian Ma Tong Zhang 40 0 0 25 Oct 2024
Emphasizing Discriminative Features for Dataset Distillation in Complex Scenarios Kai Wang Zekai Li Zhi-Qi Cheng Samir Khaki A. Sajedi Ramakrishna Vedantam Konstantinos N. Plataniotis Alexander G. Hauptmann Yang You DD 70 5 0 22 Oct 2024
Automatic Differentiation of Optimization Algorithms with Time-Varying Updates Sheheryar Mehmood Peter Ochs 29 1 0 21 Oct 2024
Optimizing importance weighting in the presence of sub-population shifts Floris Holstege Bram Wouters Noud van Giersbergen C. Diks 36 0 0 18 Oct 2024
Towards Differentiable Multilevel Optimization: A Gradient-Based Approach Yuntian Gu Xuzheng Chen 16 0 0 15 Oct 2024
A Stochastic Approach to Bi-Level Optimization for Hyperparameter Optimization and Meta Learning Minyoung Kim Timothy M. Hospedales BDL 26 0 0 14 Oct 2024
BiDoRA: Bi-level Optimization-Based Weight-Decomposed Low-Rank Adaptation Peijia Qin Ruiyi Zhang Pengtao Xie 34 1 0 13 Oct 2024
Automating Data Science Pipelines with Tensor Completion Shaan Pakala Bryce Graw Dawon Ahn Tam Dinh Mehnaz Tabassum Mahin Vassilis Tsotras Jia Chen Evangelos E. Papalexakis 28 1 0 08 Oct 2024
SAFLEX: Self-Adaptive Augmentation via Feature Label Extrapolation Mucong Ding Bang An Yuancheng Xu Anirudh Satheesh Furong Huang 27 1 0 03 Oct 2024
Differentially Private Bilevel Optimization Guy Kornowski 201 0 0 29 Sep 2024
Online Nonconvex Bilevel Optimization with Bregman Divergences Jason Bohne David Rosenberg Gary Kazantsev Pawel Polak 35 0 0 16 Sep 2024
Enhancing Fairness through Reweighting: A Path to Attain the Sufficiency Rule Xuan Zhao Klaus Broelemann Salvatore Ruggieri Gjergji Kasneci 42 0 0 26 Aug 2024
Optimal Layer Selection for Latent Data Augmentation Tomoumi Takase Ryo Karakida 47 1 0 24 Aug 2024
JacNet: Learning Functions with Structured Jacobians Jonathan Lorraine Safwan Hossain 48 6 0 23 Aug 2024
FIVB ranking: Misstep in the right direction Salma Tenni Daniel Gomes de Pinho Zanco L. Szczecinski 32 0 0 02 Aug 2024
Self-Supervised Video Representation Learning in a Heuristic Decoupled Perspective Zeen Song Wenwen Qiang Jianqi Zhang Changwen Zheng Wenwen Qiang SSL 66 0 0 19 Jul 2024
Improving SAM Requires Rethinking its Optimization Formulation Wanyun Xie Fabian Latorre Kimon Antonakopoulos Thomas Pethick V. Cevher 42 1 0 17 Jul 2024
ScaleBiO: Scalable Bilevel Optimization for LLM Data Reweighting Rui Pan Jipeng Zhang Xingyuan Pan Renjie Pi Xiaoyu Wang Tong Zhang 58 5 0 28 Jun 2024
Improving Hyperparameter Optimization with Checkpointed Model Weights Nikhil Mehta Jonathan Lorraine Steve Masson Ramanathan Arunachalam Zaid Pervaiz Bhat James Lucas Arun George Zachariah 53 4 0 26 Jun 2024
Memory-Efficient Gradient Unrolling for Large-Scale Bi-level Optimization Qianli Shen Yezhen Wang Zhouhao Yang Xiang Li Haonan Wang Yang Zhang Jonathan Scarlett Zhanxing Zhu Kenji Kawaguchi AI4CE 76 4 0 20 Jun 2024
First-Order Methods for Linearly Constrained Bilevel Optimization Guy Kornowski Swati Padmanabhan Kai Wang Zhe Zhang S. Sra 82 5 0 18 Jun 2024
Meta-Learning Neural Procedural Biases Christian Raymond Qi Chen Bing Xue Mengjie Zhan 52 1 0 12 Jun 2024
Let Go of Your Labels with Unsupervised Transfer Artyom Gadetsky Yulun Jiang Maria Brbić VLM 45 6 0 11 Jun 2024
Calibrated Dataset Condensation for Faster Hyperparameter Search Mucong Ding Yuancheng Xu Tahseen Rabbani Xiaoyu Liu Brian J Gravelle Teresa M. Ranadive Tai-Ching Tuan Furong Huang DD 34 0 0 27 May 2024
Derivatives of Stochastic Gradient Descent F. Iutzeler Edouard Pauwels Samuel Vaiter 42 1 0 24 May 2024
Enhancing lattice kinetic schemes for fluid dynamics with Lattice-Equivariant Neural Networks Giulio Ortali Alessandro Gabbana Imre Atmodimedjo Alessandro Corbetta AI4CE 73 1 0 22 May 2024
A Lightweight Neural Architecture Search Model for Medical Image Classification Lunchen Xie Eugenio Lomurno Matteo Gambella Danilo Ardagna Manuel Roveri Matteo Matteucci Qingjiang Shi 24 2 0 06 May 2024
Generative Dataset Distillation: Balancing Global Structure and Local Details Longzhen Li Guang Li Ren Togo Keisuke Maeda Takahiro Ogawa Miki Haseyama DD 42 5 0 26 Apr 2024
Functional Bilevel Optimization for Machine Learning Ieva Petrulionyte Julien Mairal Michael Arbel 51 2 0 29 Mar 2024