Forward and Reverse Gradient-Based Hyperparameter Optimization

6 March 2017

Luca Franceschi

Papers citing "Forward and Reverse Gradient-Based Hyperparameter Optimization"

50 / 76 papers shown

Title
A stochastic gradient method for trilevel optimization Tommaso Giovannelli G. Kent Luis Nunes Vicente 29 0 0 11 May 2025
Efficient Curvature-Aware Hypergradient Approximation for Bilevel Optimization Youran Dong Junfeng Yang Wei-Ting Yao Jin Zhang 121 0 0 04 May 2025
Scalable Meta-Learning via Mixed-Mode Differentiation Iurii Kemaev Dan A Calian Luisa M Zintgraf Gregory Farquhar H. V. Hasselt 57 0 0 01 May 2025
Frozen Layers: Memory-efficient Many-fidelity Hyperparameter Optimization Timur Carstensen Neeratyoy Mallik Frank Hutter Martin Rapp AI4CE 24 0 0 14 Apr 2025
Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomy Chen Wang Kaiyi Ji Junyi Geng Zhongqiang Ren Taimeng Fu ... Yi Du Qihang Li Y. Yang Xiao Lin Zhipeng Zhao SSL 78 9 0 28 Jan 2025
DeepEyeNet: Adaptive Genetic Bayesian Algorithm Based Hybrid ConvNeXtTiny Framework For Multi-Feature Glaucoma Eye Diagnosis Angshuman Roy Anuvab Sen Soumyajit Gupta Soham Haldar Subhrajit Deb Taraka Nithin Vankala Arkapravo Das 36 0 0 19 Jan 2025
Fully First-Order Methods for Decentralized Bilevel Optimization Xiaoyu Wang Xuxing Chen Shiqian Ma Tong Zhang 36 0 0 25 Oct 2024
Differentially Private Bilevel Optimization Guy Kornowski 124 0 0 29 Sep 2024
Learning to Select the Best Forecasting Tasks for Clinical Outcome Prediction Yuan Xue Nan Du A. Mottram Martin G. Seneviratne Andrew M. Dai AI4TS 23 0 0 28 Jul 2024
First-Order Methods for Linearly Constrained Bilevel Optimization Guy Kornowski Swati Padmanabhan Kai Wang Zhe Zhang S. Sra 70 5 0 18 Jun 2024
Bilevel reinforcement learning via the development of hyper-gradient without lower-level convexity Yan Yang Bin Gao Ya-xiang Yuan 38 2 0 30 May 2024
Moreau Envelope for Nonconvex Bi-Level Optimization: A Single-loop and Hessian-free Solution Strategy Risheng Liu Zhu Liu Wei-Ting Yao Shangzhi Zeng Jin Zhang 35 6 0 16 May 2024
LancBiO: dynamic Lanczos-aided bilevel optimization via Krylov subspace Bin Gao Yan Yang Ya-xiang Yuan 34 2 0 04 Apr 2024
A First-Order Multi-Gradient Algorithm for Multi-Objective Bi-Level Optimization Feiyang Ye Baijiong Lin Xiao-Qun Cao Yu Zhang Ivor Tsang 45 6 0 17 Jan 2024
MADA: Meta-Adaptive Optimizers through hyper-gradient Descent Kaan Ozkara Can Karakus Parameswaran Raman Mingyi Hong Shoham Sabach B. Kveton V. Cevher 19 2 0 17 Jan 2024
Doubly Robust Instance-Reweighted Adversarial Training Daouda Sow Sen-Fon Lin Zhangyang Wang Yitao Liang AAML OOD 33 2 0 01 Aug 2023
Differentiable Instruction Optimization for Cross-Task Generalization Masaru Isonuma Junichiro Mori Ichiro Sakata 23 0 0 16 Jun 2023
Federated Multi-Sequence Stochastic Approximation with Local Hypergradient Estimation Davoud Ataee Tarzanagh Mingchen Li Pranay Sharma Samet Oymak 24 0 0 02 Jun 2023
FedAVO: Improving Communication Efficiency in Federated Learning with African Vultures Optimizer Md Zarif Hossain Ahmed Imteaj FedML 24 5 0 02 May 2023
OpenBox: A Python Toolkit for Generalized Black-box Optimization Huaijun Jiang Yu Shen Yang Li Beicheng Xu Sixian Du Wentao Zhang Ce Zhang Bin Cui 30 4 0 26 Apr 2023
Variance-Reduced Gradient Estimation via Noise-Reuse in Online Evolution Strategies Oscar Li James Harrison Jascha Narain Sohl-Dickstein Virginia Smith Luke Metz 44 5 0 21 Apr 2023
Achieving Hierarchy-Free Approximation for Bilevel Programs With Equilibrium Constraints Jiayang Li J. Yu Boyi Liu Zhaoran Wang Y. Nie 29 6 0 20 Feb 2023
Efficient Gradient Approximation Method for Constrained Bilevel Optimization Siyuan Xu Minghui Zhu 19 19 0 03 Feb 2023
Dataset Distillation: A Comprehensive Review Ruonan Yu Songhua Liu Xinchao Wang DD 39 121 0 17 Jan 2023
First-order penalty methods for bilevel optimization Zhaosong Lu Sanyou Mei 62 31 0 04 Jan 2023
A Penalty-Based Method for Communication-Efficient Decentralized Bilevel Programming Parvin Nazari Ahmad Mousavi Davoud Ataee Tarzanagh George Michailidis 19 4 0 08 Nov 2022
Passage-Mask: A Learnable Regularization Strategy for Retriever-Reader Models Shujian Zhang Chengyue Gong Xingchao Liu RALM 37 6 0 02 Nov 2022
One-Shot Learning of Stochastic Differential Equations with Data Adapted Kernels Matthieu Darcy B. Hamzi Giulia Livieri H. Owhadi P. Tavallali 36 25 0 24 Sep 2022
Learning Symbolic Model-Agnostic Loss Functions via Meta-Learning Christian Raymond Qi Chen Bing Xue Mengjie Zhang FedML 27 11 0 19 Sep 2022
Dataset Condensation with Latent Space Knowledge Factorization and Sharing Haebeom Lee Dong Bok Lee Sung Ju Hwang DD 21 37 0 21 Aug 2022
Fixed-Point Automatic Differentiation of Forward--Backward Splitting Algorithms for Partly Smooth Functions Sheheryar Mehmood Peter Ochs 33 3 0 05 Aug 2022
Gradient-based Bi-level Optimization for Deep Learning: A Survey Can Chen Xiangshan Chen Chen-li Ma Zixuan Liu Xue Liu 84 35 0 24 Jul 2022
Online Knowledge Distillation via Mutual Contrastive Learning for Visual Recognition Chuanguang Yang Zhulin An Helong Zhou Fuzhen Zhuang Yongjun Xu Qian Zhang 31 50 0 23 Jul 2022
Online Bilevel Optimization: Regret Analysis of Online Alternating Gradient Methods Davoud Ataee Tarzanagh Parvin Nazari Bojian Hou Li Shen Laura Balzano 41 10 0 06 Jul 2022
Betty: An Automatic Differentiation Library for Multilevel Optimization Sang Keun Choe W. Neiswanger P. Xie Eric P. Xing AI4CE 28 29 0 05 Jul 2022
Optimization-Derived Learning with Essential Convergence Analysis of Training and Hyper-training Risheng Liu Xuan Liu Shangzhi Zeng Jin Zhang Yixuan Zhang 35 6 0 16 Jun 2022
A Survey of Automated Data Augmentation Algorithms for Deep Learning-based Image Classification Tasks Z. Yang Richard Sinnott James Bailey Qiuhong Ke 20 39 0 14 Jun 2022
Remember the Past: Distilling Datasets into Addressable Memories for Neural Networks Zhiwei Deng Olga Russakovsky FedML DD 28 92 0 06 Jun 2022
Revisiting GANs by Best-Response Constraint: Perspective, Methodology, and Application Risheng Liu Jiaxin Gao Xuan Liu Xin-Yue Fan 29 4 0 20 May 2022
FedNest: Federated Bilevel, Minimax, and Compositional Optimization Davoud Ataee Tarzanagh Mingchen Li Christos Thrampoulidis Samet Oymak FedML 38 73 0 04 May 2022
Local Stochastic Bilevel Optimization with Momentum-Based Variance Reduction Junyi Li Feihu Huang Heng-Chiao Huang FedML 19 27 0 03 May 2022
On the Convergence of Momentum-Based Algorithms for Federated Bilevel Optimization Problems Hongchang Gao FedML 18 1 0 28 Apr 2022
Learning the Effect of Registration Hyperparameters with HyperMorph Andrew Hoopes Malte Hoffmann Douglas N. Greve Bruce Fischl John Guttag Adrian V. Dalca 28 38 0 30 Mar 2022
Meta Mirror Descent: Optimiser Learning for Fast Convergence Boyan Gao H. Gouk Haebeom Lee Timothy M. Hospedales 11 6 0 05 Mar 2022
Amortized Proximal Optimization Juhan Bae Paul Vicol Jeff Z. HaoChen Roger C. Grosse ODL 25 14 0 28 Feb 2022
Short-answer scoring with ensembles of pretrained language models Christopher M. Ormerod 31 8 0 23 Feb 2022
Tutorial on amortized optimization Brandon Amos OffRL 75 43 0 01 Feb 2022
On Training Implicit Models Zhengyang Geng Xinyu Zhang Shaojie Bai Yisen Wang Zhouchen Lin 59 69 0 09 Nov 2021
Scalable One-Pass Optimisation of High-Dimensional Weight-Update Hyperparameters by Implicit Differentiation Ross M. Clarke E. T. Oldewage José Miguel Hernández-Lobato 15 9 0 20 Oct 2021
HRKD: Hierarchical Relational Knowledge Distillation for Cross-domain Language Model Compression Chenhe Dong Yaliang Li Ying Shen Minghui Qiu VLM 30 7 0 16 Oct 2021