Online Learning Rate Adaptation with Hypergradient Descent

14 March 2017

Papers citing "Online Learning Rate Adaptation with Hypergradient Descent"

49 / 49 papers shown

Title
B-STaR: Monitoring and Balancing Exploration and Exploitation in Self-Taught Reasoners Weihao Zeng Yuzhen Huang Lulu Zhao Yijun Wang Zifei Shan Junxian He LRM 40 7 0 23 Dec 2024
DeepMpMRI: Tensor-decomposition Regularized Learning for Fast and High-Fidelity Multi-Parametric Microstructural MR Imaging Wen-Jie Fan Jian Cheng Cheng Li Xinrui Ma Jing Yang ... Ruo-Nan Wu Zan Chen Yuanjing Feng Hairong Zheng Shanshan Wang MedIm 25 1 0 06 May 2024
Downstream Task Guided Masking Learning in Masked Autoencoders Using Multi-Level Optimization Han Guo Ramtin Hosseini Ruiyi Zhang Sai Ashish Somayajula Ranak Roy Chowdhury Rajesh K. Gupta Pengtao Xie 36 0 0 28 Feb 2024
MADA: Meta-Adaptive Optimizers through hyper-gradient Descent Kaan Ozkara Can Karakus Parameswaran Raman Mingyi Hong Shoham Sabach B. Kveton V. Cevher 27 2 0 17 Jan 2024
SCoTTi: Save Computation at Training Time with an adaptive framework Ziyu Li Enzo Tartaglione Van-Tam Nguyen 33 0 0 19 Dec 2023
Learning How to Propagate Messages in Graph Neural Networks Teng Xiao Zhengyu Chen Donglin Wang Suhang Wang GNN 28 76 0 01 Oct 2023
Differentiable Arbitrating in Zero-sum Markov Games Jing Wang Meichen Song Feng Gao Boyi Liu Zhaoran Wang Yi Wu 37 2 0 20 Feb 2023
Contrastive Learning with Consistent Representations Zihu Wang Yu Wang Hanbin Hu Peng Li CLL 29 5 0 03 Feb 2023
QLABGrad: a Hyperparameter-Free and Convergence-Guaranteed Scheme for Deep Learning Minghan Fu Fang-Xiang Wu ODL 22 6 0 01 Feb 2023
Online Loss Function Learning Christian Raymond Qi Chen Bing Xue Mengjie Zhang 30 5 0 30 Jan 2023
Read the Signs: Towards Invariance to Gradient Descent's Hyperparameter Initialization Davood Wadi M. Fredette S. Sénécal ODL AI4CE 8 0 0 24 Jan 2023
A Nonstochastic Control Approach to Optimization Xinyi Chen Elad Hazan 47 5 0 19 Jan 2023
Federated Automatic Differentiation Keith Rush Zachary B. Charles Zachary Garrett FedML 34 1 0 18 Jan 2023
Federated Hypergradient Descent A. K. Kan FedML 37 3 0 03 Nov 2022
Differentiable Self-Adaptive Learning Rate Bozhou Chen Hongzhi Wang Chenmin Ba ODL 17 4 0 19 Oct 2022
On Stability and Generalization of Bilevel Optimization Problem Meng Ding Ming Lei Yunwen Lei Di Wang Jinhui Xu 32 0 0 03 Oct 2022
Learning to Learn with Generative Models of Neural Network Checkpoints William S. Peebles Ilija Radosavovic Tim Brooks Alexei A. Efros Jitendra Malik UQCV 75 64 0 26 Sep 2022
A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases James Harrison Luke Metz Jascha Narain Sohl-Dickstein 47 22 0 22 Sep 2022
Learning Symbolic Model-Agnostic Loss Functions via Meta-Learning Christian Raymond Qi Chen Bing Xue Mengjie Zhang FedML 29 11 0 19 Sep 2022
Online Meta-Learning for Model Update Aggregation in Federated Learning for Click-Through Rate Prediction Xianghang Liu Bartlomiej Twardowski Tri Kurniawan Wijaya FedML 26 1 0 30 Aug 2022
Betty: An Automatic Differentiation Library for Multilevel Optimization Sang Keun Choe W. Neiswanger P. Xie Eric P. Xing AI4CE 31 30 0 05 Jul 2022
Grad-GradaGrad? A Non-Monotone Adaptive Stochastic Gradient Method Aaron Defazio Baoyu Zhou Lin Xiao ODL 24 5 0 14 Jun 2022
Automated Dynamic Algorithm Configuration Steven Adriaensen André Biedenkapp Gresa Shala Noor H. Awad Theresa Eimer Marius Lindauer Frank Hutter 32 36 0 27 May 2022
DLTTA: Dynamic Learning Rate for Test-time Adaptation on Cross-domain Medical Images Hongzheng Yang Cheng Chen Meirui Jiang Quande Liu Jianfeng Cao Pheng Ann Heng Qi Dou OOD 37 26 0 27 May 2022
Amortized Proximal Optimization Juhan Bae Paul Vicol Jeff Z. HaoChen Roger C. Grosse ODL 25 14 0 28 Feb 2022
Step-size Adaptation Using Exponentiated Gradient Updates Ehsan Amid Rohan Anil Christopher Fifty Manfred K. Warmuth 35 9 0 31 Jan 2022
A Stochastic Bundle Method for Interpolating Networks Alasdair Paren Leonard Berrada Rudra P. K. Poudel M. P. Kumar 24 4 0 29 Jan 2022
DDG-DA: Data Distribution Generation for Predictable Concept Drift Adaptation Wendi Li Xiao Yang Weiqing Liu Yingce Xia Jiang Bian DiffM AI4TS 20 50 0 11 Jan 2022
Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies Paul Vicol Luke Metz Jascha Narain Sohl-Dickstein 27 67 0 27 Dec 2021
Automated Deep Learning: Neural Architecture Search Is Not the End Xuanyi Dong D. Kedziora Katarzyna Musial Bogdan Gabrys 25 26 0 16 Dec 2021
Scalable One-Pass Optimisation of High-Dimensional Weight-Update Hyperparameters by Implicit Differentiation Ross M. Clarke E. T. Oldewage José Miguel Hernández-Lobato 28 9 0 20 Oct 2021
Learning by Examples Based on Multi-level Optimization Shentong Mo P. Xie 26 0 0 22 Sep 2021
Graph Contrastive Learning Automated Yuning You Tianlong Chen Yang Shen Zhangyang Wang 24 447 0 10 Jun 2021
AngularGrad: A New Optimization Technique for Angular Convergence of Convolutional Neural Networks S. K. Roy Mercedes Eugenia Paoletti J. Haut S. Dubey Purushottam Kar A. Plaza B. B. Chaudhuri ODL 19 18 0 21 May 2021
Robust MAML: Prioritization task buffer with adaptive learning process for model-agnostic meta-learning Thanh Nguyen Tung M. Luu T. Pham Sanzhar Rakhimkul Chang D. Yoo 16 10 0 15 Mar 2021
Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond Risheng Liu Jiaxin Gao Jin Zhang Deyu Meng Zhouchen Lin AI4CE 53 222 0 27 Jan 2021
Self-Tuning Stochastic Optimization with Curvature-Aware Gradient Filtering Ricky T. Q. Chen Dami Choi Lukas Balles David Duvenaud Philipp Hennig ODL 41 6 0 09 Nov 2020
A straightforward line search approach on the expected empirical loss for stochastic deep learning problems Max Mutschler A. Zell 30 0 0 02 Oct 2020
Improved Bilevel Model: Fast and Optimal Algorithm with Theoretical Guarantee Junyi Li Bin Gu Heng-Chiao Huang 16 41 0 01 Sep 2020
Meta-Learning in Neural Networks: A Survey Timothy M. Hospedales Antreas Antoniou P. Micaelli Amos Storkey OOD 46 1,928 0 11 Apr 2020
Meta Pseudo Labels Hieu H. Pham Zihang Dai Qizhe Xie Minh-Thang Luong Quoc V. Le VLM 259 656 0 23 Mar 2020
Statistical Adaptive Stochastic Gradient Methods Pengchuan Zhang Hunter Lang Qiang Liu Lin Xiao ODL 15 11 0 25 Feb 2020
On the adequacy of untuned warmup for adaptive optimization Jerry Ma Denis Yarats 53 70 0 09 Oct 2019
Learning an Adaptive Learning Rate Schedule Zhen Xu Andrew M. Dai Jonas Kemp Luke Metz 14 61 0 20 Sep 2019
Block stochastic gradient descent for large-scale tomographic reconstruction in a parallel network Yushan Gao A. Biguri T. Blumensath 18 3 0 28 Mar 2019
Truncated Back-propagation for Bilevel Optimization Amirreza Shaban Ching-An Cheng Nathan Hatch Byron Boots 24 261 0 25 Oct 2018
Quasi-hyperbolic momentum and Adam for deep learning Jerry Ma Denis Yarats ODL 84 129 0 16 Oct 2018
Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition Hamed Karimi J. Nutini Mark W. Schmidt 139 1,199 0 16 Aug 2016
Automatic differentiation in machine learning: a survey A. G. Baydin Barak A. Pearlmutter Alexey Radul J. Siskind PINN AI4CE ODL 39 2,746 0 20 Feb 2015