Learning an Adaptive Learning Rate Schedule

20 September 2019

Papers citing "Learning an Adaptive Learning Rate Schedule"

21 / 21 papers shown

Title
A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules Kairong Luo Haodong Wen Shengding Hu Zhenbo Sun Zhiyuan Liu Maosong Sun Kaifeng Lyu Wenguang Chen CLL 67 2 0 17 Mar 2025
Narrowing the Focus: Learned Optimizers for Pretrained Models Gus Kristiansen Mark Sandler A. Zhmoginov Nolan Miller Anirudh Goyal Jihwan Lee Max Vladymyrov 39 1 0 17 Aug 2024
Neural Optimizer Equation, Decay Function, and Learning Rate Schedule Joint Evolution Brandon Morgan Dean Frederick Hougen ODL 25 0 0 10 Apr 2024
Dynamic Layer Tying for Parameter-Efficient Transformers Tamir David Hay Lior Wolf 33 3 0 23 Jan 2024
Generalisable Agents for Neural Network Optimisation Kale-ab Tessera C. Tilbury Sasha Abramowitz Ruan de Kock Omayma Mahjoub Benjamin Rosman Sara Hooker Arnu Pretorius AI4CE 20 0 0 30 Nov 2023
Studying K-FAC Heuristics by Viewing Adam through a Second-Order Lens Ross M. Clarke José Miguel Hernández-Lobato 46 2 0 23 Oct 2023
Intelligent gradient amplification for deep neural networks S. Basodi K. Pusuluri Xueli Xiao Yi Pan ODL 21 1 0 29 May 2023
VeLO: Training Versatile Learned Optimizers by Scaling Up Luke Metz James Harrison C. Freeman Amil Merchant Lucas Beyer ... Naman Agrawal Ben Poole Igor Mordatch Adam Roberts Jascha Narain Sohl-Dickstein 35 60 0 17 Nov 2022
Learning to Learn with Generative Models of Neural Network Checkpoints William S. Peebles Ilija Radosavovic Tim Brooks Alexei A. Efros Jitendra Malik UQCV 75 65 0 26 Sep 2022
A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases James Harrison Luke Metz Jascha Narain Sohl-Dickstein 47 22 0 22 Sep 2022
Automated Dynamic Algorithm Configuration Steven Adriaensen André Biedenkapp Gresa Shala Noor H. Awad Theresa Eimer Marius Lindauer Frank Hutter 32 36 0 27 May 2022
Practical tradeoffs between memory, compute, and performance in learned optimizers Luke Metz C. Freeman James Harrison Niru Maheswaranathan Jascha Narain Sohl-Dickstein 38 32 0 22 Mar 2022
A Simple Guard for Learned Optimizers Isabeau Prémont-Schwarz Jaroslav Vítkru Jan Feyereisl 49 7 0 28 Jan 2022
AutoBalance: Optimized Loss Functions for Imbalanced Data Mingchen Li Xuechen Zhang Christos Thrampoulidis Jiasi Chen Samet Oymak 19 67 0 04 Jan 2022
Efficient Meta Subspace Optimization Yoni Choukroun Michael Katz 25 1 0 28 Oct 2021
To Raise or Not To Raise: The Autonomous Learning Rate Question Xiaomeng Dong Tao Tan Michael Potter Yun-Chan Tsai Gaurav Kumar V. R. Saripalli Theodore Trafalis OOD 13 2 0 16 Jun 2021
A Generalizable Approach to Learning Optimizers Diogo Almeida Clemens Winter Jie Tang Wojciech Zaremba AI4CE 19 29 0 02 Jun 2021
Reinforced Attention for Few-Shot Learning and Beyond Jie Hong Pengfei Fang Weihao Li Tong Zhang Christian Simon Mehrtash Harandi L. Petersson 12 48 0 09 Apr 2021
Learning to Optimize: A Primer and A Benchmark Tianlong Chen Xiaohan Chen Wuyang Chen Howard Heaton Jialin Liu Zhangyang Wang W. Yin 40 225 0 23 Mar 2021
Learning the Step-size Policy for the Limited-Memory Broyden-Fletcher-Goldfarb-Shanno Algorithm Lucas N. Egidio A. Hansson B. Wahlberg 16 12 0 03 Oct 2020
Tasks, stability, architecture, and compute: Training more effective learned optimizers, and using them to train themselves Luke Metz Niru Maheswaranathan C. Freeman Ben Poole Jascha Narain Sohl-Dickstein 33 62 0 23 Sep 2020