Good regularity creates large learning rate implicit biases: edge of stability, balancing, and catapult

26 October 2023

Papers citing "Good regularity creates large learning rate implicit biases: edge of stability, balancing, and catapult"

8 / 8 papers shown

Title
Minimax Optimal Convergence of Gradient Descent in Logistic Regression via Large and Adaptive Stepsizes Ruiqi Zhang Jingfeng Wu Licong Lin Peter L. Bartlett 30 0 0 05 Apr 2025
A Minimalist Example of Edge-of-Stability and Progressive Sharpening Liming Liu Zixuan Zhang S. Du T. Zhao 79 0 0 04 Mar 2025
Universal Sharpness Dynamics in Neural Network Training: Fixed Point Analysis, Edge of Stability, and Route to Chaos Dayal Singh Kalra Tianyu He M. Barkeshli 52 4 0 17 Feb 2025
Provable Acceleration of Nesterov's Accelerated Gradient for Rectangular Matrix Factorization and Linear Neural Networks Zhenghao Xu Yuqing Wang T. Zhao Rachel Ward Molei Tao 29 0 0 12 Oct 2024
Large Stepsize Gradient Descent for Non-Homogeneous Two-Layer Networks: Margin Improvement and Fast Optimization Yuhang Cai Jingfeng Wu Song Mei Michael Lindsey Peter L. Bartlett 34 2 0 12 Jun 2024
Gradient Descent on Logistic Regression with Non-Separable Data and Large Step Sizes Si Yi Meng Antonio Orvieto Daniel Yiming Cao Christopher De Sa 32 1 0 07 Jun 2024
Deep Support Vectors Junhoo Lee Hyunho Lee Kyomin Hwang Nojun Kwak 46 0 0 26 Mar 2024
Gradient Descent with Polyak's Momentum Finds Flatter Minima via Large Catapults Prin Phunyaphibarn Junghyun Lee Bohan Wang Huishuai Zhang Chulhee Yun 23 0 0 25 Nov 2023