A PDE-based Explanation of Extreme Numerical Sensitivities and Edge of Stability in Training Neural Networks

4 June 2022

Papers citing "A PDE-based Explanation of Extreme Numerical Sensitivities and Edge of Stability in Training Neural Networks"

5 / 5 papers shown

Title
Understanding Edge-of-Stability Training Dynamics with a Minimalist Example Xingyu Zhu Zixuan Wang Xiang Wang Mo Zhou Rong Ge 66 35 0 07 Oct 2022
Limitations of neural network training due to numerical instability of backpropagation Clemens Karner V. Kazeev P. Petersen 37 3 0 03 Oct 2022
Understanding Gradient Descent on Edge of Stability in Deep Learning Sanjeev Arora Zhiyuan Li A. Panigrahi MLT 83 90 0 19 May 2022
Channel-Directed Gradients for Optimization of Convolutional Neural Networks Dong Lao Peihao Zhu Peter Wonka G. Sundaramoorthi 40 3 0 25 Aug 2020
A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights Weijie Su Stephen P. Boyd Emmanuel J. Candes 108 1,157 0 04 Mar 2015