SGD in the Large: Average-case Analysis, Asymptotics, and Stepsize
Criticality

SGD in the Large: Average-case Analysis, Asymptotics, and Stepsize Criticality

8 February 2021

Courtney Paquette

Fabian Pedregosa

Elliot Paquette

Papers citing "SGD in the Large: Average-case Analysis, Asymptotics, and Stepsize Criticality"

12 / 12 papers shown

Title
Estimating Generalization Performance Along the Trajectory of Proximal SGD in Robust Regression Kai Tan Pierre C. Bellec 26 0 0 03 Oct 2024
How Feature Learning Can Improve Neural Scaling Laws Blake Bordelon Alexander B. Atanasov Cengiz Pehlevan 57 12 0 26 Sep 2024
High dimensional analysis reveals conservative sharpening and a stochastic edge of stability Atish Agarwala Jeffrey Pennington 41 3 0 30 Apr 2024
High-dimensional limit of one-pass SGD on least squares Elizabeth Collins-Woodfin Elliot Paquette 36 3 0 13 Apr 2023
High-dimensional scaling limits and fluctuations of online least-squares SGD with smooth covariance Krishnakumar Balasubramanian Promit Ghosal Ye He 38 5 0 03 Apr 2023
Statistical Inference for Linear Functionals of Online SGD in High-dimensional Linear Regression Bhavya Agrawalla Krishnakumar Balasubramanian Promit Ghosal 25 2 0 20 Feb 2023
SAM operates far from home: eigenvalue regularization as a dynamical phenomenon Atish Agarwala Yann N. Dauphin 21 20 0 17 Feb 2023
A Nonstochastic Control Approach to Optimization Xinyi Chen Elad Hazan 47 5 0 19 Jan 2023
High-dimensional limit theorems for SGD: Effective dynamics and critical scaling Gerard Ben Arous Reza Gheissari Aukosh Jagannath 62 58 0 08 Jun 2022
Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics D. Kunin Javier Sagastuy-Breña Surya Ganguli Daniel L. K. Yamins Hidenori Tanaka 107 77 0 08 Dec 2020
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 308 2,890 0 15 Sep 2016
A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights Weijie Su Stephen P. Boyd Emmanuel J. Candes 108 1,157 0 04 Mar 2015