Implicit Regularization or Implicit Conditioning? Exact Risk Trajectories of SGD in High Dimensions

15 June 2022

Papers citing "Implicit Regularization or Implicit Conditioning? Exact Risk Trajectories of SGD in High Dimensions"

8 / 8 papers shown

Title
High dimensional analysis reveals conservative sharpening and a stochastic edge of stability Atish Agarwala Jeffrey Pennington 46 3 0 30 Apr 2024
High-dimensional scaling limits and fluctuations of online least-squares SGD with smooth covariance Krishnakumar Balasubramanian Promit Ghosal Ye He 51 5 0 03 Apr 2023
Statistical Inference for Linear Functionals of Online SGD in High-dimensional Linear Regression Bhavya Agrawalla Krishnakumar Balasubramanian Promit Ghosal 35 2 0 20 Feb 2023
SAM operates far from home: eigenvalue regularization as a dynamical phenomenon Atish Agarwala Yann N. Dauphin 28 20 0 17 Feb 2023
Deep Double Descent via Smooth Interpolation Matteo Gamba Erik Englesson Mårten Björkman Hossein Azizpour 70 11 0 21 Sep 2022
Last Iterate Risk Bounds of SGD with Decaying Stepsize for Overparameterized Linear Regression Jingfeng Wu Difan Zou Vladimir Braverman Quanquan Gu Sham Kakade 109 20 0 12 Oct 2021
The large learning rate phase of deep learning: the catapult mechanism Aitor Lewkowycz Yasaman Bahri Ethan Dyer Jascha Narain Sohl-Dickstein Guy Gur-Ari ODL 159 236 0 04 Mar 2020
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 312 2,896 0 15 Sep 2016