Fluctuation-dissipation relations for stochastic gradient descent

28 September 2018

Papers citing "Fluctuation-dissipation relations for stochastic gradient descent"

20 / 20 papers shown

Title
Formation of Representations in Neural Networks Liu Ziyin Isaac Chuang Tomer Galanti T. Poggio 39 4 0 03 Oct 2024
Dynamics of Supervised and Reinforcement Learning in the Non-Linear Perceptron Christian Schmid James M. Murray 40 0 0 05 Sep 2024
How Neural Networks Learn the Support is an Implicit Regularization Effect of SGD Pierfrancesco Beneventano Andrea Pinto Tomaso A. Poggio MLT 32 1 0 17 Jun 2024
Correlated Noise in Epoch-Based Stochastic Gradient Descent: Implications for Weight Variances Marcel Kühn B. Rosenow 19 3 0 08 Jun 2023
Machine learning in and out of equilibrium Shishir Adhikari Alkan Kabakcciouglu A. Strang Deniz Yuret M. Hinczewski 22 4 0 06 Jun 2023
Climate Intervention Analysis using AI Model Guided by Statistical Physics Principles S. K. Kim Kalai Ramea Salva Rühling Cachay H. Hirasawa Subhashis Hazarika D. Hingmire Peetak Mitra P. Rasch Hansi K. A. Singh AI4CE 35 0 0 07 Feb 2023
On a continuous time model of gradient descent dynamics and instability in deep learning Mihaela Rosca Yan Wu Chongli Qin Benoit Dherin 18 6 0 03 Feb 2023
Taming Fat-Tailed ("Heavier-Tailed'' with Potentially Infinite Variance) Noise in Federated Learning Haibo Yang Pei-Yuan Qiu Jia Liu FedML 29 12 0 03 Oct 2022
Shift-Curvature, SGD, and Generalization Arwen V. Bradley C. Gomez-Uribe Manish Reddy Vuyyuru 35 2 0 21 Aug 2021
The Limiting Dynamics of SGD: Modified Loss, Phase Space Oscillations, and Anomalous Diffusion D. Kunin Javier Sagastuy-Breña Lauren Gillespie Eshed Margalit Hidenori Tanaka Surya Ganguli Daniel L. K. Yamins 31 15 0 19 Jul 2021
Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms A. Camuto George Deligiannidis Murat A. Erdogdu Mert Gurbuzbalaban Umut cSimcsekli Lingjiong Zhu 33 29 0 09 Jun 2021
How to decay your learning rate Aitor Lewkowycz 41 24 0 23 Mar 2021
SVRG Meets AdaGrad: Painless Variance Reduction Benjamin Dubois-Taine Sharan Vaswani Reza Babanezhad Mark W. Schmidt Simon Lacoste-Julien 18 18 0 18 Feb 2021
Shape Matters: Understanding the Implicit Bias of the Noise Covariance Jeff Z. HaoChen Colin Wei J. Lee Tengyu Ma 29 93 0 15 Jun 2020
Statistical Adaptive Stochastic Gradient Methods Pengchuan Zhang Hunter Lang Qiang Liu Lin Xiao ODL 15 11 0 25 Feb 2020
The Early Phase of Neural Network Training Jonathan Frankle D. Schwab Ari S. Morcos 19 171 0 24 Feb 2020
Understanding the Role of Momentum in Stochastic Gradient Methods Igor Gitman Hunter Lang Pengchuan Zhang Lin Xiao 33 94 0 30 Oct 2019
From complex to simple : hierarchical free-energy landscape renormalized in deep neural networks H. Yoshino 14 6 0 22 Oct 2019
Measurements of Three-Level Hierarchical Structure in the Outliers in the Spectrum of Deepnet Hessians Vardan Papyan 22 87 0 24 Jan 2019
A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks Umut Simsekli Levent Sagun Mert Gurbuzbalaban 20 237 0 18 Jan 2019