Non-Gaussianity of Stochastic Gradient Noise

Non-Gaussianity of Stochastic Gradient Noise

21 October 2019

Praneeth Netrapalli

Papers citing "Non-Gaussianity of Stochastic Gradient Noise"

12 / 12 papers shown

Title
Distributed Stochastic Gradient Descent with Staleness: A Stochastic Delay Differential Equation Based Framework Siyuan Yu Wei Chen H. V. Poor 32 0 0 17 Jun 2024
An SDE for Modeling SAM: Theory and Insights Enea Monzio Compagnoni Luca Biggio Antonio Orvieto F. Proske Hans Kersting Aurelien Lucchi 23 13 0 19 Jan 2023
On the Overlooked Structure of Stochastic Gradients Zeke Xie Qian-Yuan Tang Mingming Sun P. Li 31 6 0 05 Dec 2022
Two Facets of SDE Under an Information-Theoretic Lens: Generalization of SGD via Training Trajectories and via Terminal States Ziqiao Wang Yongyi Mao 30 10 0 19 Nov 2022
Taming Fat-Tailed ("Heavier-Tailed'' with Potentially Infinite Variance) Noise in Federated Learning Haibo Yang Pei-Yuan Qiu Jia Liu FedML 27 12 0 03 Oct 2022
Heavy-Tail Phenomenon in Decentralized SGD Mert Gurbuzbalaban Yuanhan Hu Umut Simsekli Kun Yuan Lingjiong Zhu 38 8 0 13 May 2022
Anticorrelated Noise Injection for Improved Generalization Antonio Orvieto Hans Kersting F. Proske Francis R. Bach Aurelien Lucchi 55 44 0 06 Feb 2022
On the Power-Law Hessian Spectrums in Deep Learning Zeke Xie Qian-Yuan Tang Yunfeng Cai Mingming Sun P. Li ODL 42 9 0 31 Jan 2022
Biases in In Silico Evaluation of Molecular Optimization Methods and Bias-Reduced Evaluation Methodology Hiroshi Kajino Kohei Miyaguchi Takayuki Osogami 59 1 0 28 Jan 2022
Exponential escape efficiency of SGD from sharp minima in non-stationary regime Hikaru Ibayashi Masaaki Imaizumi 31 4 0 07 Nov 2021
Improved generalization by noise enhancement Takashi Mori Masahito Ueda 18 3 0 28 Sep 2020
Hausdorff Dimension, Heavy Tails, and Generalization in Neural Networks Umut Simsekli Ozan Sener George Deligiannidis Murat A. Erdogdu 44 55 0 16 Jun 2020