The Full Spectrum of Deepnet Hessians at Scale: Dynamics with SGD Training and Sample Size

16 November 2018

Papers citing "The Full Spectrum of Deepnet Hessians at Scale: Dynamics with SGD Training and Sample Size"

14 / 14 papers shown

Title
Symmetric Neural-Collapse Representations with Supervised Contrastive Loss: The Impact of ReLU and Batching Ganesh Ramachandra Kini V. Vakilian Tina Behnia Jaidev Gill Christos Thrampoulidis 22 1 0 13 Jun 2023
MIO : Mutual Information Optimization using Self-Supervised Binary Contrastive Learning Siladittya Manna Umapada Pal Saumik Bhattacharya SSL 35 1 0 24 Nov 2021
A Deeper Look at the Hessian Eigenspectrum of Deep Neural Networks and its Applications to Regularization Adepu Ravi Sankar Yash Khasbage Rahul Vigneswaran V. Balasubramanian 25 42 0 07 Dec 2020
Prevalence of Neural Collapse during the terminal phase of deep learning training Vardan Papyan Xuemei Han D. Donoho 35 554 0 18 Aug 2020
Exploring Weight Importance and Hessian Bias in Model Pruning Mingchen Li Yahya Sattar Christos Thrampoulidis Samet Oymak 30 3 0 19 Jun 2020
Learning Rates as a Function of Batch Size: A Random Matrix Theory Approach to Neural Network Training Diego Granziol S. Zohren Stephen J. Roberts ODL 39 49 0 16 Jun 2020
Geometry of learning neural quantum states Chae-Yeun Park M. Kastoryano 32 60 0 24 Oct 2019
GradVis: Visualization and Second Order Analysis of Optimization Surfaces during the Training of Deep Neural Networks Avraam Chatzimichailidis Franz-Josef Pfreundt N. Gauger J. Keuper 23 10 0 26 Sep 2019
Hessian based analysis of SGD for Deep Nets: Dynamics and Generalization Xinyan Li Qilong Gu Yingxue Zhou Tiancong Chen A. Banerjee ODL 42 51 0 24 Jul 2019
First Exit Time Analysis of Stochastic Gradient Descent Under Heavy-Tailed Gradient Noise T. H. Nguyen Umut Simsekli Mert Gurbuzbalaban G. Richard 12 60 0 21 Jun 2019
Negative eigenvalues of the Hessian in deep neural networks Guillaume Alain Nicolas Le Roux Pierre-Antoine Manzagol 24 42 0 06 Feb 2019
An Investigation into Neural Net Optimization via Hessian Eigenvalue Density Behrooz Ghorbani Shankar Krishnan Ying Xiao ODL 18 317 0 29 Jan 2019
Ambitious Data Science Can Be Painless Hatef Monajemi Riccardo Murri Eric Jonas Percy Liang V. Stodden D. Donoho 29 13 0 25 Jan 2019
Measurements of Three-Level Hierarchical Structure in the Outliers in the Spectrum of Deepnet Hessians Vardan Papyan 30 87 0 24 Jan 2019