v1v2 (latest)

Heavy-Tailed Universality Predicts Trends in Test Accuracies for Very Large Pre-Trained Deep Neural Networks

24 January 2019

Papers citing "Heavy-Tailed Universality Predicts Trends in Test Accuracies for Very Large Pre-Trained Deep Neural Networks"

20 / 20 papers shown

Title
Leveraging Gradients for Unsupervised Accuracy Estimation under Distribution Shift Renchunzi Xie Ambroise Odonnat Vasilii Feofanov I. Redko Jianfeng Zhang Bo An UQCV 136 1 0 17 Jan 2024
Traditional and Heavy-Tailed Self Regularization in Neural Network Models Charles H. Martin Michael W. Mahoney 87 126 0 24 Jan 2019
Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning Charles H. Martin Michael W. Mahoney AI4CE 120 201 0 02 Oct 2018
The jamming transition as a paradigm to understand the loss landscape of deep neural networks Mario Geiger S. Spigler Stéphane dÁscoli Levent Sagun Marco Baity-Jesi Giulio Biroli Matthieu Wyart 82 143 0 25 Sep 2018
A Surprising Linear Relationship Predicts Test Performance in Deep Networks Q. Liao Brando Miranda Andrzej Banburski Jack Hidary T. Poggio 61 32 0 25 Jul 2018
Theory IIIb: Generalization in Deep Networks T. Poggio Q. Liao Brando Miranda Andrzej Banburski Xavier Boix Jack Hidary ODL AI4CE 92 26 0 29 Jun 2018
Understanding Generalization and Optimization Performance of Deep CNNs Pan Zhou Jiashi Feng MLT 114 50 0 28 May 2018
On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization Sanjeev Arora Nadav Cohen Elad Hazan 118 487 0 19 Feb 2018
Stronger generalization bounds for deep nets via a compression approach Sanjeev Arora Rong Ge Behnam Neyshabur Yi Zhang MLT AI4CE 89 643 0 14 Feb 2018
The Implicit Bias of Gradient Descent on Separable Data Daniel Soudry Elad Hoffer Mor Shpigel Nacson Suriya Gunasekar Nathan Srebro 168 924 0 27 Oct 2017
Rethinking generalization requires revisiting old ideas: statistical mechanics approaches and complex learning behavior Charles H. Martin Michael W. Mahoney AI4CE 69 64 0 26 Oct 2017
Generalization in Deep Learning Kenji Kawaguchi L. Kaelbling Yoshua Bengio ODL 133 460 0 16 Oct 2017
A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks Behnam Neyshabur Srinadh Bhojanapalli Nathan Srebro 88 610 0 29 Jul 2017
Exploring Generalization in Deep Learning Behnam Neyshabur Srinadh Bhojanapalli David A. McAllester Nathan Srebro FAtt 162 1,259 0 27 Jun 2017
Spectrally-normalized margin bounds for neural networks Peter L. Bartlett Dylan J. Foster Matus Telgarsky ODL 212 1,225 0 26 Jun 2017
Spectral Norm Regularization for Improving the Generalizability of Deep Learning Yuichi Yoshida Takeru Miyato 83 335 0 31 May 2017
Norm-Based Capacity Control in Neural Networks Behnam Neyshabur Ryota Tomioka Nathan Srebro 292 591 0 27 Feb 2015
In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning Behnam Neyshabur Ryota Tomioka Nathan Srebro AI4CE 99 663 0 20 Dec 2014
Limit Theory for the largest eigenvalues of sample covariance matrices with heavy-tails Richard A. Davis Oliver Pfaffel R. Stelzer 111 37 0 27 Aug 2011
Learning with Spectral Kernels and Heavy-Tailed Data Michael W. Mahoney Hariharan Narayanan 95 3 0 24 Jun 2009