High-dimensional dynamics of generalization error in neural networks

10 October 2017

Papers citing "High-dimensional dynamics of generalization error in neural networks"

46 / 296 papers shown

Title
Student Specialization in Deep ReLU Networks With Finite Width and Input Dimension Yuandong Tian MLT 14 8 0 30 Sep 2019
Modelling the influence of data structure on learning in neural networks: the hidden manifold model Sebastian Goldt M. Mézard Florent Krzakala Lenka Zdeborová BDL 23 51 0 25 Sep 2019
The generalization error of random features regression: Precise asymptotics and double descent curve Song Mei Andrea Montanari 51 626 0 14 Aug 2019
Disentangling feature and lazy training in deep neural networks Mario Geiger S. Spigler Arthur Jacot M. Wyart 15 17 0 19 Jun 2019
Dynamics of stochastic gradient descent for two-layer neural networks in the teacher-student setup Sebastian Goldt Madhu S. Advani Andrew M. Saxe Florent Krzakala Lenka Zdeborová MLT 19 140 0 18 Jun 2019
Understanding overfitting peaks in generalization error: Analytical risk curves for $l_2$ and $l_1$ penalized interpolation P. Mitra 18 50 0 09 Jun 2019
Dimensionality compression and expansion in Deep Neural Networks Stefano Recanatesi M. Farrell Madhu S. Advani Timothy Moore Guillaume Lajoie E. Shea-Brown 18 72 0 02 Jun 2019
Implicit Regularization in Deep Matrix Factorization Sanjeev Arora Nadav Cohen Wei Hu Yuping Luo AI4CE 26 491 0 31 May 2019
Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networks Guodong Zhang James Martens Roger C. Grosse ODL 22 124 0 27 May 2019
Meta-learners' learning dynamics are unlike learners' Neil C. Rabinowitz OffRL 23 16 0 03 May 2019
Similarity of Neural Network Representations Revisited Simon Kornblith Mohammad Norouzi Honglak Lee Geoffrey E. Hinton 32 1,354 0 01 May 2019
Implicit Regularization of Discrete Gradient Dynamics in Linear Neural Networks Gauthier Gidel Francis R. Bach Simon Lacoste-Julien AI4CE 6 150 0 30 Apr 2019
Layer Dynamics of Linearised Neural Nets Saurav Basu Koyel Mukherjee Shrihari Vasudevan AI4CE 6 1 0 24 Apr 2019
Surprises in High-Dimensional Ridgeless Least Squares Interpolation Trevor Hastie Andrea Montanari Saharon Rosset R. Tibshirani 31 728 0 19 Mar 2019
Two models of double descent for weak features M. Belkin Daniel J. Hsu Ji Xu 32 375 0 18 Mar 2019
SSN: Learning Sparse Switchable Normalization via SparsestMax Wenqi Shao Jiamin Ren Jingyu Li Ruimao Zhang Yudian Li Xiaogang Wang Ping Luo 21 56 0 09 Mar 2019
Critical initialisation in continuous approximations of binary neural networks G. Stamatescu Federica Gerace C. Lucibello I. Fuss L. White 25 0 0 01 Feb 2019
Numerically Recovering the Critical Points of a Deep Linear Autoencoder Charles G. Frye Neha S. Wadia M. DeWeese K. Bouchard 19 6 0 29 Jan 2019
Generalisation dynamics of online learning in over-parameterised neural networks Sebastian Goldt Madhu S. Advani Andrew M. Saxe Florent Krzakala Lenka Zdeborová 25 14 0 25 Jan 2019
A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks Umut Simsekli Levent Sagun Mert Gurbuzbalaban 17 237 0 18 Jan 2019
Scaling description of generalization with number of parameters in deep learning Mario Geiger Arthur Jacot S. Spigler Franck Gabriel Levent Sagun Stéphane dÁscoli Giulio Biroli Clément Hongler M. Wyart 49 195 0 06 Jan 2019
Reconciling modern machine learning practice and the bias-variance trade-off M. Belkin Daniel J. Hsu Siyuan Ma Soumik Mandal 39 1,610 0 28 Dec 2018
An Empirical Study of Example Forgetting during Deep Neural Network Learning Mariya Toneva Alessandro Sordoni Rémi Tachet des Combes Adam Trischler Yoshua Bengio Geoffrey J. Gordon 46 712 0 12 Dec 2018
Gradient Descent Happens in a Tiny Subspace Guy Gur-Ari Daniel A. Roberts Ethan Dyer 28 228 0 12 Dec 2018
Shared Representational Geometry Across Neural Networks Qihong Lu Po-Hsuan Chen Jonathan W. Pillow Peter J. Ramadge K. A. Norman Uri Hasson OOD 16 11 0 28 Nov 2018
A jamming transition from under- to over-parametrization affects loss landscape and generalization S. Spigler Mario Geiger Stéphane dÁscoli Levent Sagun Giulio Biroli M. Wyart 25 151 0 22 Oct 2018
A Modern Take on the Bias-Variance Tradeoff in Neural Networks Brady Neal Sarthak Mittal A. Baratin Vinayak Tantia Matthew Scicluna Simon Lacoste-Julien Ioannis Mitliagkas 29 167 0 19 Oct 2018
Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning Charles H. Martin Michael W. Mahoney AI4CE 35 190 0 02 Oct 2018
An analytic theory of generalization dynamics and transfer learning in deep linear networks Andrew Kyle Lampinen Surya Ganguli OOD 28 127 0 27 Sep 2018
The jamming transition as a paradigm to understand the loss landscape of deep neural networks Mario Geiger S. Spigler Stéphane dÁscoli Levent Sagun Marco Baity-Jesi Giulio Biroli M. Wyart 22 141 0 25 Sep 2018
On the Learning Dynamics of Deep Neural Networks Rémi Tachet des Combes Mohammad Pezeshki Samira Shabanian Aaron Courville Yoshua Bengio 16 38 0 18 Sep 2018
Towards Understanding Regularization in Batch Normalization Ping Luo Xinjiang Wang Wenqi Shao Zhanglin Peng MLT AI4CE 23 179 0 04 Sep 2018
Generalization Error in Deep Learning Daniel Jakubovitz Raja Giryes M. Rodrigues AI4CE 32 109 0 03 Aug 2018
On the Relation Between the Sharpest Directions of DNN Loss and the SGD Step Length Stanislaw Jastrzebski Zachary Kenton Nicolas Ballas Asja Fischer Yoshua Bengio Amos Storkey ODL 18 114 0 13 Jul 2018
On the Spectral Bias of Neural Networks Nasim Rahaman A. Baratin Devansh Arpit Felix Dräxler Min-Bin Lin Fred Hamprecht Yoshua Bengio Aaron Courville 54 1,390 0 22 Jun 2018
Learning Dynamics of Linear Denoising Autoencoders Arnu Pretorius Steve Kroon Herman Kamper AI4CE 21 25 0 14 Jun 2018
Minnorm training: an algorithm for training over-parameterized deep neural networks Yamini Bansal Madhu S. Advani David D. Cox Andrew M. Saxe ODL 13 18 0 03 Jun 2018
The Dynamics of Learning: A Random Matrix Approach Zhenyu Liao Romain Couillet AI4CE 16 42 0 30 May 2018
Optimal ridge penalty for real-world high-dimensional data can be zero or negative due to the implicit ridge regularization D. Kobak Jonathan Lomond Benoit Sanchez 30 89 0 28 May 2018
Entropy and mutual information in models of deep neural networks Marylou Gabrié Andre Manoel Clément Luneau Jean Barbier N. Macris Florent Krzakala Lenka Zdeborová 30 178 0 24 May 2018
Deep learning generalizes because the parameter-function map is biased towards simple functions Guillermo Valle Pérez Chico Q. Camargo A. Louis MLT AI4CE 16 225 0 22 May 2018
A Study on Overfitting in Deep Reinforcement Learning Chiyuan Zhang Oriol Vinyals Rémi Munos Samy Bengio OffRL OnRL 16 383 0 18 Apr 2018
A high-bias, low-variance introduction to Machine Learning for physicists Pankaj Mehta Marin Bukov Ching-Hao Wang A. G. Day C. Richardson Charles K. Fisher D. Schwab AI4CE 21 866 0 23 Mar 2018
A Walk with SGD Chen Xing Devansh Arpit Christos Tsirigotis Yoshua Bengio 24 118 0 24 Feb 2018
Towards Understanding the Generalization Bias of Two Layer Convolutional Linear Classifiers with Gradient Descent Yifan Wu Barnabás Póczós Aarti Singh MLT 22 8 0 13 Feb 2018
Fix your classifier: the marginal value of training the last weight layer Elad Hoffer Itay Hubara Daniel Soudry 35 101 0 14 Jan 2018