High-dimensional dynamics of generalization error in neural networks

10 October 2017

Papers citing "High-dimensional dynamics of generalization error in neural networks"

50 / 296 papers shown

Title
On the geometry of generalization and memorization in deep neural networks Cory Stephenson Suchismita Padhy Abhinav Ganesh Yue Hui Hanlin Tang SueYeon Chung TDI AI4CE 16 73 0 30 May 2021
A Theory of Neural Tangent Kernel Alignment and Its Influence on Training H. Shan Blake Bordelon 11 11 0 29 May 2021
Towards Understanding the Condensation of Neural Networks at Initial Training Hanxu Zhou Qixuan Zhou Tao Luo Yaoyu Zhang Z. Xu MLT AI4CE 19 26 0 25 May 2021
Relative stability toward diffeomorphisms indicates performance in deep nets Leonardo Petrini Alessandro Favero Mario Geiger M. Wyart OOD 38 15 0 06 May 2021
The Geometry of Over-parameterized Regression and Adversarial Perturbations J. Rocks Pankaj Mehta AAML 11 8 0 25 Mar 2021
The Shape of Learning Curves: a Review T. Viering Marco Loog 18 122 0 19 Mar 2021
The Low-Rank Simplicity Bias in Deep Networks Minyoung Huh H. Mobahi Richard Y. Zhang Brian Cheung Pulkit Agrawal Phillip Isola 22 109 0 18 Mar 2021
On the interplay between data structure and loss function in classification problems Stéphane dÁscoli Marylou Gabrié Levent Sagun Giulio Biroli 29 17 0 09 Mar 2021
On the Generalization Power of Overfitted Two-Layer Neural Tangent Kernel Models Peizhong Ju Xiaojun Lin Ness B. Shroff MLT 24 10 0 09 Mar 2021
Asymptotics of Ridge Regression in Convolutional Models Mojtaba Sahraee-Ardakan Tung Mai Anup B. Rao Ryan Rossi S. Rangan A. Fletcher MLT 16 2 0 08 Mar 2021
Exact Gap between Generalization Error and Uniform Convergence in Random Feature Models Zitong Yang Yu Bai Song Mei 16 17 0 08 Mar 2021
Asymptotic Risk of Overparameterized Likelihood Models: Double Descent Theory for Deep Neural Networks Ryumei Nakada Masaaki Imaizumi 21 2 0 28 Feb 2021
Two-way kernel matrix puncturing: towards resource-efficient PCA and spectral clustering Romain Couillet Florent Chatelain N. L. Bihan 19 8 0 24 Feb 2021
Implicit Regularization in Tensor Factorization Noam Razin Asaf Maman Nadav Cohen 27 48 0 19 Feb 2021
Double-descent curves in neural networks: a new perspective using Gaussian processes Ouns El Harzli Bernardo Cuenca Grau Guillermo Valle Pérez A. Louis 15 6 0 14 Feb 2021
Learning by Turning: Neural Architecture Aware Optimisation Yang Liu Jeremy Bernstein M. Meister Yisong Yue ODL 41 26 0 14 Feb 2021
Distilling Double Descent Andrew Cotter A. Menon Harikrishna Narasimhan A. S. Rawat Sashank J. Reddi Yichen Zhou 25 7 0 13 Feb 2021
Explaining Neural Scaling Laws Yasaman Bahri Ethan Dyer Jared Kaplan Jaehoon Lee Utkarsh Sharma 27 250 0 12 Feb 2021
Meta-learning with negative learning rates A. Bernacchia 17 17 0 01 Feb 2021
A Statistician Teaches Deep Learning G. Babu David L. Banks Hyunsoo Cho David Han Hailin Sang Shouyi Wang 20 2 0 29 Jan 2021
Self-Adaptive Training: Bridging Supervised and Self-Supervised Learning Lang Huang Chaoning Zhang Hongyang R. Zhang SSL 33 24 0 21 Jan 2021
On Data-Augmentation and Consistency-Based Semi-Supervised Learning Atin Ghosh Alexandre Hoang Thiery 73 20 0 18 Jan 2021
Phases of learning dynamics in artificial neural networks: with or without mislabeled data Yu Feng Y. Tu 25 2 0 16 Jan 2021
Perspective: A Phase Diagram for Deep Learning unifying Jamming, Feature Learning and Lazy Training Mario Geiger Leonardo Petrini M. Wyart DRL 23 11 0 30 Dec 2020
Analysis of the Scalability of a Deep-Learning Network for Steganography "Into the Wild" Hugo Ruiz Marc Chaumont Mehdi Yedroudj A. Amara Frédéric Comby Gérard Subsol 21 9 0 29 Dec 2020
Avoiding The Double Descent Phenomenon of Random Feature Models Using Hybrid Regularization Kelvin K. Kan J. Nagy Lars Ruthotto AI4CE 37 6 0 11 Dec 2020
Statistical Mechanics of Deep Linear Neural Networks: The Back-Propagating Kernel Renormalization Qianyi Li H. Sompolinsky 16 69 0 07 Dec 2020
Align, then memorise: the dynamics of learning with feedback alignment Maria Refinetti Stéphane dÁscoli Ruben Ohana Sebastian Goldt 26 36 0 24 Nov 2020
Gradient Starvation: A Learning Proclivity in Neural Networks Mohammad Pezeshki Sekouba Kaba Yoshua Bengio Aaron Courville Doina Precup Guillaume Lajoie MLT 50 257 0 18 Nov 2020
Understanding Double Descent Requires a Fine-Grained Bias-Variance Decomposition Ben Adlam Jeffrey Pennington UD 37 93 0 04 Nov 2020
Memorizing without overfitting: Bias, variance, and interpolation in over-parameterized models J. Rocks Pankaj Mehta 13 41 0 26 Oct 2020
A Dynamical View on Optimization Algorithms of Overparameterized Neural Networks Zhiqi Bu Shiyun Xu Kan Chen 25 17 0 25 Oct 2020
What causes the test error? Going beyond bias-variance via ANOVA Licong Lin Yan Sun 22 34 0 11 Oct 2020
On the Universality of the Double Descent Peak in Ridgeless Regression David Holzmüller 10 12 0 05 Oct 2020
Small Data, Big Decisions: Model Selection in the Small-Data Regime J. Bornschein Francesco Visin Simon Osindero 13 36 0 26 Sep 2020
Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don't E. Weinan Chao Ma Stephan Wojtowytsch Lei Wu AI4CE 22 133 0 22 Sep 2020
Distributional Generalization: A New Kind of Generalization Preetum Nakkiran Yamini Bansal OOD 21 41 0 17 Sep 2020
Asymptotics of Wide Convolutional Neural Networks Anders Andreassen Ethan Dyer 14 22 0 19 Aug 2020
The Neural Tangent Kernel in High Dimensions: Triple Descent and a Multi-Scale Theory of Generalization Ben Adlam Jeffrey Pennington 15 124 0 15 Aug 2020
Provable More Data Hurt in High Dimensional Least Squares Estimator Zeng Li Chuanlong Xie Qinwen Wang 15 6 0 14 Aug 2020
The Slow Deterioration of the Generalization Error of the Random Feature Model Chao Ma Lei Wu E. Weinan 20 15 0 13 Aug 2020
Shallow Univariate ReLu Networks as Splines: Initialization, Loss Surface, Hessian, & Gradient Flow Dynamics Justin Sahs Ryan Pyle Aneel Damaraju J. O. Caro Onur Tavaslioglu Andy Lu Ankit B. Patel 18 19 0 04 Aug 2020
Multiple Descent: Design Your Own Generalization Curve Lin Chen Yifei Min M. Belkin Amin Karbasi DRL 23 61 0 03 Aug 2020
Finite Versus Infinite Neural Networks: an Empirical Study Jaehoon Lee S. Schoenholz Jeffrey Pennington Ben Adlam Lechao Xiao Roman Novak Jascha Narain Sohl-Dickstein 17 207 0 31 Jul 2020
Geometric compression of invariant manifolds in neural nets J. Paccolat Leonardo Petrini Mario Geiger Kevin Tyloo M. Wyart MLT 55 34 0 22 Jul 2020
Early Stopping in Deep Networks: Double Descent and How to Eliminate it Reinhard Heckel Fatih Yilmaz 26 43 0 20 Jul 2020
Data-driven effective model shows a liquid-like deep learning Wenxuan Zou Haiping Huang 24 2 0 16 Jul 2020
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding Dmitry Lepikhin HyoukJoong Lee Yuanzhong Xu Dehao Chen Orhan Firat Yanping Huang M. Krikun Noam M. Shazeer Z. Chen MoE 20 1,106 0 30 Jun 2020
Statistical Mechanical Analysis of Neural Network Pruning Rupam Acharyya Ankani Chattoraj Boyu Zhang Shouman Das Daniel Stefankovic 24 0 0 30 Jun 2020
The Gaussian equivalence of generative models for learning with shallow neural networks Sebastian Goldt Bruno Loureiro Galen Reeves Florent Krzakala M. Mézard Lenka Zdeborová BDL 41 100 0 25 Jun 2020