Model, sample, and epoch-wise descents: exact solution of gradient flow in the random feature model

22 October 2021

Papers citing "Model, sample, and epoch-wise descents: exact solution of gradient flow in the random feature model"

30 / 30 papers shown

Title
Learning time-scales in two-layers neural networks Raphael Berthier Andrea Montanari Kangjie Zhou 143 38 0 28 Feb 2023
Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets Alethea Power Yuri Burda Harrison Edwards Igor Babuschkin Vedant Misra 88 364 0 06 Jan 2022
Rank-one matrix estimation: analytic time evolution of gradient descent dynamics A. Bodin N. Macris 44 5 0 25 May 2021
Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem Francesca Mignacco Pierfrancesco Urbani Lenka Zdeborová 71 36 0 08 Mar 2021
The Neural Tangent Kernel in High Dimensions: Triple Descent and a Multi-Scale Theory of Generalization Ben Adlam Jeffrey Pennington 53 125 0 15 Aug 2020
Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification Francesca Mignacco Florent Krzakala Pierfrancesco Urbani Lenka Zdeborová MLT 69 69 0 10 Jun 2020
A Random Matrix Analysis of Random Fourier Features: Beyond the Gaussian Kernel, a Precise Phase Transition, and the Corresponding Double Descent Zhenyu Liao Romain Couillet Michael W. Mahoney 79 93 0 09 Jun 2020
Triple descent and the two kinds of overfitting: Where & why do they appear? Stéphane dÁscoli Levent Sagun Giulio Biroli 49 80 0 05 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 868 42,379 0 28 May 2020
Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regime Stéphane dÁscoli Maria Refinetti Giulio Biroli Florent Krzakala 159 153 0 02 Mar 2020
Generalisation error in learning with random features and the hidden manifold model Federica Gerace Bruno Loureiro Florent Krzakala M. Mézard Lenka Zdeborová 67 172 0 21 Feb 2020
Implicit Regularization of Random Feature Models Arthur Jacot Berfin Simsek Francesco Spadaro Clément Hongler Franck Gabriel 64 83 0 19 Feb 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 611 4,905 0 23 Jan 2020
Exact expressions for double descent and implicit regularization via surrogate random design Michal Derezinski Feynman T. Liang Michael W. Mahoney 65 78 0 10 Dec 2019
Deep Double Descent: Where Bigger Models and More Data Hurt Preetum Nakkiran Gal Kaplun Yamini Bansal Tristan Yang Boaz Barak Ilya Sutskever 123 945 0 04 Dec 2019
A Random Matrix Perspective on Mixtures of Nonlinearities for Deep Learning Ben Adlam J. Levinson Jeffrey Pennington 59 25 0 02 Dec 2019
A Model of Double Descent for High-dimensional Binary Linear Classification Zeyu Deng A. Kammoun Christos Thrampoulidis 85 148 0 13 Nov 2019
The generalization error of random features regression: Precise asymptotics and double descent curve Song Mei Andrea Montanari 97 639 0 14 Aug 2019
Benign Overfitting in Linear Regression Peter L. Bartlett Philip M. Long Gábor Lugosi Alexander Tsigler MLT 88 778 0 26 Jun 2019
Harmless interpolation of noisy data in regression Vidya Muthukumar Kailas Vodrahalli Vignesh Subramanian A. Sahai 82 202 0 21 Mar 2019
Surprises in High-Dimensional Ridgeless Least Squares Interpolation Trevor Hastie Andrea Montanari Saharon Rosset Robert Tibshirani 197 746 0 19 Mar 2019
Two models of double descent for weak features M. Belkin Daniel J. Hsu Ji Xu 106 375 0 18 Mar 2019
Scaling description of generalization with number of parameters in deep learning Mario Geiger Arthur Jacot S. Spigler Franck Gabriel Levent Sagun Stéphane dÁscoli Giulio Biroli Clément Hongler Matthieu Wyart 83 196 0 06 Jan 2019
Reconciling modern machine learning practice and the bias-variance trade-off M. Belkin Daniel J. Hsu Siyuan Ma Soumik Mandal 242 1,655 0 28 Dec 2018
A jamming transition from under- to over-parametrization affects loss landscape and generalization S. Spigler Mario Geiger Stéphane dÁscoli Levent Sagun Giulio Biroli Matthieu Wyart 64 150 0 22 Oct 2018
Does data interpolation contradict statistical optimality? M. Belkin Alexander Rakhlin Alexandre B. Tsybakov 82 220 0 25 Jun 2018
The Dynamics of Learning: A Random Matrix Approach Zhenyu Liao Romain Couillet AI4CE 57 43 0 30 May 2018
To understand deep learning we need to understand kernel learning M. Belkin Siyuan Ma Soumik Mandal 70 420 0 05 Feb 2018
High-dimensional dynamics of generalization error in neural networks Madhu S. Advani Andrew M. Saxe AI4CE 141 472 0 10 Oct 2017
Understanding deep learning requires rethinking generalization Chiyuan Zhang Samy Bengio Moritz Hardt Benjamin Recht Oriol Vinyals HAI 348 4,635 0 10 Nov 2016