The generalization error of random features regression: Precise asymptotics and double descent curve

14 August 2019

Papers citing "The generalization error of random features regression: Precise asymptotics and double descent curve"

50 / 129 papers shown

Title
Generalisation under gradient descent via deterministic PAC-Bayes Eugenio Clerico Tyler Farghly George Deligiannidis Benjamin Guedj Arnaud Doucet 31 4 0 06 Sep 2022
Information FOMO: The unhealthy fear of missing out on information. A method for removing misleading data for healthier models Ethan Pickering T. Sapsis 24 6 0 27 Aug 2022
Benign, Tempered, or Catastrophic: A Taxonomy of Overfitting Neil Rohit Mallinar James B. Simon Amirhesam Abedsoltan Parthe Pandit M. Belkin Preetum Nakkiran 24 37 0 14 Jul 2022
Regularization-wise double descent: Why it occurs and how to eliminate it Fatih Yilmaz Reinhard Heckel 30 11 0 03 Jun 2022
A Blessing of Dimensionality in Membership Inference through Regularization Jasper Tan Daniel LeJeune Blake Mason Hamid Javadi Richard G. Baraniuk 32 18 0 27 May 2022
Sharp Asymptotics of Kernel Ridge Regression Beyond the Linear Regime Hong Hu Yue M. Lu 53 15 0 13 May 2022
An Equivalence Principle for the Spectrum of Random Inner-Product Kernel Matrices with Polynomial Scalings Yue M. Lu H. Yau 24 24 0 12 May 2022
High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation Jimmy Ba Murat A. Erdogdu Taiji Suzuki Zhichao Wang Denny Wu Greg Yang MLT 40 121 0 03 May 2022
Concentration of Random Feature Matrices in High-Dimensions Zhijun Chen Hayden Schaeffer Rachel A. Ward 22 6 0 14 Apr 2022
On the (Non-)Robustness of Two-Layer Neural Networks in Different Learning Regimes Elvis Dohmatob A. Bietti AAML 32 13 0 22 Mar 2022
Generalization Through The Lens Of Leave-One-Out Error Gregor Bachmann Thomas Hofmann Aurelien Lucchi 52 7 0 07 Mar 2022
Estimation under Model Misspecification with Fake Features Martin Hellkvist Ayça Özçelikkale Anders Ahlén 27 11 0 07 Mar 2022
Contrasting random and learned features in deep Bayesian linear regression Jacob A. Zavatone-Veth William L. Tong Cengiz Pehlevan BDL MLT 28 26 0 01 Mar 2022
HARFE: Hard-Ridge Random Feature Expansion Esha Saha Hayden Schaeffer Giang Tran 38 14 0 06 Feb 2022
Quantifying Relevance in Learning and Inference M. Marsili Y. Roudi 14 18 0 01 Feb 2022
The Effect of Model Size on Worst-Group Generalization Alan Pham Eunice Chan V. Srivatsa Dhruba Ghosh Yaoqing Yang Yaodong Yu Ruiqi Zhong Joseph E. Gonzalez Jacob Steinhardt 23 5 0 08 Dec 2021
Understanding Square Loss in Training Overparametrized Neural Network Classifiers Tianyang Hu Jun Wang Wei Cao Zhenguo Li UQCV AAML 41 19 0 07 Dec 2021
Multi-scale Feature Learning Dynamics: Insights for Double Descent Mohammad Pezeshki Amartya Mitra Yoshua Bengio Guillaume Lajoie 61 25 0 06 Dec 2021
Tight bounds for minimum l1-norm interpolation of noisy data Guillaume Wang Konstantin Donhauser Fanny Yang 79 20 0 10 Nov 2021
Model, sample, and epoch-wise descents: exact solution of gradient flow in the random feature model A. Bodin N. Macris 37 13 0 22 Oct 2021
Conditioning of Random Feature Matrices: Double Descent and Generalization Error Zhijun Chen Hayden Schaeffer 35 12 0 21 Oct 2021
Learning through atypical "phase transitions" in overparameterized neural networks Carlo Baldassi Clarissa Lauditi Enrico M. Malatesta R. Pacelli Gabriele Perugini R. Zecchina 26 26 0 01 Oct 2021
Classification and Adversarial examples in an Overparameterized Linear Model: A Signal Processing Perspective Adhyyan Narang Vidya Muthukumar A. Sahai SILM AAML 36 1 0 27 Sep 2021
Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networks Zhichao Wang Yizhe Zhu 35 18 0 20 Sep 2021
A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning Yehuda Dar Vidya Muthukumar Richard G. Baraniuk 31 71 0 06 Sep 2021
Interpolation can hurt robust generalization even when there is no noise Konstantin Donhauser Alexandru cTifrea Michael Aerni Reinhard Heckel Fanny Yang 34 14 0 05 Aug 2021
Random feature neural networks learn Black-Scholes type PDEs without curse of dimensionality Lukas Gonon 21 35 0 14 Jun 2021
Towards an Understanding of Benign Overfitting in Neural Networks Zhu Li Zhi-Hua Zhou Arthur Gretton MLT 33 35 0 06 Jun 2021
Fundamental tradeoffs between memorization and robustness in random features and neural tangent regimes Elvis Dohmatob 25 9 0 04 Jun 2021
A Universal Law of Robustness via Isoperimetry Sébastien Bubeck Mark Sellke 13 213 0 26 May 2021
A Geometric Analysis of Neural Collapse with Unconstrained Features Zhihui Zhu Tianyu Ding Jinxin Zhou Xiao Li Chong You Jeremias Sulam Qing Qu 27 194 0 06 May 2021
AdaBoost and robust one-bit compressed sensing Geoffrey Chinot Felix Kuchelmeister Matthias Löffler Sara van de Geer 35 5 0 05 May 2021
Generalization Guarantees for Neural Architecture Search with Train-Validation Split Samet Oymak Mingchen Li Mahdi Soltanolkotabi AI4CE OOD 36 13 0 29 Apr 2021
Label-Imbalanced and Group-Sensitive Classification under Overparameterization Ganesh Ramachandra Kini Orestis Paraskevas Samet Oymak Christos Thrampoulidis 27 93 0 02 Mar 2021
Adversarially Robust Kernel Smoothing Jia-Jie Zhu Christina Kouridi Yassine Nemmour Bernhard Schölkopf 28 7 0 16 Feb 2021
Learning curves of generic features maps for realistic datasets with a teacher-student model Bruno Loureiro Cédric Gerbelot Hugo Cui Sebastian Goldt Florent Krzakala M. Mézard Lenka Zdeborová 35 135 0 16 Feb 2021
Appearance of Random Matrix Theory in Deep Learning Nicholas P. Baskerville Diego Granziol J. Keating 15 11 0 12 Feb 2021
Explaining Neural Scaling Laws Yasaman Bahri Ethan Dyer Jared Kaplan Jaehoon Lee Utkarsh Sharma 27 250 0 12 Feb 2021
Fundamental Tradeoffs in Distributionally Adversarial Training M. Mehrabi Adel Javanmard Ryan A. Rossi Anup B. Rao Tung Mai AAML 20 17 0 15 Jan 2021
Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networks Quynh N. Nguyen Marco Mondelli Guido Montúfar 25 81 0 21 Dec 2020
Provable Benefits of Overparameterization in Model Compression: From Double Descent to Pruning Neural Networks Xiangyu Chang Yingcong Li Samet Oymak Christos Thrampoulidis 35 50 0 16 Dec 2020
Sparse sketches with small inversion bias Michal Derezinski Zhenyu Liao Yan Sun Michael W. Mahoney 23 21 0 21 Nov 2020
Gradient Starvation: A Learning Proclivity in Neural Networks Mohammad Pezeshki Sekouba Kaba Yoshua Bengio Aaron Courville Doina Precup Guillaume Lajoie MLT 50 257 0 18 Nov 2020
Underspecification Presents Challenges for Credibility in Modern Machine Learning Alexander DÁmour Katherine A. Heller D. Moldovan Ben Adlam B. Alipanahi ... Kellie Webster Steve Yadlowsky T. Yun Xiaohua Zhai D. Sculley OffRL 77 670 0 06 Nov 2020
Understanding Double Descent Requires a Fine-Grained Bias-Variance Decomposition Ben Adlam Jeffrey Pennington UD 39 93 0 04 Nov 2020
Memorizing without overfitting: Bias, variance, and interpolation in over-parameterized models J. Rocks Pankaj Mehta 18 41 0 26 Oct 2020
Precise High-Dimensional Asymptotics for Quantifying Heterogeneous Transfers Fan Yang Hongyang R. Zhang Sen Wu Christopher Ré Weijie J. Su 58 10 0 22 Oct 2020
Precise Statistical Analysis of Classification Accuracies for Adversarial Training Adel Javanmard Mahdi Soltanolkotabi AAML 28 62 0 21 Oct 2020
On the proliferation of support vectors in high dimensions Daniel J. Hsu Vidya Muthukumar Ji Xu 26 42 0 22 Sep 2020
Multiple Descent: Design Your Own Generalization Curve Lin Chen Yifei Min M. Belkin Amin Karbasi DRL 28 61 0 03 Aug 2020