The generalization error of random features regression: Precise asymptotics and double descent curve

14 August 2019

Papers citing "The generalization error of random features regression: Precise asymptotics and double descent curve"

50 / 125 papers shown

Title
Information-theoretic reduction of deep neural networks to linear models in the overparametrized proportional regime Francesco Camilli D. Tieplova Eleonora Bergamin Jean Barbier 114 0 0 06 May 2025
A dynamic view of the double descent Vivek Shripad Borkar 65 0 0 03 May 2025
Online Federation For Mixtures of Proprietary Agents with Black-Box Encoders Xuwei Yang Fatemeh Tavakoli D. B. Emerson Anastasis Kratsios FedML 62 0 0 30 Apr 2025
Deep learning with missing data Tianyi Ma Tengyao Wang R. Samworth 66 0 0 21 Apr 2025
Quantifying Overfitting along the Regularization Path for Two-Part-Code MDL in Supervised Classification Xiaohan Zhu Nathan Srebro 55 0 0 03 Mar 2025
Asymptotic Analysis of Two-Layer Neural Networks after One Gradient Step under Gaussian Mixtures Data with Structure Samet Demir Zafer Dogan MLT 34 0 0 02 Mar 2025
Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer Blake Bordelon C. Pehlevan AI4CE 64 1 0 04 Feb 2025
How more data can hurt: Instability and regularization in next-generation reservoir computing Yuanzhao Zhang Edmilson Roque dos Santos Sean P. Cornelius 85 2 0 28 Jan 2025
Analysis of High-dimensional Gaussian Labeled-unlabeled Mixture Model via Message-passing Algorithm Xiaosi Gu Tomoyuki Obuchi 74 0 0 29 Nov 2024
High-dimensional Analysis of Knowledge Distillation: Weak-to-Strong Generalization and Scaling Laws M. E. Ildiz Halil Alperen Gozeten Ege Onur Taga Marco Mondelli Samet Oymak 56 2 0 24 Oct 2024
Provable Weak-to-Strong Generalization via Benign Overfitting David X. Wu A. Sahai 73 6 0 06 Oct 2024
LoRanPAC: Low-rank Random Features and Pre-trained Models for Bridging Theory and Practice in Continual Learning Liangzu Peng Juan Elenter Joshua Agterberg Alejandro Ribeiro René Vidal VLM CLL 46 1 0 01 Oct 2024
Asymptotic theory of in-context learning by linear attention Yue M. Lu Mary I. Letey Jacob A. Zavatone-Veth Anindita Maiti C. Pehlevan 29 10 0 20 May 2024
Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks Fanghui Liu L. Dadi V. Cevher 82 2 0 29 Apr 2024
Understanding Optimal Feature Transfer via a Fine-Grained Bias-Variance Analysis Yufan Li Subhabrata Sen Ben Adlam MLT 51 1 0 18 Apr 2024
Predictive Churn with the Set of Good Models J. Watson-Daniels Flavio du Pin Calmon Alexander DÁmour Carol Xuan Long David C. Parkes Berk Ustun 83 7 0 12 Feb 2024
Characterizing Overfitting in Kernel Ridgeless Regression Through the Eigenspectrum Tin Sum Cheng Aurelien Lucchi Anastasis Kratsios David Belius 40 8 0 02 Feb 2024
Weak Correlations as the Underlying Principle for Linearization of Gradient-Based Learning Systems Ori Shem-Ur Yaron Oz 19 0 0 08 Jan 2024
Flat Minima in Linear Estimation and an Extended Gauss Markov Theorem Simon Segert 32 0 0 18 Nov 2023
A Theory of Non-Linear Feature Learning with One Gradient Step in Two-Layer Neural Networks Behrad Moniri Donghwan Lee Hamed Hassani Yan Sun MLT 40 19 0 11 Oct 2023
Quantitative CLTs in Deep Neural Networks Stefano Favaro Boris Hanin Domenico Marinucci I. Nourdin G. Peccati BDL 31 11 0 12 Jul 2023
Precise Asymptotic Generalization for Multiclass Classification with Overparameterized Linear Models David X. Wu A. Sahai 26 2 0 23 Jun 2023
Quantifying lottery tickets under label noise: accuracy, calibration, and complexity V. Arora Daniele Irto Sebastian Goldt G. Sanguinetti 36 2 0 21 Jun 2023
The RL Perceptron: Generalisation Dynamics of Policy Learning in High Dimensions Nishil Patel Sebastian Lee Stefano Sarao Mannelli Sebastian Goldt Adrew Saxe OffRL 28 3 0 17 Jun 2023
Gibbs-Based Information Criteria and the Over-Parameterized Regime Haobo Chen Yuheng Bu Greg Wornell 27 1 0 08 Jun 2023
Unraveling Projection Heads in Contrastive Learning: Insights from Expansion and Shrinkage Yu Gui Cong Ma Yiqiao Zhong 22 6 0 06 Jun 2023
Generalized equivalences between subsampling and ridge regularization Pratik V. Patil Jin-Hong Du 31 5 0 29 May 2023
Moment-Based Adjustments of Statistical Inference in High-Dimensional Generalized Linear Models Kazuma Sawaya Yoshimasa Uematsu Masaaki Imaizumi 29 2 0 28 May 2023
Bayesian Analysis for Over-parameterized Linear Model via Effective Spectra Tomoya Wakayama Masaaki Imaizumi 57 1 0 25 May 2023
How Spurious Features Are Memorized: Precise Analysis for Random and NTK Features Simone Bombari Marco Mondelli AAML 28 4 0 20 May 2023
Subsample Ridge Ensembles: Equivalences and Generalized Cross-Validation Jin-Hong Du Pratik V. Patil Arun K. Kuchibhotla 24 11 0 25 Apr 2023
Do deep neural networks have an inbuilt Occam's razor? Chris Mingard Henry Rees Guillermo Valle Pérez A. Louis UQCV BDL 21 16 0 13 Apr 2023
Wide neural networks: From non-gaussian random fields at initialization to the NTK geometry of training Luís Carvalho Joao L. Costa José Mourao Gonccalo Oliveira AI4CE 26 1 0 06 Apr 2023
On the Stepwise Nature of Self-Supervised Learning James B. Simon Maksis Knutins Liu Ziyin Daniel Geisz Abraham J. Fetterman Joshua Albrecht SSL 37 30 0 27 Mar 2023
Online Learning for the Random Feature Model in the Student-Teacher Framework Roman Worschech B. Rosenow 46 0 0 24 Mar 2023
Over-parametrization via Lifting for Low-rank Matrix Sensing: Conversion of Spurious Solutions to Strict Saddle Points Ziye Ma Igor Molybog Javad Lavaei Somayeh Sojoudi 25 3 0 15 Feb 2023
Precise Asymptotic Analysis of Deep Random Feature Models David Bosch Ashkan Panahi B. Hassibi 35 19 0 13 Feb 2023
Beyond the Universal Law of Robustness: Sharper Laws for Random Features and Neural Tangent Kernels Simone Bombari Shayan Kiyani Marco Mondelli AAML 36 10 0 03 Feb 2023
Bayes-optimal Learning of Deep Random Networks of Extensive-width Hugo Cui Florent Krzakala Lenka Zdeborová BDL 20 35 0 01 Feb 2023
Demystifying Disagreement-on-the-Line in High Dimensions Dong-Hwan Lee Behrad Moniri Xinmeng Huang Yan Sun Hamed Hassani 21 8 0 31 Jan 2023
Generalization on the Unseen, Logic Reasoning and Degree Curriculum Emmanuel Abbe Samy Bengio Aryo Lotfi Kevin Rizk LRM 39 49 0 30 Jan 2023
Gradient flow in the gaussian covariate model: exact solution of learning curves and multiple descent structures Antione Bodin N. Macris 34 4 0 13 Dec 2022
High Dimensional Binary Classification under Label Shift: Phase Transition and Regularization Jiahui Cheng Minshuo Chen Hao Liu Tuo Zhao Wenjing Liao 36 0 0 01 Dec 2022
Understanding the double descent curve in Machine Learning Luis Sa-Couto J. M. Ramos Miguel Almeida Andreas Wichert 29 1 0 18 Nov 2022
Designing Universal Causal Deep Learning Models: The Case of Infinite-Dimensional Dynamical Systems from Stochastic Analysis Luca Galimberti Anastasis Kratsios Giulia Livieri OOD 28 14 0 24 Oct 2022
On the Impossible Safety of Large AI Models El-Mahdi El-Mhamdi Sadegh Farhadkhani R. Guerraoui Nirupam Gupta L. Hoang Rafael Pinot Sébastien Rouault John Stephan 30 31 0 30 Sep 2022
Neural Networks Efficiently Learn Low-Dimensional Representations with SGD Alireza Mousavi-Hosseini Sejun Park M. Girotti Ioannis Mitliagkas Murat A. Erdogdu MLT 324 48 0 29 Sep 2022
Deep Linear Networks can Benignly Overfit when Shallow Ones Do Niladri S. Chatterji Philip M. Long 23 8 0 19 Sep 2022
Importance Tempering: Group Robustness for Overparameterized Models Yiping Lu Wenlong Ji Zachary Izzo Lexing Ying 42 7 0 19 Sep 2022
Generalisation under gradient descent via deterministic PAC-Bayes Eugenio Clerico Tyler Farghly George Deligiannidis Benjamin Guedj Arnaud Doucet 31 4 0 06 Sep 2022