ResearchTrend.AI
The generalization error of random features regression: Precise asymptotics and double descent curve
v1v2v3v4v5 (latest)

The generalization error of random features regression: Precise asymptotics and double descent curve

Song Mei
Andrea Montanari
ArXiv (abs)PDFHTML

Papers citing "The generalization error of random features regression: Precise asymptotics and double descent curve"

50 / 227 papers shown
Title
Eigenspectrum Analysis of Neural Networks without Aspect Ratio Bias
Eigenspectrum Analysis of Neural Networks without Aspect Ratio Bias
Yuanzhe Hu
Kinshuk Goel
Vlad Killiakov
Yaoqing Yang
56
2
0
06 Jun 2025
Models of Heavy-Tailed Mechanistic Universality
Models of Heavy-Tailed Mechanistic Universality
Liam Hodgkinson
Zhichao Wang
Michael W. Mahoney
63
1
0
04 Jun 2025
On the Emergence of Weak-to-Strong Generalization: A Bias-Variance Perspective
On the Emergence of Weak-to-Strong Generalization: A Bias-Variance Perspective
Gengze Xu
Wei Yao
Ziqiao Wang
Yong Liu
30
0
0
30 May 2025
Statistical mechanics of extensive-width Bayesian neural networks near interpolation
Statistical mechanics of extensive-width Bayesian neural networks near interpolation
Jean Barbier
Francesco Camilli
Minh-Toan Nguyen
Mauro Pastore
Rudy Skerk
28
0
0
30 May 2025
Directional Convergence, Benign Overfitting of Gradient Descent in leaky ReLU two-layer Neural Networks
Directional Convergence, Benign Overfitting of Gradient Descent in leaky ReLU two-layer Neural Networks
Ichiro Hashimoto
MLT
79
0
0
22 May 2025
Deep learning with missing data
Deep learning with missing data
Tianyi Ma
Tengyao Wang
R. Samworth
266
1
0
21 Apr 2025
Quantifying Overfitting along the Regularization Path for Two-Part-Code MDL in Supervised Classification
Xiaohan Zhu
Nathan Srebro
110
0
0
03 Mar 2025
Asymptotic Analysis of Two-Layer Neural Networks after One Gradient Step under Gaussian Mixtures Data with Structure
Asymptotic Analysis of Two-Layer Neural Networks after One Gradient Step under Gaussian Mixtures Data with Structure
Samet Demir
Zafer Dogan
MLT
92
0
0
02 Mar 2025
Correlating and Predicting Human Evaluations of Language Models from Natural Language Processing Benchmarks
Correlating and Predicting Human Evaluations of Language Models from Natural Language Processing Benchmarks
Rylan Schaeffer
Punit Singh Koura
Binh Tang
R. Subramanian
Aaditya K. Singh
...
Vedanuj Goswami
Sergey Edunov
Dieuwke Hupkes
Sanmi Koyejo
Sharan Narang
ALM
144
1
0
24 Feb 2025
Feature maps for the Laplacian kernel and its generalizations
Feature maps for the Laplacian kernel and its generalizations
Sudhendu Ahir
Parthe Pandit
99
0
0
24 Feb 2025
Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer
Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer
Blake Bordelon
Cengiz Pehlevan
AI4CE
239
1
0
04 Feb 2025
Spurious Correlations in High Dimensional Regression: The Roles of Regularization, Simplicity Bias and Over-Parameterization
Spurious Correlations in High Dimensional Regression: The Roles of Regularization, Simplicity Bias and Over-Parameterization
Simone Bombari
Marco Mondelli
297
0
0
03 Feb 2025
How more data can hurt: Instability and regularization in next-generation reservoir computing
How more data can hurt: Instability and regularization in next-generation reservoir computing
Yuanzhao Zhang
Edmilson Roque dos Santos
Sean P. Cornelius
153
2
0
28 Jan 2025
A High Dimensional Statistical Model for Adversarial Training: Geometry and Trade-Offs
A High Dimensional Statistical Model for Adversarial Training: Geometry and Trade-Offs
Kasimir Tanner
Matteo Vilucchio
Bruno Loureiro
Florent Krzakala
AAML
97
1
0
31 Dec 2024
Analysis of High-dimensional Gaussian Labeled-unlabeled Mixture Model via Message-passing Algorithm
Analysis of High-dimensional Gaussian Labeled-unlabeled Mixture Model via Message-passing Algorithm
Xiaosi Gu
Tomoyuki Obuchi
130
0
0
29 Nov 2024
Double Descent Meets Out-of-Distribution Detection: Theoretical Insights and Empirical Analysis on the role of model complexity
Double Descent Meets Out-of-Distribution Detection: Theoretical Insights and Empirical Analysis on the role of model complexity
Mouin Ben Ammar
David Brellmann
Arturo Mendoza
Antoine Manzanera
Gianni Franchi
OODD
116
0
0
04 Nov 2024
High-dimensional Analysis of Knowledge Distillation: Weak-to-Strong Generalization and Scaling Laws
High-dimensional Analysis of Knowledge Distillation: Weak-to-Strong Generalization and Scaling Laws
M. E. Ildiz
Halil Alperen Gozeten
Ege Onur Taga
Marco Mondelli
Samet Oymak
130
5
0
24 Oct 2024
Theoretical Limitations of Ensembles in the Age of Overparameterization
Theoretical Limitations of Ensembles in the Age of Overparameterization
Niclas Dern
John P. Cunningham
Geoff Pleiss
BDLUQCV
90
1
0
21 Oct 2024
Provable Weak-to-Strong Generalization via Benign Overfitting
Provable Weak-to-Strong Generalization via Benign Overfitting
David X. Wu
A. Sahai
165
10
0
06 Oct 2024
On the Pinsker bound of inner product kernel regression in large dimensions
On the Pinsker bound of inner product kernel regression in large dimensions
Weihao Lu
Jialin Ding
Haobo Zhang
Qian Lin
87
1
0
02 Sep 2024
Learning Analysis of Kernel Ridgeless Regression with Asymmetric Kernel Learning
Learning Analysis of Kernel Ridgeless Regression with Asymmetric Kernel Learning
Fan He
Mingzhe He
Lei Shi
Xiaolin Huang
Johan A. K. Suykens
66
1
0
03 Jun 2024
High-dimensional Learning with Noisy Labels
High-dimensional Learning with Noisy Labels
Aymane El Firdoussi
M. Seddik
NoLa
21
0
0
23 May 2024
Asymptotic theory of in-context learning by linear attention
Asymptotic theory of in-context learning by linear attention
Yue M. Lu
Mary I. Letey
Jacob A. Zavatone-Veth
Anindita Maiti
Cengiz Pehlevan
94
16
0
20 May 2024
Understanding Optimal Feature Transfer via a Fine-Grained Bias-Variance Analysis
Understanding Optimal Feature Transfer via a Fine-Grained Bias-Variance Analysis
Yufan Li
Subhabrata Sen
Ben Adlam
MLT
175
1
0
18 Apr 2024
Predictive Churn with the Set of Good Models
Predictive Churn with the Set of Good Models
J. Watson-Daniels
Flavio du Pin Calmon
Alexander DÁmour
Carol Xuan Long
David C. Parkes
Berk Ustun
132
8
0
12 Feb 2024
Learning from Aggregate responses: Instance Level versus Bag Level Loss Functions
Learning from Aggregate responses: Instance Level versus Bag Level Loss Functions
Adel Javanmard
Lin Chen
Vahab Mirrokni
Ashwinkumar Badanidiyuru
Gang Fu
46
2
0
20 Jan 2024
Flat Minima in Linear Estimation and an Extended Gauss Markov Theorem
Flat Minima in Linear Estimation and an Extended Gauss Markov Theorem
Simon Segert
90
0
0
18 Nov 2023
Orthogonal Random Features: Explicit Forms and Sharp Inequalities
Orthogonal Random Features: Explicit Forms and Sharp Inequalities
N. Demni
Hachem Kadri
72
1
0
11 Oct 2023
A Theory of Non-Linear Feature Learning with One Gradient Step in Two-Layer Neural Networks
A Theory of Non-Linear Feature Learning with One Gradient Step in Two-Layer Neural Networks
Behrad Moniri
Donghwan Lee
Hamed Hassani
Yan Sun
MLT
94
23
0
11 Oct 2023
Six Lectures on Linearized Neural Networks
Six Lectures on Linearized Neural Networks
Theodor Misiakiewicz
Andrea Montanari
134
13
0
25 Aug 2023
Quantitative CLTs in Deep Neural Networks
Quantitative CLTs in Deep Neural Networks
Stefano Favaro
Boris Hanin
Domenico Marinucci
I. Nourdin
G. Peccati
BDL
112
16
0
12 Jul 2023
Precise Asymptotic Generalization for Multiclass Classification with Overparameterized Linear Models
Precise Asymptotic Generalization for Multiclass Classification with Overparameterized Linear Models
David X. Wu
A. Sahai
105
3
0
23 Jun 2023
Benign Overfitting in Deep Neural Networks under Lazy Training
Benign Overfitting in Deep Neural Networks under Lazy Training
Zhenyu Zhu
Fanghui Liu
Grigorios G. Chrysos
Francesco Locatello
Volkan Cevher
AI4CE
66
10
0
30 May 2023
Generalized equivalences between subsampling and ridge regularization
Generalized equivalences between subsampling and ridge regularization
Pratik V. Patil
Jin-Hong Du
91
5
0
29 May 2023
How Two-Layer Neural Networks Learn, One (Giant) Step at a Time
How Two-Layer Neural Networks Learn, One (Giant) Step at a Time
Yatin Dandi
Florent Krzakala
Bruno Loureiro
Luca Pesce
Ludovic Stephan
MLT
122
29
0
29 May 2023
How many samples are needed to leverage smoothness?
How many samples are needed to leverage smoothness?
Vivien A. Cabannes
Stefano Vigogna
62
2
0
25 May 2023
From Tempered to Benign Overfitting in ReLU Neural Networks
From Tempered to Benign Overfitting in ReLU Neural Networks
Guy Kornowski
Gilad Yehudai
Ohad Shamir
91
13
0
24 May 2023
Per-Example Gradient Regularization Improves Learning Signals from Noisy Data
Per-Example Gradient Regularization Improves Learning Signals from Noisy Data
Xuran Meng
Yuan Cao
Difan Zou
67
5
0
31 Mar 2023
Kernel interpolation generalizes poorly
Kernel interpolation generalizes poorly
Yicheng Li
Haobo Zhang
Qian Lin
71
11
0
28 Mar 2023
Online Learning for the Random Feature Model in the Student-Teacher Framework
Online Learning for the Random Feature Model in the Student-Teacher Framework
Roman Worschech
B. Rosenow
86
0
0
24 Mar 2023
Some Fundamental Aspects about Lipschitz Continuity of Neural Networks
Some Fundamental Aspects about Lipschitz Continuity of Neural Networks
Grigory Khromov
Sidak Pal Singh
155
8
0
21 Feb 2023
Precise Asymptotic Analysis of Deep Random Feature Models
Precise Asymptotic Analysis of Deep Random Feature Models
David Bosch
Ashkan Panahi
B. Hassibi
93
19
0
13 Feb 2023
Bayes-optimal Learning of Deep Random Networks of Extensive-width
Bayes-optimal Learning of Deep Random Networks of Extensive-width
Hugo Cui
Florent Krzakala
Lenka Zdeborová
BDL
103
39
0
01 Feb 2023
Bayesian Interpolation with Deep Linear Networks
Bayesian Interpolation with Deep Linear Networks
Boris Hanin
Alexander Zlokapa
137
26
0
29 Dec 2022
Gradient flow in the gaussian covariate model: exact solution of learning curves and multiple descent structures
Gradient flow in the gaussian covariate model: exact solution of learning curves and multiple descent structures
Antione Bodin
N. Macris
72
4
0
13 Dec 2022
High Dimensional Binary Classification under Label Shift: Phase Transition and Regularization
High Dimensional Binary Classification under Label Shift: Phase Transition and Regularization
Jiahui Cheng
Minshuo Chen
Hao Liu
Tuo Zhao
Wenjing Liao
62
1
0
01 Dec 2022
Neural networks trained with SGD learn distributions of increasing complexity
Neural networks trained with SGD learn distributions of increasing complexity
Maria Refinetti
Alessandro Ingrosso
Sebastian Goldt
UQCV
139
43
0
21 Nov 2022
Spectral Evolution and Invariance in Linear-width Neural Networks
Spectral Evolution and Invariance in Linear-width Neural Networks
Zhichao Wang
A. Engel
Anand D. Sarwate
Ioana Dumitriu
Tony Chiang
114
18
0
11 Nov 2022
Designing Universal Causal Deep Learning Models: The Case of Infinite-Dimensional Dynamical Systems from Stochastic Analysis
Designing Universal Causal Deep Learning Models: The Case of Infinite-Dimensional Dynamical Systems from Stochastic Analysis
Luca Galimberti
Anastasis Kratsios
Giulia Livieri
OOD
121
14
0
24 Oct 2022
On double-descent in uncertainty quantification in overparametrized models
On double-descent in uncertainty quantification in overparametrized models
Lucas Clarté
Bruno Loureiro
Florent Krzakala
Lenka Zdeborová
UQCV
151
14
0
23 Oct 2022

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.