Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regime

2 March 2020

Papers citing "Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regime"

31 / 31 papers shown

Title
Information-theoretic reduction of deep neural networks to linear models in the overparametrized proportional regime Francesco Camilli D. Tieplova Eleonora Bergamin Jean Barbier 106 0 0 06 May 2025
The Double Descent Behavior in Two Layer Neural Network for Binary Classification Chathurika S Abeykoon A. Beknazaryan Hailin Sang 51 1 0 27 Apr 2025
Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer Blake Bordelon C. Pehlevan AI4CE 61 1 0 04 Feb 2025
How more data can hurt: Instability and regularization in next-generation reservoir computing Yuanzhao Zhang Edmilson Roque dos Santos Sean P. Cornelius 77 2 0 28 Jan 2025
Investigating the Impact of Model Complexity in Large Language Models Jing Luo Huiyuan Wang Weiran Huang 34 0 0 01 Oct 2024
Random Matrix Analysis to Balance between Supervised and Unsupervised Learning under the Low Density Separation Assumption Vasilii Feofanov Malik Tiomoko Aladin Virmaux 31 5 0 20 Oct 2023
Quantifying lottery tickets under label noise: accuracy, calibration, and complexity V. Arora Daniele Irto Sebastian Goldt G. Sanguinetti 34 2 0 21 Jun 2023
Gibbs-Based Information Criteria and the Over-Parameterized Regime Haobo Chen Yuheng Bu Greg Wornell 21 1 0 08 Jun 2023
Double Descent of Discrepancy: A Task-, Data-, and Model-Agnostic Phenomenon Yi-Xiao Luo Bin Dong 26 0 0 25 May 2023
Subsample Ridge Ensembles: Equivalences and Generalized Cross-Validation Jin-Hong Du Pratik V. Patil Arun K. Kuchibhotla 16 11 0 25 Apr 2023
Online Learning for the Random Feature Model in the Student-Teacher Framework Roman Worschech B. Rosenow 36 0 0 24 Mar 2023
Pathologies of Predictive Diversity in Deep Ensembles Taiga Abe E. Kelly Buchanan Geoff Pleiss John P. Cunningham UQCV 38 13 0 01 Feb 2023
Gradient flow in the gaussian covariate model: exact solution of learning curves and multiple descent structures Antione Bodin N. Macris 31 4 0 13 Dec 2022
Sparse Double Descent: Where Network Pruning Aggravates Overfitting Zhengqi He Zeke Xie Quanzhi Zhu Zengchang Qin 69 27 0 17 Jun 2022
Regularization-wise double descent: Why it occurs and how to eliminate it Fatih Yilmaz Reinhard Heckel 25 11 0 03 Jun 2022
Sharp Asymptotics of Kernel Ridge Regression Beyond the Linear Regime Hong Hu Yue M. Lu 49 15 0 13 May 2022
Generalization Through The Lens Of Leave-One-Out Error Gregor Bachmann Thomas Hofmann Aurélien Lucchi 44 7 0 07 Mar 2022
Contrasting random and learned features in deep Bayesian linear regression Jacob A. Zavatone-Veth William L. Tong C. Pehlevan BDL MLT 28 26 0 01 Mar 2022
Understanding the bias-variance tradeoff of Bregman divergences Ben Adlam Neha Gupta Zelda E. Mariet Jamie Smith UQCV UD 15 6 0 08 Feb 2022
SHRIMP: Sparser Random Feature Models via Iterative Magnitude Pruning Yuege Xie Bobby Shi Hayden Schaeffer Rachel A. Ward 78 9 0 07 Dec 2021
Multi-scale Feature Learning Dynamics: Insights for Double Descent Mohammad Pezeshki Amartya Mitra Yoshua Bengio Guillaume Lajoie 58 25 0 06 Dec 2021
Model, sample, and epoch-wise descents: exact solution of gradient flow in the random feature model A. Bodin N. Macris 29 13 0 22 Oct 2021
Learning through atypical "phase transitions" in overparameterized neural networks Carlo Baldassi Clarissa Lauditi Enrico M. Malatesta R. Pacelli Gabriele Perugini R. Zecchina 26 26 0 01 Oct 2021
A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning Yehuda Dar Vidya Muthukumar Richard G. Baraniuk 29 71 0 06 Sep 2021
Double Descent and Other Interpolation Phenomena in GANs Lorenzo Luzi Yehuda Dar Richard Baraniuk 18 5 0 07 Jun 2021
Understanding Double Descent Requires a Fine-Grained Bias-Variance Decomposition Ben Adlam Jeffrey Pennington UD 26 92 0 04 Nov 2020
Geometric compression of invariant manifolds in neural nets J. Paccolat Leonardo Petrini Mario Geiger Kevin Tyloo M. Wyart MLT 49 34 0 22 Jul 2020
When Does Preconditioning Help or Hurt Generalization? S. Amari Jimmy Ba Roger C. Grosse Xuechen Li Atsushi Nitanda Taiji Suzuki Denny Wu Ji Xu 34 32 0 18 Jun 2020
Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networks Z. Fan Zhichao Wang 29 72 0 25 May 2020
Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyond Fanghui Liu Xiaolin Huang Yudong Chen Johan A. K. Suykens BDL 34 172 0 23 Apr 2020
Implicit Regularization of Random Feature Models Arthur Jacot Berfin Simsek Francesco Spadaro Clément Hongler Franck Gabriel 18 82 0 19 Feb 2020