Understanding Double Descent Requires a Fine-Grained Bias-Variance Decomposition

4 November 2020

Papers citing "Understanding Double Descent Requires a Fine-Grained Bias-Variance Decomposition"

27 / 27 papers shown

Title
Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer Blake Bordelon Cengiz Pehlevan AI4CE 111 1 0 04 Feb 2025
Understanding Optimal Feature Transfer via a Fine-Grained Bias-Variance Analysis Yufan Li Subhabrata Sen Ben Adlam MLT 110 1 0 18 Apr 2024
A Theory of Non-Linear Feature Learning with One Gradient Step in Two-Layer Neural Networks Behrad Moniri Donghwan Lee Hamed Hassani Yan Sun MLT 57 21 0 11 Oct 2023
The Neural Tangent Kernel in High Dimensions: Triple Descent and a Multi-Scale Theory of Generalization Ben Adlam Jeffrey Pennington 25 123 0 15 Aug 2020
Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regime Stéphane dÁscoli Maria Refinetti Giulio Biroli Florent Krzakala 130 152 0 02 Mar 2020
Rethinking Bias-Variance Trade-off for Generalization of Neural Networks Zitong Yang Yaodong Yu Chong You Jacob Steinhardt Yi-An Ma 45 182 0 26 Feb 2020
Implicit Regularization of Random Feature Models Arthur Jacot Berfin Simsek Francesco Spadaro Clément Hongler Franck Gabriel 50 82 0 19 Feb 2020
Towards a Human-like Open-Domain Chatbot Daniel De Freitas Minh-Thang Luong David R. So Jamie Hall Noah Fiedel ... Zi Yang Apoorv Kulshreshtha Gaurav Nemade Yifeng Lu Quoc V. Le 61 931 0 27 Jan 2020
Deep Double Descent: Where Bigger Models and More Data Hurt Preetum Nakkiran Gal Kaplun Yamini Bansal Tristan Yang Boaz Barak Ilya Sutskever 97 925 0 04 Dec 2019
The generalization error of random features regression: Precise asymptotics and double descent curve Song Mei Andrea Montanari 68 631 0 14 Aug 2019
Understanding overfitting peaks in generalization error: Analytical risk curves for $l_2$ and $l_1$ penalized interpolation P. Mitra 35 50 0 09 Jun 2019
Linearized two-layers neural networks in high dimension Behrooz Ghorbani Song Mei Theodor Misiakiewicz Andrea Montanari MLT 34 243 0 27 Apr 2019
Surprises in High-Dimensional Ridgeless Least Squares Interpolation Trevor Hastie Andrea Montanari Saharon Rosset Robert Tibshirani 115 737 0 19 Mar 2019
Two models of double descent for weak features M. Belkin Daniel J. Hsu Ji Xu 80 375 0 18 Mar 2019
Scaling description of generalization with number of parameters in deep learning Mario Geiger Arthur Jacot S. Spigler Franck Gabriel Levent Sagun Stéphane dÁscoli Giulio Biroli Clément Hongler Matthieu Wyart 68 195 0 06 Jan 2019
Reconciling modern machine learning practice and the bias-variance trade-off M. Belkin Daniel J. Hsu Siyuan Ma Soumik Mandal 168 1,628 0 28 Dec 2018
A Modern Take on the Bias-Variance Tradeoff in Neural Networks Brady Neal Sarthak Mittal A. Baratin Vinayak Tantia Matthew Scicluna Simon Lacoste-Julien Ioannis Mitliagkas 62 169 0 19 Oct 2018
Just Interpolate: Kernel "Ridgeless" Regression Can Generalize Tengyuan Liang Alexander Rakhlin 38 353 0 01 Aug 2018
Does data interpolation contradict statistical optimality? M. Belkin Alexander Rakhlin Alexandre B. Tsybakov 60 218 0 25 Jun 2018
Neural Tangent Kernel: Convergence and Generalization in Neural Networks Arthur Jacot Franck Gabriel Clément Hongler 159 3,160 0 20 Jun 2018
Overfitting or perfect fitting? Risk bounds for classification and regression rules that interpolate M. Belkin Daniel J. Hsu P. Mitra AI4CE 122 256 0 13 Jun 2018
Optimal ridge penalty for real-world high-dimensional data can be zero or negative due to the implicit ridge regularization D. Kobak Jonathan Lomond Benoit Sanchez 42 89 0 28 May 2018
To understand deep learning we need to understand kernel learning M. Belkin Siyuan Ma Soumik Mandal 32 414 0 05 Feb 2018
High-dimensional dynamics of generalization error in neural networks Madhu S. Advani Andrew M. Saxe AI4CE 118 467 0 10 Oct 2017
A Random Matrix Approach to Neural Networks Cosme Louart Zhenyu Liao Romain Couillet 41 161 0 17 Feb 2017
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer Noam M. Shazeer Azalia Mirhoseini Krzysztof Maziarz Andy Davis Quoc V. Le Geoffrey E. Hinton J. Dean MoE 136 2,582 0 23 Jan 2017
Understanding deep learning requires rethinking generalization Chiyuan Zhang Samy Bengio Moritz Hardt Benjamin Recht Oriol Vinyals HAI 250 4,612 0 10 Nov 2016