Understanding the Role of Optimization in Double Descent

6 December 2023

Papers citing "Understanding the Role of Optimization in Double Descent"

21 / 21 papers shown

Title
Unraveling the Enigma of Double Descent: An In-depth Analysis through the Lens of Learned Feature Space Yufei Gu Xiaoqing Zheng T. Aste 53 3 0 20 Oct 2023
Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle Rylan Schaeffer Mikail Khona Zachary Robertson Akhilan Boopathy Kateryna Pistunova J. Rocks Ila Rani Fiete Oluwasanmi Koyejo 108 34 0 24 Mar 2023
DSD $^2$ : Can We Dodge Sparse Double Descent and Compress the Neural Network Worry-Free? Victor Quétu Enzo Tartaglione 49 7 0 02 Mar 2023
Can we avoid Double Descent in Deep Neural Networks? Victor Quétu Enzo Tartaglione AI4CE 46 3 0 26 Feb 2023
Optimal Activation Functions for the Random Features Regression Model Jianxin Wang José Bento 47 3 0 31 May 2022
Model, sample, and epoch-wise descents: exact solution of gradient flow in the random feature model A. Bodin N. Macris 87 13 0 22 Oct 2021
Conditioning of Random Feature Matrices: Double Descent and Generalization Error Zhijun Chen Hayden Schaeffer 67 12 0 21 Oct 2021
Mitigating deep double descent by concatenating inputs John Chen Qihan Wang Anastasios Kyrillidis BDL 27 3 0 02 Jul 2021
Asymptotics of Ridge Regression in Convolutional Models Mojtaba Sahraee-Ardakan Tung Mai Anup B. Rao Ryan Rossi S. Rangan A. Fletcher MLT 23 2 0 08 Mar 2021
Low Curvature Activations Reduce Overfitting in Adversarial Training Vasu Singla Sahil Singla David Jacobs Soheil Feizi AAML 56 47 0 15 Feb 2021
Distilling Double Descent Andrew Cotter A. Menon Harikrishna Narasimhan A. S. Rawat Sashank J. Reddi Yichen Zhou 37 7 0 13 Feb 2021
Provable Benefits of Overparameterization in Model Compression: From Double Descent to Pruning Neural Networks Xiangyu Chang Yingcong Li Samet Oymak Christos Thrampoulidis 56 50 0 16 Dec 2020
Dimensionality reduction, regularization, and generalization in overparameterized regressions Ningyuan Huang D. Hogg Soledad Villar 36 15 0 23 Nov 2020
Evaluation of Neural Architectures Trained with Square Loss vs Cross-Entropy in Classification Tasks Like Hui M. Belkin UQCV AAML VLM 46 171 0 12 Jun 2020
Double Double Descent: On Generalization Errors in Transfer Learning between Linear Regression Tasks Yehuda Dar Richard G. Baraniuk 106 19 0 12 Jun 2020
Overfitting in adversarially robust deep learning Leslie Rice Eric Wong Zico Kolter 94 800 0 26 Feb 2020
Rethinking Bias-Variance Trade-off for Generalization of Neural Networks Zitong Yang Yaodong Yu Chong You Jacob Steinhardt Yi-An Ma 65 184 0 26 Feb 2020
Surprises in High-Dimensional Ridgeless Least Squares Interpolation Trevor Hastie Andrea Montanari Saharon Rosset Robert Tibshirani 165 743 0 19 Mar 2019
Reconciling modern machine learning practice and the bias-variance trade-off M. Belkin Daniel J. Hsu Siyuan Ma Soumik Mandal 213 1,638 0 28 Dec 2018
Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms Han Xiao Kashif Rasul Roland Vollgraf 252 8,856 0 25 Aug 2017
SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives Aaron Defazio Francis R. Bach Simon Lacoste-Julien ODL 131 1,823 0 01 Jul 2014