Surprises in High-Dimensional Ridgeless Least Squares Interpolation

19 March 2019

Papers citing "Surprises in High-Dimensional Ridgeless Least Squares Interpolation"

50 / 134 papers shown

Title
A dynamic view of the double descent Vivek Shripad Borkar 60 0 0 03 May 2025
Sobolev norm inconsistency of kernel interpolation Yunfei Yang 34 0 0 29 Apr 2025
auto-fpt: Automating Free Probability Theory Calculations for Machine Learning Theory Arjun Subramonian Elvis Dohmatob 24 0 0 14 Apr 2025
Gradient Descent Robustly Learns the Intrinsic Dimension of Data in Training Convolutional Neural Networks Chenyang Zhang Peifeng Gao Difan Zou Yuan Cao OOD MLT 59 0 0 11 Apr 2025
Quantifying Overfitting along the Regularization Path for Two-Part-Code MDL in Supervised Classification Xiaohan Zhu Nathan Srebro 53 0 0 03 Mar 2025
Order-Robust Class Incremental Learning: Graph-Driven Dynamic Similarity Grouping Guannan Lai Yujie Li Xiangkun Wang J. Zhang Tianrui Li Xin Yang CLL 47 0 0 27 Feb 2025
Analysis of Overparameterization in Continual Learning under a Linear Model Daniel Goldfarb Paul Hand CLL 39 0 0 11 Feb 2025
High-dimensional Analysis of Knowledge Distillation: Weak-to-Strong Generalization and Scaling Laws M. E. Ildiz Halil Alperen Gozeten Ege Onur Taga Marco Mondelli Samet Oymak 54 2 0 24 Oct 2024
Efficient Optimization Algorithms for Linear Adversarial Training Antônio H. Ribeiro Thomas B. Schon Dave Zahariah Francis Bach AAML 45 1 0 16 Oct 2024
Investigating the Impact of Model Complexity in Large Language Models Jing Luo Huiyuan Wang Weiran Huang 34 0 0 01 Oct 2024
Breaking Neural Network Scaling Laws with Modularity Akhilan Boopathy Sunshine Jiang William Yue Jaedong Hwang Abhiram Iyer Ila Fiete OOD 39 2 0 09 Sep 2024
Overfitting Behaviour of Gaussian Kernel Ridgeless Regression: Varying Bandwidth or Dimensionality Marko Medvedev Gal Vardi Nathan Srebro 68 3 0 05 Sep 2024
Theoretical Insights into Overparameterized Models in Multi-Task and Replay-Based Continual Learning Mohammadamin Banayeeanzade Mahdi Soltanolkotabi Mohammad Rostami CLL LRM 97 1 0 29 Aug 2024
Remove Symmetries to Control Model Expressivity and Improve Optimization Liu Ziyin Yizhou Xu Isaac Chuang AAML 38 1 0 28 Aug 2024
Meta-Analysis with Untrusted Data Shiva Kaul Geoffrey J. Gordon CML 32 1 0 12 Jul 2024
Asymptotic theory of in-context learning by linear attention Yue M. Lu Mary I. Letey Jacob A. Zavatone-Veth Anindita Maiti C. Pehlevan 23 10 0 20 May 2024
Thermodynamic limit in learning period three Yuichiro Terasaki Kohei Nakajima 40 1 0 12 May 2024
Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks Fanghui Liu L. Dadi V. Cevher 79 2 0 29 Apr 2024
Understanding Optimal Feature Transfer via a Fine-Grained Bias-Variance Analysis Yufan Li Subhabrata Sen Ben Adlam MLT 48 1 0 18 Apr 2024
High-dimensional analysis of ridge regression for non-identically distributed data with a variance profile Jérémie Bigot Issa-Mbenard Dabo Camille Male 31 4 0 29 Mar 2024
Random features models: a way to study the success of naive imputation Alexis Ayme Claire Boyer Lpsm Aymeric Dieuleveut Erwan Scornet 24 3 0 06 Feb 2024
Prevalidated ridge regression is a highly-efficient drop-in replacement for logistic regression for high-dimensional data Angus Dempster Geoffrey I. Webb Daniel F. Schmidt 26 0 0 28 Jan 2024
Benign Overfitting and Grokking in ReLU Networks for XOR Cluster Data Zhiwei Xu Yutong Wang Spencer Frei Gal Vardi Wei Hu MLT 28 23 0 04 Oct 2023
Towards a statistical theory of data selection under weak supervision Germain Kolossov Andrea Montanari Pulkit Tandon 14 14 0 25 Sep 2023
Adaptive Lasso, Transfer Lasso, and Beyond: An Asymptotic Perspective Masaaki Takada Hironori Fujisawa 16 1 0 30 Aug 2023
The Interpolating Information Criterion for Overparameterized Models Liam Hodgkinson Christopher van der Heide Roberto Salomone Fred Roosta Michael W. Mahoney 20 7 0 15 Jul 2023
Precise Asymptotic Generalization for Multiclass Classification with Overparameterized Linear Models David X. Wu A. Sahai 26 2 0 23 Jun 2023
Quantifying lottery tickets under label noise: accuracy, calibration, and complexity V. Arora Daniele Irto Sebastian Goldt G. Sanguinetti 36 2 0 21 Jun 2023
Gibbs-Based Information Criteria and the Over-Parameterized Regime Haobo Chen Yuheng Bu Greg Wornell 27 1 0 08 Jun 2023
Maximally Machine-Learnable Portfolios Philippe Goulet Coulombe Maximilian Göbel 21 3 0 08 Jun 2023
Unraveling Projection Heads in Contrastive Learning: Insights from Expansion and Shrinkage Yu Gui Cong Ma Yiqiao Zhong 22 6 0 06 Jun 2023
Generalized equivalences between subsampling and ridge regularization Pratik V. Patil Jin-Hong Du 29 5 0 29 May 2023
Moment-Based Adjustments of Statistical Inference in High-Dimensional Generalized Linear Models Kazuma Sawaya Yoshimasa Uematsu Masaaki Imaizumi 29 2 0 28 May 2023
Bayesian Analysis for Over-parameterized Linear Model via Effective Spectra Tomoya Wakayama Masaaki Imaizumi 51 0 0 25 May 2023
Subsample Ridge Ensembles: Equivalences and Generalized Cross-Validation Jin-Hong Du Pratik V. Patil Arun K. Kuchibhotla 16 11 0 25 Apr 2023
Analysis of Interpolating Regression Models and the Double Descent Phenomenon T. McKelvey 4 0 0 17 Apr 2023
Do deep neural networks have an inbuilt Occam's razor? Chris Mingard Henry Rees Guillermo Valle Pérez A. Louis UQCV BDL 21 15 0 13 Apr 2023
Mathematical Challenges in Deep Learning V. Nia Guojun Zhang I. Kobyzev Michael R. Metel Xinlin Li ... S. Hemati M. Asgharian Linglong Kong Wulong Liu Boxing Chen AI4CE VLM 37 1 0 24 Mar 2023
Online Learning for the Random Feature Model in the Student-Teacher Framework Roman Worschech B. Rosenow 38 0 0 24 Mar 2023
General Loss Functions Lead to (Approximate) Interpolation in High Dimensions Kuo-Wei Lai Vidya Muthukumar 21 5 0 13 Mar 2023
DSD $^2$ : Can We Dodge Sparse Double Descent and Compress the Neural Network Worry-Free? Victor Quétu Enzo Tartaglione 26 7 0 02 Mar 2023
Extrapolated cross-validation for randomized ensembles Jin-Hong Du Pratik V. Patil Kathryn Roeder Arun K. Kuchibhotla 11 5 0 27 Feb 2023
Statistical Inference for Linear Functionals of Online SGD in High-dimensional Linear Regression Bhavya Agrawalla Krishnakumar Balasubramanian Promit Ghosal 23 2 0 20 Feb 2023
Precise Asymptotic Analysis of Deep Random Feature Models David Bosch Ashkan Panahi B. Hassibi 32 19 0 13 Feb 2023
Beyond the Universal Law of Robustness: Sharper Laws for Random Features and Neural Tangent Kernels Simone Bombari Shayan Kiyani Marco Mondelli AAML 28 10 0 03 Feb 2023
Implicit Regularization Leads to Benign Overfitting for Sparse Linear Regression Mo Zhou Rong Ge 27 2 0 01 Feb 2023
Demystifying Disagreement-on-the-Line in High Dimensions Dong-Hwan Lee Behrad Moniri Xinmeng Huang Edgar Dobriban Hamed Hassani 21 8 0 31 Jan 2023
A Simple Algorithm For Scaling Up Kernel Methods Tengyu Xu Bryan T. Kelly Semyon Malamud 11 0 0 26 Jan 2023
Gradient flow in the gaussian covariate model: exact solution of learning curves and multiple descent structures Antione Bodin N. Macris 34 4 0 13 Dec 2022
High Dimensional Binary Classification under Label Shift: Phase Transition and Regularization Jiahui Cheng Minshuo Chen Hao Liu Tuo Zhao Wenjing Liao 29 0 0 01 Dec 2022