The Interpolation Phase Transition in Neural Networks: Memorization and Generalization under Lazy Training

25 July 2020

Papers citing "The Interpolation Phase Transition in Neural Networks: Memorization and Generalization under Lazy Training"

50 / 74 papers shown

Title
Information-theoretic reduction of deep neural networks to linear models in the overparametrized proportional regime Francesco Camilli D. Tieplova Eleonora Bergamin Jean Barbier 103 0 0 06 May 2025
High-dimensional Analysis of Knowledge Distillation: Weak-to-Strong Generalization and Scaling Laws M. E. Ildiz Halil Alperen Gozeten Ege Onur Taga Marco Mondelli Samet Oymak 54 2 0 24 Oct 2024
MUSO: Achieving Exact Machine Unlearning in Over-Parameterized Regimes Ruikai Yang M. He Zhengbao He Youmei Qiu X. Huang MU BDL 37 1 0 11 Oct 2024
On the Convergence of FedProx with Extrapolation and Inexact Prox Hanmin Li Peter Richtárik FedML 27 0 0 02 Oct 2024
A General Framework of the Consistency for Large Neural Networks Haoran Zhan Yingcun Xia 37 0 0 21 Sep 2024
Learning Analysis of Kernel Ridgeless Regression with Asymmetric Kernel Learning Fan He Mingzhe He Lei Shi Xiaolin Huang Johan A. K. Suykens 31 1 0 03 Jun 2024
Bounds for the smallest eigenvalue of the NTK for arbitrary spherical data of arbitrary dimension Kedar Karhadkar Michael Murray Guido Montúfar 32 2 0 23 May 2024
The Positivity of the Neural Tangent Kernel Luís Carvalho Joao L. Costa José Mourao Gonccalo Oliveira 17 1 0 19 Apr 2024
Analyzing the Neural Tangent Kernel of Periodically Activated Coordinate Networks Hemanth Saratchandran Shin-Fang Chng Simon Lucey 16 1 0 07 Feb 2024
Generalization in Kernel Regression Under Realistic Assumptions Daniel Barzilai Ohad Shamir 29 14 0 26 Dec 2023
Analysis of the expected $L_2$ error of an over-parametrized deep neural network estimate learned by gradient descent without regularization Selina Drews Michael Kohler 25 2 0 24 Nov 2023
Weight fluctuations in (deep) linear neural networks and a derivation of the inverse-variance flatness relation Markus Gross A. Raulf Christoph Räth 38 0 0 23 Nov 2023
On the Convergence of Encoder-only Shallow Transformers Yongtao Wu Fanghui Liu Grigorios G. Chrysos V. Cevher 42 5 0 02 Nov 2023
Infinite Width Graph Neural Networks for Node Regression/ Classification Yunus Cobanoglu AI4CE 13 1 0 12 Oct 2023
Why Does Sharpness-Aware Minimization Generalize Better Than SGD? Zixiang Chen Junkai Zhang Yiwen Kou Xiangning Chen Cho-Jui Hsieh Quanquan Gu 24 11 0 11 Oct 2023
What do larger image classifiers memorise? Michal Lukasik Vaishnavh Nagarajan A. S. Rawat A. Menon Sanjiv Kumar 30 5 0 09 Oct 2023
Enhancing Kernel Flexibility via Learning Asymmetric Locally-Adaptive Kernels Fan He Ming-qian He Lei Shi Xiaolin Huang Johan A. K. Suykens 13 1 0 08 Oct 2023
Benign Overfitting in Two-Layer ReLU Convolutional Neural Networks for XOR Data Xuran Meng Difan Zou Yuan Cao MLT 29 7 0 03 Oct 2023
Universality of max-margin classifiers Andrea Montanari Feng Ruan Basil Saeed Youngtak Sohn 26 3 0 29 Sep 2023
Six Lectures on Linearized Neural Networks Theodor Misiakiewicz Andrea Montanari 34 12 0 25 Aug 2023
Memory capacity of two layer neural networks with smooth activations Liam Madden Christos Thrampoulidis MLT 13 5 0 03 Aug 2023
Controlling the Inductive Bias of Wide Neural Networks by Modifying the Kernel's Spectrum Amnon Geifman Daniel Barzilai Ronen Basri Meirav Galun 24 4 0 26 Jul 2023
Fundamental limits of overparametrized shallow neural networks for supervised learning Francesco Camilli D. Tieplova Jean Barbier 27 9 0 11 Jul 2023
Unraveling Projection Heads in Contrastive Learning: Insights from Expansion and Shrinkage Yu Gui Cong Ma Yiqiao Zhong 22 6 0 06 Jun 2023
How Spurious Features Are Memorized: Precise Analysis for Random and NTK Features Simone Bombari Marco Mondelli AAML 19 4 0 20 May 2023
Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks Eshaan Nichani Alexandru Damian Jason D. Lee MLT 36 13 0 11 May 2023
On the Eigenvalue Decay Rates of a Class of Neural-Network Related Kernel Functions Defined on General Domains Yicheng Li Zixiong Yu Y. Cotronis Qian Lin 55 13 0 04 May 2023
Kernel interpolation generalizes poorly Yicheng Li Haobo Zhang Qian Lin 29 10 0 28 Mar 2023
Controlled Descent Training Viktor Andersson B. Varga Vincent Szolnoky Andreas Syrén Rebecka Jörnsten Balázs Kulcsár 33 1 0 16 Mar 2023
Benign Overfitting for Two-layer ReLU Convolutional Neural Networks Yiwen Kou Zi-Yuan Chen Yuanzhou Chen Quanquan Gu MLT 49 12 0 07 Mar 2023
High-dimensional analysis of double descent for linear regression with random projections Francis R. Bach 26 33 0 02 Mar 2023
Learning time-scales in two-layers neural networks Raphael Berthier Andrea Montanari Kangjie Zhou 36 33 0 28 Feb 2023
Some Fundamental Aspects about Lipschitz Continuity of Neural Networks Grigory Khromov Sidak Pal Singh 24 7 0 21 Feb 2023
$Generalization Ability of Wide Neural Networks on $\mathbb{R}$$ Generalization Ability of Wide Neural Networks on $\mathbb{R}$ Jianfa Lai Manyun Xu Rui Chen Qi-Rong Lin 13 21 0 12 Feb 2023
Beyond the Universal Law of Robustness: Sharper Laws for Random Features and Neural Tangent Kernels Simone Bombari Shayan Kiyani Marco Mondelli AAML 28 10 0 03 Feb 2023
ResMem: Learn what you can and memorize the rest Zitong Yang Michal Lukasik Vaishnavh Nagarajan Zong-xiao Li A. S. Rawat Manzil Zaheer A. Menon Surinder Kumar VLM 32 8 0 03 Feb 2023
Bayesian Interpolation with Deep Linear Networks Boris Hanin Alexander Zlokapa 34 25 0 29 Dec 2022
Improved Convergence Guarantees for Shallow Neural Networks A. Razborov ODL 25 1 0 05 Dec 2022
Dense Hebbian neural networks: a replica symmetric picture of supervised learning E. Agliari L. Albanese Francesco Alemanno Andrea Alessandrelli Adriano Barra F. Giannotti Daniele Lotito D. Pedreschi 11 15 0 25 Nov 2022
Overparameterized random feature regression with nearly orthogonal data Zhichao Wang Yizhe Zhu 15 3 0 11 Nov 2022
Finite Sample Identification of Wide Shallow Neural Networks with Biases M. Fornasier T. Klock Marco Mondelli Michael Rauchensteiner 17 6 0 08 Nov 2022
Mean-field analysis for heavy ball methods: Dropout-stability, connectivity, and global convergence Diyuan Wu Vyacheslav Kungurtsev Marco Mondelli 15 3 0 13 Oct 2022
Generalization Properties of NAS under Activation and Skip Connection Search Zhenyu Zhu Fanghui Liu Grigorios G. Chrysos V. Cevher AI4CE 15 14 0 15 Sep 2022
Multiple Descent in the Multiple Random Feature Model Xuran Meng Jianfeng Yao Yuan Cao 26 7 0 21 Aug 2022
Intersection of Parallels as an Early Stopping Criterion Ali Vardasbi Maarten de Rijke Mostafa Dehghani MoMe 33 5 0 19 Aug 2022
On the generalization of learning algorithms that do not converge N. Chandramoorthy Andreas Loukas Khashayar Gatmiry Stefanie Jegelka MLT 14 11 0 16 Aug 2022
Adaptive Step-Size Methods for Compressed SGD Adarsh M. Subramaniam A. Magesh V. Veeravalli 14 1 0 20 Jul 2022
Lazy Estimation of Variable Importance for Large Neural Networks Yue Gao Abby Stevens Rebecca Willett Garvesh Raskutti 33 4 0 19 Jul 2022
Overparametrized linear dimensionality reductions: From projection pursuit to two-layer neural networks Andrea Montanari Kangjie Zhou 16 2 0 14 Jun 2022
Identifying good directions to escape the NTK regime and efficiently learn low-degree plus sparse polynomials Eshaan Nichani Yunzhi Bai Jason D. Lee 24 10 0 08 Jun 2022