Learning sparse features can lead to overfitting in neural networks

Learning sparse features can lead to overfitting in neural networks

24 June 2022

Leonardo Petrini

Francesco Cagnetta

Eric Vanden-Eijnden

Papers citing "Learning sparse features can lead to overfitting in neural networks"

16 / 16 papers shown

Title
Weak-to-Strong Generalization Even in Random Feature Networks, Provably Marko Medvedev Kaifeng Lyu Dingli Yu Sanjeev Arora Zhiyuan Li Nathan Srebro 107 0 0 04 Mar 2025
Features are fate: a theory of transfer learning in high-dimensional regression Javan Tahir Surya Ganguli Grant M. Rotskoff 32 1 0 10 Oct 2024
The Optimization Landscape of SGD Across the Feature Learning Strength Alexander B. Atanasov Alexandru Meterez James B. Simon C. Pehlevan 43 2 0 06 Oct 2024
Repetita Iuvant: Data Repetition Allows SGD to Learn High-Dimensional Multi-Index Functions Luca Arnaboldi Yatin Dandi Florent Krzakala Luca Pesce Ludovic Stephan 68 12 0 24 May 2024
CDMPP: A Device-Model Agnostic Framework for Latency Prediction of Tensor Programs Hanpeng Hu Junwei Su Juntao Zhao Yanghua Peng Yibo Zhu Haibin Lin Chuan Wu 16 1 0 16 Nov 2023
Local Kernel Renormalization as a mechanism for feature learning in overparametrized Convolutional Neural Networks R. Aiudi R. Pacelli A. Vezzani R. Burioni P. Rotondo MLT 21 15 0 21 Jul 2023
How Two-Layer Neural Networks Learn, One (Giant) Step at a Time Yatin Dandi Florent Krzakala Bruno Loureiro Luca Pesce Ludovic Stephan MLT 34 25 0 29 May 2023
Tight conditions for when the NTK approximation is valid Enric Boix-Adserà Etai Littwin 30 0 0 22 May 2023
Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Models Guillermo Ortiz-Jiménez Alessandro Favero P. Frossard MoMe 48 106 0 22 May 2023
Neural Networks Efficiently Learn Low-Dimensional Representations with SGD Alireza Mousavi-Hosseini Sejun Park M. Girotti Ioannis Mitliagkas Murat A. Erdogdu MLT 324 48 0 29 Sep 2022
Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs Etienne Boursier Loucas Pillaud-Vivien Nicolas Flammarion ODL 19 58 0 02 Jun 2022
Data-driven emergence of convolutional structure in neural networks Alessandro Ingrosso Sebastian Goldt 50 38 0 01 Feb 2022
Towards Learning Convolutions from Scratch Behnam Neyshabur SSL 220 71 0 27 Jul 2020
Geometric compression of invariant manifolds in neural nets J. Paccolat Leonardo Petrini Mario Geiger Kevin Tyloo M. Wyart MLT 52 34 0 22 Jul 2020
Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networks Blake Bordelon Abdulkadir Canatar C. Pehlevan 139 201 0 07 Feb 2020
Norm-Based Capacity Control in Neural Networks Behnam Neyshabur Ryota Tomioka Nathan Srebro 119 577 0 27 Feb 2015