More Than a Toy: Random Matrix Models Predict How Real-World Neural Representations Generalize

11 March 2022

Papers citing "More Than a Toy: Random Matrix Models Predict How Real-World Neural Representations Generalize"

27 / 27 papers shown

Title
Supervised Models Can Generalize Also When Trained on Random Labels Oskar Allerbo Thomas B. Schön OOD SSL 34 0 0 16 May 2025
Training NTK to Generalize with KARE Johannes Schwab Bryan Kelly Semyon Malamud Teng Andrea Xu 32 0 0 16 May 2025
A theoretical framework for overfitting in energy-based modeling Giovanni Catania A. Decelle Cyril Furtlehner Beatriz Seoane 69 2 0 31 Jan 2025
A High Dimensional Statistical Model for Adversarial Training: Geometry and Trade-Offs Kasimir Tanner Matteo Vilucchio Bruno Loureiro Florent Krzakala AAML 70 0 0 31 Dec 2024
High-dimensional Analysis of Knowledge Distillation: Weak-to-Strong Generalization and Scaling Laws M. E. Ildiz Halil Alperen Gozeten Ege Onur Taga Marco Mondelli Samet Oymak 69 2 0 24 Oct 2024
Provable Weak-to-Strong Generalization via Benign Overfitting David X. Wu A. Sahai 79 6 0 06 Oct 2024
How Feature Learning Can Improve Neural Scaling Laws Blake Bordelon Alexander B. Atanasov Cengiz Pehlevan 59 13 0 26 Sep 2024
Breaking Neural Network Scaling Laws with Modularity Akhilan Boopathy Sunshine Jiang William Yue Jaedong Hwang Abhiram Iyer Ila Fiete OOD 62 2 0 09 Sep 2024
Overfitting Behaviour of Gaussian Kernel Ridgeless Regression: Varying Bandwidth or Dimensionality Marko Medvedev Gal Vardi Nathan Srebro 70 3 0 05 Sep 2024
Parameter-Efficient Fine-Tuning for Continual Learning: A Neural Tangent Kernel Perspective Jingren Liu Zhong Ji YunLong Yu Jiale Cao Yanwei Pang Jungong Han Xuelong Li CLL 46 5 0 24 Jul 2024
Scaling Laws in Linear Regression: Compute, Parameters, and Data Licong Lin Jingfeng Wu Sham Kakade Peter L. Bartlett Jason D. Lee LRM 49 15 0 12 Jun 2024
NTK-Guided Few-Shot Class Incremental Learning Jingren Liu Zhong Ji Yanwei Pang YunLong Yu CLL 47 3 0 19 Mar 2024
Active Few-Shot Fine-Tuning Jonas Hübotter Bhavya Sukhija Lenart Treven Yarden As Andreas Krause 48 1 0 13 Feb 2024
Robust Linear Regression: Phase-Transitions and Precise Tradeoffs for General Norms Elvis Dohmatob M. Scetbon AAML OOD 30 0 0 01 Aug 2023
Precise Asymptotic Generalization for Multiclass Classification with Overparameterized Linear Models David X. Wu A. Sahai 29 2 0 23 Jun 2023
Subsample Ridge Ensembles: Equivalences and Generalized Cross-Validation Jin-Hong Du Pratik V. Patil Arun K. Kuchibhotla 37 11 0 25 Apr 2023
On the Stepwise Nature of Self-Supervised Learning James B. Simon Maksis Knutins Liu Ziyin Daniel Geisz Abraham J. Fetterman Joshua Albrecht SSL 42 30 0 27 Mar 2023
Differentially Private Neural Tangent Kernels for Privacy-Preserving Data Generation Yilin Yang Kamil Adamczewski Danica J. Sutherland Xiaoxiao Li Mijung Park 38 14 0 03 Mar 2023
Spectral Evolution and Invariance in Linear-width Neural Networks Zhichao Wang A. Engel Anand D. Sarwate Ioana Dumitriu Tony Chiang 45 14 0 11 Nov 2022
A picture of the space of typical learnable tasks Rahul Ramesh Jialin Mao Itay Griniasty Rubing Yang H. Teoh Mark K. Transtrum James P. Sethna Pratik Chaudhari SSL DRL 46 5 0 31 Oct 2022
A Solvable Model of Neural Scaling Laws A. Maloney Daniel A. Roberts J. Sully 52 51 0 30 Oct 2022
A Kernel-Based View of Language Model Fine-Tuning Sadhika Malladi Alexander Wettig Dingli Yu Danqi Chen Sanjeev Arora VLM 78 63 0 11 Oct 2022
Making Look-Ahead Active Learning Strategies Feasible with Neural Tangent Kernels Mohamad Amin Mohamadi Wonho Bae Danica J. Sutherland 35 20 0 25 Jun 2022
Universal characteristics of deep neural network loss surfaces from random matrix theory Nicholas P. Baskerville J. Keating F. Mezzadri J. Najnudel Diego Granziol 37 4 0 17 May 2022
Explaining Neural Scaling Laws Yasaman Bahri Ethan Dyer Jared Kaplan Jaehoon Lee Utkarsh Sharma 32 250 0 12 Feb 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 266 4,532 0 23 Jan 2020
Norm-Based Capacity Control in Neural Networks Behnam Neyshabur Ryota Tomioka Nathan Srebro 127 577 0 27 Feb 2015