Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networks

23 June 2020

Papers citing "Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networks"

39 / 39 papers shown

Title
Generalization through variance: how noise shapes inductive biases in diffusion models John J. Vastola DiffM 185 2 0 16 Apr 2025
On the similarity of bandwidth-tuned quantum kernels and classical kernels Roberto Flórez Ablan M. Roth Jan Schnabel 60 2 0 07 Mar 2025
Estimating the Spectral Moments of the Kernel Integral Operator from Finite Sample Matrices Chanwoo Chun SueYeon Chung Daniel D. Lee 31 1 0 23 Oct 2024
Breaking Neural Network Scaling Laws with Modularity Akhilan Boopathy Sunshine Jiang William Yue Jaedong Hwang Abhiram Iyer Ila Fiete OOD 41 2 0 09 Sep 2024
Overfitting Behaviour of Gaussian Kernel Ridgeless Regression: Varying Bandwidth or Dimensionality Marko Medvedev Gal Vardi Nathan Srebro 70 3 0 05 Sep 2024
Parameter-Efficient Fine-Tuning for Continual Learning: A Neural Tangent Kernel Perspective Jingren Liu Zhong Ji YunLong Yu Jiale Cao Yanwei Pang Jungong Han Xuelong Li CLL 42 5 0 24 Jul 2024
When does compositional structure yield compositional generalization? A kernel theory Samuel Lippl Kim Stachenfeld NAI CoGe 73 5 0 26 May 2024
Dissecting the Interplay of Attention Paths in a Statistical Mechanics Theory of Transformers Lorenzo Tiberi Francesca Mignacco Kazuki Irie H. Sompolinsky 44 6 0 24 May 2024
NTK-Guided Few-Shot Class Incremental Learning Jingren Liu Zhong Ji Yanwei Pang YunLong Yu CLL 39 3 0 19 Mar 2024
Flat Minima in Linear Estimation and an Extended Gauss Markov Theorem Simon Segert 32 0 0 18 Nov 2023
Modify Training Directions in Function Space to Reduce Generalization Error Yi Yu Wenlian Lu Boyu Chen 27 0 0 25 Jul 2023
Higher-order topological kernels via quantum computation Massimiliano Incudini F. Martini Alessandra Di Pierro 22 1 0 14 Jul 2023
Sparsity-depth Tradeoff in Infinitely Wide Deep Neural Networks Chanwoo Chun Daniel D. Lee BDL 35 2 0 17 May 2023
Do deep neural networks have an inbuilt Occam's razor? Chris Mingard Henry Rees Guillermo Valle Pérez A. Louis UQCV BDL 21 16 0 13 Apr 2023
On the Stepwise Nature of Self-Supervised Learning James B. Simon Maksis Knutins Liu Ziyin Daniel Geisz Abraham J. Fetterman Joshua Albrecht SSL 37 30 0 27 Mar 2023
Analyzing Convergence in Quantum Neural Networks: Deviations from Neural Tangent Kernels Xuchen You Shouvanik Chakrabarti Boyang Chen Xiaodi Wu 34 10 0 26 Mar 2023
Bayes-optimal Learning of Deep Random Networks of Extensive-width Hugo Cui Florent Krzakala Lenka Zdeborová BDL 20 35 0 01 Feb 2023
A Solvable Model of Neural Scaling Laws A. Maloney Daniel A. Roberts J. Sully 36 51 0 30 Oct 2022
Automatic and effective discovery of quantum kernels Massimiliano Incudini Daniele Lizzio Bosco F. Martini Michele Grossi Giuseppe Serra Alessandra Di Pierro 31 4 0 22 Sep 2022
On the Activation Function Dependence of the Spectral Bias of Neural Networks Q. Hong Jonathan W. Siegel Qinyan Tan Jinchao Xu 34 23 0 09 Aug 2022
Benign, Tempered, or Catastrophic: A Taxonomy of Overfitting Neil Rohit Mallinar James B. Simon Amirhesam Abedsoltan Parthe Pandit M. Belkin Preetum Nakkiran 24 37 0 14 Jul 2022
Target alignment in truncated kernel ridge regression Arash A. Amini R. Baumgartner Dai Feng 14 3 0 28 Jun 2022
Overcoming the Spectral Bias of Neural Value Approximation Ge Yang Anurag Ajay Pulkit Agrawal 34 25 0 09 Jun 2022
Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks Blake Bordelon Cengiz Pehlevan MLT 40 78 0 19 May 2022
Sharp Asymptotics of Kernel Ridge Regression Beyond the Linear Regime Hong Hu Yue M. Lu 53 15 0 13 May 2022
Contrasting random and learned features in deep Bayesian linear regression Jacob A. Zavatone-Veth William L. Tong Cengiz Pehlevan BDL MLT 28 26 0 01 Mar 2022
Tight Convergence Rate Bounds for Optimization Under Power Law Spectral Conditions Maksim Velikanov Dmitry Yarotsky 9 6 0 02 Feb 2022
Separation of Scales and a Thermodynamic Description of Feature Learning in Some CNNs Inbar Seroussi Gadi Naveh Zohar Ringel 33 50 0 31 Dec 2021
Learning Curves for Continual Learning in Neural Networks: Self-Knowledge Transfer and Forgetting Ryo Karakida S. Akaho CLL 32 11 0 03 Dec 2021
Learning with convolution and pooling operations in kernel methods Theodor Misiakiewicz Song Mei MLT 15 29 0 16 Nov 2021
Representation Learning via Quantum Neural Tangent Kernels Junyu Liu F. Tacchino Jennifer R. Glick Liang Jiang Antonio Mezzacapo 24 60 0 08 Nov 2021
Neural Networks as Kernel Learners: The Silent Alignment Effect Alexander B. Atanasov Blake Bordelon Cengiz Pehlevan MLT 26 75 0 29 Oct 2021
Locality defeats the curse of dimensionality in convolutional teacher-student scenarios Alessandro Favero Francesco Cagnetta M. Wyart 30 31 0 16 Jun 2021
A self consistent theory of Gaussian Processes captures feature learning effects in finite CNNs Gadi Naveh Zohar Ringel SSL MLT 36 31 0 08 Jun 2021
The Inductive Bias of Quantum Kernels Jonas M. Kubler Simon Buchholz Bernhard Schölkopf 19 119 0 07 Jun 2021
Explaining Neural Scaling Laws Yasaman Bahri Ethan Dyer Jared Kaplan Jaehoon Lee Utkarsh Sharma 27 250 0 12 Feb 2021
Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regime Stéphane dÁscoli Maria Refinetti Giulio Biroli Florent Krzakala 93 152 0 02 Mar 2020
Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networks Blake Bordelon Abdulkadir Canatar Cengiz Pehlevan 146 201 0 07 Feb 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 264 4,489 0 23 Jan 2020