v1v2 (latest)

Neural Networks as Kernel Learners: The Silent Alignment Effect

29 October 2021

Alexander B. Atanasov

Papers citing "Neural Networks as Kernel Learners: The Silent Alignment Effect"

38 / 38 papers shown

Title
Training NTK to Generalize with KARE Johannes Schwab Bryan Kelly Semyon Malamud Teng Andrea Xu 130 0 0 16 May 2025
Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer Blake Bordelon Cengiz Pehlevan AI4CE 193 1 0 04 Feb 2025
Flexible task abstractions emerge in linear networks with fast and bounded units Kai Sandbrink Jan P. Bauer A. Proca Andrew M. Saxe Christopher Summerfield Ali Hummos 113 2 0 17 Jan 2025
Geometric Inductive Biases of Deep Networks: The Role of Data and Architecture Sajad Movahedi Antonio Orvieto Seyed-Mohsen Moosavi-Dezfooli AI4CE AAML 576 0 0 15 Oct 2024
SGD with memory: fundamental properties and stochastic acceleration Dmitry Yarotsky Maksim Velikanov 98 1 0 05 Oct 2024
How Feature Learning Can Improve Neural Scaling Laws Blake Bordelon Alexander B. Atanasov Cengiz Pehlevan 114 17 0 26 Sep 2024
Repetita Iuvant: Data Repetition Allows SGD to Learn High-Dimensional Multi-Index Functions Luca Arnaboldi Yatin Dandi Florent Krzakala Luca Pesce Ludovic Stephan 113 18 0 24 May 2024
Early Directional Convergence in Deep Homogeneous Neural Networks for Small Initializations Akshay Kumar Jarvis Haupt ODL 93 4 0 12 Mar 2024
A Framework and Benchmark for Deep Batch Active Learning for Regression David Holzmüller Viktor Zaverkin Johannes Kastner Ingo Steinwart UQCV BDL GP 98 37 0 17 Mar 2022
Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction Dominik Stöger Mahdi Soltanolkotabi ODL 74 78 0 28 Jun 2021
Every Model Learned by Gradient Descent Is Approximately a Kernel Machine Pedro M. Domingos MLT 87 73 0 30 Nov 2020
Feature Learning in Infinite-Width Neural Networks Greg Yang J. E. Hu MLT 99 156 0 30 Nov 2020
Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel Stanislav Fort Gintare Karolina Dziugaite Mansheej Paul Sepideh Kharaghani Daniel M. Roy Surya Ganguli 109 193 0 28 Oct 2020
Label-Aware Neural Tangent Kernel: Toward Better Generalization and Local Elasticity Shuxiao Chen Hangfeng He Weijie J. Su 45 24 0 22 Oct 2020
A Unifying View on Implicit Bias in Training Linear Neural Networks Chulhee Yun Shankar Krishnan H. Mobahi MLT 92 82 0 06 Oct 2020
On the linearity of large non-linear models: when and why the tangent kernel is constant Chaoyue Liu Libin Zhu M. Belkin 129 143 0 02 Oct 2020
Asymptotics of Wide Convolutional Neural Networks Anders Andreassen Ethan Dyer 72 23 0 19 Aug 2020
Finite Versus Infinite Neural Networks: an Empirical Study Jaehoon Lee S. Schoenholz Jeffrey Pennington Ben Adlam Lechao Xiao Roman Novak Jascha Narain Sohl-Dickstein 80 214 0 31 Jul 2020
Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networks Abdulkadir Canatar Blake Bordelon Cengiz Pehlevan 118 189 0 23 Jun 2020
On the asymptotics of wide networks with polynomial activations Kyle Aitken Guy Gur-Ari 54 23 0 11 Jun 2020
Taylorized Training: Towards Better Approximation of Neural Network Training at Finite Width Yu Bai Ben Krause Huan Wang Caiming Xiong R. Socher 64 22 0 10 Feb 2020
Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networks Blake Bordelon Abdulkadir Canatar Cengiz Pehlevan 261 208 0 07 Feb 2020
Neural Tangents: Fast and Easy Infinite Neural Networks in Python Roman Novak Lechao Xiao Jiri Hron Jaehoon Lee Alexander A. Alemi Jascha Narain Sohl-Dickstein S. Schoenholz 88 230 0 05 Dec 2019
Asymptotics of Wide Networks from Feynman Diagrams Ethan Dyer Guy Gur-Ari 95 115 0 25 Sep 2019
Kernel and Rich Regimes in Overparametrized Models Blake E. Woodworth Suriya Gunasekar Pedro H. P. Savarese E. Moroshko Itay Golan Jason D. Lee Daniel Soudry Nathan Srebro 84 366 0 13 Jun 2019
On Exact Computation with an Infinitely Wide Neural Net Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruslan Salakhutdinov Ruosong Wang 243 928 0 26 Apr 2019
Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent Jaehoon Lee Lechao Xiao S. Schoenholz Yasaman Bahri Roman Novak Jascha Narain Sohl-Dickstein Jeffrey Pennington 213 1,110 0 18 Feb 2019
On Lazy Training in Differentiable Programming Lénaïc Chizat Edouard Oyallon Francis R. Bach 111 840 0 19 Dec 2018
Just Interpolate: Kernel "Ridgeless" Regression Can Generalize Tengyuan Liang Alexander Rakhlin 89 355 0 01 Aug 2018
Neural Tangent Kernel: Convergence and Generalization in Neural Networks Arthur Jacot Franck Gabriel Clément Hongler 275 3,225 0 20 Jun 2018
Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balanced S. Du Wei Hu Jason D. Lee MLT 145 243 0 04 Jun 2018
On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization Sanjeev Arora Nadav Cohen Elad Hazan 120 487 0 19 Feb 2018
To understand deep learning we need to understand kernel learning M. Belkin Siyuan Ma Soumik Mandal 75 420 0 05 Feb 2018
Understanding deep learning requires rethinking generalization Chiyuan Zhang Samy Bengio Moritz Hardt Benjamin Recht Oriol Vinyals HAI 351 4,636 0 10 Nov 2016
Understanding intermediate layers using linear classifier probes Guillaume Alain Yoshua Bengio FAtt 175 958 0 05 Oct 2016
Wide Residual Networks Sergey Zagoruyko N. Komodakis 362 8,005 0 23 May 2016
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 2.1K 150,433 0 22 Dec 2014
Algorithms for Learning Kernels Based on Centered Alignment Corinna Cortes M. Mohri Afshin Rostamizadeh 98 547 0 02 Mar 2012