On the linearity of large non-linear models: when and why the tangent kernel is constant

2 October 2020

Papers citing "On the linearity of large non-linear models: when and why the tangent kernel is constant"

50 / 98 papers shown

Title
Ultra-fast feature learning for the training of two-layer neural networks in the two-timescale regime Raphael Barboni Gabriel Peyré François-Xavier Vialard MLT 34 0 0 25 Apr 2025
Divergence of Empirical Neural Tangent Kernel in Classification Problems Zixiong Yu Songtao Tian Guhan Chen 23 0 0 15 Apr 2025
Globally Convergent Variational Inference Declan McNamara J. Loper Jeffrey Regier 53 0 0 14 Jan 2025
Infinite Width Limits of Self Supervised Neural Networks Maximilian Fleissner Gautham Govind Anil D. Ghoshdastidar SSL 145 0 0 17 Nov 2024
Sharper Guarantees for Learning Neural Network Classifiers with Gradient Methods Hossein Taheri Christos Thrampoulidis Arya Mazumdar MLT 33 0 0 13 Oct 2024
NTK-DFL: Enhancing Decentralized Federated Learning in Heterogeneous Settings via Neural Tangent Kernel Gabriel Thompson Kai Yue Chau-Wai Wong H. Dai FedML 13 0 0 02 Oct 2024
Peer-to-Peer Learning Dynamics of Wide Neural Networks Shreyas Chaudhari Srinivasa Pranav Emile Anand José M. F. Moura 37 3 0 23 Sep 2024
Variational Search Distributions Daniel M. Steinberg Rafael Oliveira Cheng Soon Ong Edwin V. Bonilla 33 0 0 10 Sep 2024
Continual learning with the neural tangent ensemble Ari S. Benjamin Christian Pehle Kyle Daruwalla UQCV 67 0 0 30 Aug 2024
Provable Robustness of (Graph) Neural Networks Against Data Poisoning and Backdoor Attacks Lukas Gosch Mahalakshmi Sabanayagam D. Ghoshdastidar Stephan Günnemann AAML 38 3 0 15 Jul 2024
Simplifying Deep Temporal Difference Learning Matteo Gallici Mattie Fellows Benjamin Ellis B. Pou Ivan Masmitja Jakob Foerster Mario Martin OffRL 59 14 0 05 Jul 2024
Accelerated Stochastic Min-Max Optimization Based on Bias-corrected Momentum H. Cai Sulaiman A. Alghunaim Ali H.Sayed 43 1 0 18 Jun 2024
Loss Gradient Gaussian Width based Generalization and Optimization Guarantees A. Banerjee Qiaobo Li Yingxue Zhou 44 0 0 11 Jun 2024
Reparameterization invariance in approximate Bayesian inference Hrittik Roy M. Miani Carl Henrik Ek Philipp Hennig Marvin Pfortner Lukas Tatzel Søren Hauberg BDL 47 8 0 05 Jun 2024
Scalable Optimization in the Modular Norm Tim Large Yang Liu Minyoung Huh Hyojin Bahng Phillip Isola Jeremy Bernstein 44 12 0 23 May 2024
Bounds for the smallest eigenvalue of the NTK for arbitrary spherical data of arbitrary dimension Kedar Karhadkar Michael Murray Guido Montúfar 34 2 0 23 May 2024
An Improved Finite-time Analysis of Temporal Difference Learning with Deep Neural Networks Zhifa Ke Zaiwen Wen Junyu Zhang 29 0 0 07 May 2024
Physics-Informed Neural Networks: Minimizing Residual Loss with Wide Networks and Effective Activations Nima Hosseini Dashtbayaz G. Farhani Boyu Wang Charles X. Ling 26 1 0 02 May 2024
Understanding the training of infinitely deep and wide ResNets with Conditional Optimal Transport Raphael Barboni Gabriel Peyré Franccois-Xavier Vialard 37 3 0 19 Mar 2024
When can we Approximate Wide Contrastive Models with Neural Tangent Kernels and Principal Component Analysis? Gautham Govind Anil P. Esser D. Ghoshdastidar 40 1 0 13 Mar 2024
"Lossless" Compression of Deep Neural Networks: A High-dimensional Neural Tangent Kernel Approach Lingyu Gu Yongqiang Du Yuan Zhang Di Xie Shiliang Pu Robert C. Qiu Zhenyu Liao 36 6 0 01 Mar 2024
The Challenges of the Nonlinear Regime for Physics-Informed Neural Networks Andrea Bonfanti Giuseppe Bruno Cristina Cipriani 29 7 0 06 Feb 2024
Challenges in Training PINNs: A Loss Landscape Perspective Pratik Rathore Weimu Lei Zachary Frangella Lu Lu Madeleine Udell AI4CE PINN ODL 33 39 0 02 Feb 2024
Weak Correlations as the Underlying Principle for Linearization of Gradient-Based Learning Systems Ori Shem-Ur Yaron Oz 14 0 0 08 Jan 2024
On the Performance of Temporal Difference Learning With Neural Networks Haoxing Tian I. Paschalidis Alexander Olshevsky 16 5 0 08 Dec 2023
GGNNs : Generalizing GNNs using Residual Connections and Weighted Message Passing Abhinav Raghuvanshi K. Malleshappa AI4CE GNN 29 0 0 26 Nov 2023
On the Convergence of Encoder-only Shallow Transformers Yongtao Wu Fanghui Liu Grigorios G. Chrysos V. Cevher 42 5 0 02 Nov 2023
Efficient kernel surrogates for neural network-based regression S. Qadeer A. Engel Amanda A. Howard Adam Tsou Max Vargas P. Stinis Tony Chiang 13 5 0 28 Oct 2023
On the Optimization and Generalization of Multi-head Attention Puneesh Deora Rouzbeh Ghaderi Hossein Taheri Christos Thrampoulidis MLT 44 33 0 19 Oct 2023
Neural Tangent Kernels Motivate Graph Neural Networks with Cross-Covariance Graphs Shervin Khalafi Saurabh Sihag Alejandro Ribeiro 18 0 0 16 Oct 2023
An operator preconditioning perspective on training in physics-informed machine learning Tim De Ryck Florent Bonnet Siddhartha Mishra Emmanuel de Bezenac AI4CE 39 14 0 09 Oct 2023
Optimization Guarantees of Unfolded ISTA and ADMM Networks With Smooth Soft-Thresholding Shaik Basheeruddin Shah Pradyumna Pradhan Wei Pu Ramunaidu Randhi Miguel R. D. Rodrigues Yonina C. Eldar 22 4 0 12 Sep 2023
Non-Parametric Representation Learning with Kernels P. Esser Maximilian Fleissner D. Ghoshdastidar SSL 27 4 0 05 Sep 2023
Representation Learning Dynamics of Self-Supervised Models P. Esser Satyaki Mukherjee D. Ghoshdastidar SSL 19 2 0 05 Sep 2023
Six Lectures on Linearized Neural Networks Theodor Misiakiewicz Andrea Montanari 34 12 0 25 Aug 2023
Modify Training Directions in Function Space to Reduce Generalization Error Yi Yu Wenlian Lu Boyu Chen 24 0 0 25 Jul 2023
Neural Hilbert Ladders: Multi-Layer Neural Networks in Function Space Zhengdao Chen 41 1 0 03 Jul 2023
Catapults in SGD: spikes in the training loss and their impact on generalization through feature learning Libin Zhu Chaoyue Liu Adityanarayanan Radhakrishnan M. Belkin 30 13 0 07 Jun 2023
Aiming towards the minimizers: fast convergence of SGD for overparametrized problems Chaoyue Liu D. Drusvyatskiy M. Belkin Damek Davis Yi-An Ma ODL 20 16 0 05 Jun 2023
On Emergence of Clean-Priority Learning in Early Stopped Neural Networks Chaoyue Liu Amirhesam Abedsoltan M. Belkin NoLa 17 4 0 05 Jun 2023
Unifying GANs and Score-Based Diffusion as Generative Particle Models Jean-Yves Franceschi Mike Gartrell Ludovic Dos Santos Thibaut Issenhuth Emmanuel de Bezenac Mickaël Chen A. Rakotomamonjy DiffM 18 20 0 25 May 2023
ReLU soothes the NTK condition number and accelerates optimization for wide neural networks Chaoyue Liu Like Hui MLT 22 9 0 15 May 2023
Graph Neural Tangent Kernel: Convergence on Large Graphs Sanjukta Krishnagopal Luana Ruiz 28 8 0 25 Jan 2023
Convergence beyond the over-parameterized regime using Rayleigh quotients David A. R. Robin Kevin Scaman Marc Lelarge 25 3 0 19 Jan 2023
Mechanism of feature learning in deep fully connected networks and kernel machines that recursively learn features Adityanarayanan Radhakrishnan Daniel Beaglehole Parthe Pandit M. Belkin FAtt MLT 29 11 0 28 Dec 2022
The Quantum Path Kernel: a Generalized Quantum Neural Tangent Kernel for Deep Quantum Machine Learning Massimiliano Incudini Michele Grossi Antonio Mandarino S. Vallecorsa Alessandra Di Pierro David Windridge 31 6 0 22 Dec 2022
Graph Neural Networks are Inherently Good Generalizers: Insights by Bridging GNNs and MLPs Chenxiao Yang Qitian Wu Jiahua Wang Junchi Yan AI4CE 19 51 0 18 Dec 2022
Transfer Learning with Kernel Methods Adityanarayanan Radhakrishnan Max Ruiz Luyten Neha Prasad Caroline Uhler 14 18 0 01 Nov 2022
Global Convergence of SGD On Two Layer Neural Nets Pulkit Gopalani Anirbit Mukherjee 23 5 0 20 Oct 2022
Analysis of Convolutions, Non-linearity and Depth in Graph Neural Networks using Neural Tangent Kernel Mahalakshmi Sabanayagam P. Esser D. Ghoshdastidar 31 2 0 18 Oct 2022