Finite Depth and Width Corrections to the Neural Tangent Kernel

13 September 2019

Papers citing "Finite Depth and Width Corrections to the Neural Tangent Kernel"

42 / 42 papers shown

Title
Statistically guided deep learning Michael Kohler A. Krzyżak ODL BDL 79 0 0 11 Apr 2025
Feature Learning Beyond the Edge of Stability Dávid Terjék MLT 46 0 0 18 Feb 2025
Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer Blake Bordelon Cengiz Pehlevan AI4CE 64 1 0 04 Feb 2025
On the Neural Tangent Kernel of Equilibrium Models Zhili Feng J. Zico Kolter 18 6 0 21 Oct 2023
Quantitative CLTs in Deep Neural Networks Stefano Favaro Boris Hanin Domenico Marinucci I. Nourdin G. Peccati BDL 33 12 0 12 Jul 2023
Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networks Blake Bordelon Cengiz Pehlevan MLT 38 29 0 06 Apr 2023
Effective Theory of Transformers at Initialization Emily Dinan Sho Yaida Susan Zhang 30 14 0 04 Apr 2023
VIPeR: Provably Efficient Algorithm for Offline RL with Neural Function Approximation Thanh Nguyen-Tang R. Arora OffRL 46 5 0 24 Feb 2023
Dataset Distillation with Convexified Implicit Gradients Noel Loo Ramin Hasani Mathias Lechner Daniela Rus DD 31 41 0 13 Feb 2023
Efficient Parametric Approximations of Neural Network Function Space Distance Nikita Dhawan Sicong Huang Juhan Bae Roger C. Grosse 16 5 0 07 Feb 2023
Understanding Reconstruction Attacks with the Neural Tangent Kernel and Dataset Distillation Noel Loo Ramin Hasani Mathias Lechner Alexander Amini Daniela Rus DD 42 5 0 02 Feb 2023
Width and Depth Limits Commute in Residual Networks Soufiane Hayou Greg Yang 42 14 0 01 Feb 2023
ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradients Guihong Li Yuedong Yang Kartikeya Bhardwaj R. Marculescu 36 61 0 26 Jan 2023
Expected Gradients of Maxout Networks and Consequences to Parameter Initialization Hanna Tseran Guido Montúfar ODL 30 0 0 17 Jan 2023
Effects of Data Geometry in Early Deep Learning Saket Tiwari George Konidaris 79 7 0 29 Dec 2022
The Curious Case of Benign Memorization Sotiris Anagnostidis Gregor Bachmann Lorenzo Noci Thomas Hofmann AAML 49 8 0 25 Oct 2022
Evolution of Neural Tangent Kernels under Benign and Adversarial Training Noel Loo Ramin Hasani Alexander Amini Daniela Rus AAML 34 13 0 21 Oct 2022
Global Convergence of SGD On Two Layer Neural Nets Pulkit Gopalani Anirbit Mukherjee 26 5 0 20 Oct 2022
Meta-Principled Family of Hyperparameter Scaling Strategies Sho Yaida 58 16 0 10 Oct 2022
Analysis of the rate of convergence of an over-parametrized deep neural network estimate learned by gradient descent Michael Kohler A. Krzyżak 32 10 0 04 Oct 2022
Approximation results for Gradient Descent trained Shallow Neural Networks in $1d$ R. Gentile G. Welper ODL 52 6 0 17 Sep 2022
On the universal consistency of an over-parametrized deep neural network estimate learned by gradient descent Selina Drews Michael Kohler 30 13 0 30 Aug 2022
Fast Finite Width Neural Tangent Kernel Roman Novak Jascha Narain Sohl-Dickstein S. Schoenholz AAML 22 53 0 17 Jun 2022
Transition to Linearity of General Neural Networks with Directed Acyclic Graph Architecture Libin Zhu Chaoyue Liu M. Belkin GNN AI4CE 23 4 0 24 May 2022
Gaussian Pre-Activations in Neural Networks: Myth or Reality? Pierre Wolinski Julyan Arbel AI4CE 73 8 0 24 May 2022
On Feature Learning in Neural Networks with Global Convergence Guarantees Zhengdao Chen Eric Vanden-Eijnden Joan Bruna MLT 36 13 0 22 Apr 2022
Offline Neural Contextual Bandits: Pessimism, Optimization and Generalization Thanh Nguyen-Tang Sunil R. Gupta A. Nguyen Svetha Venkatesh OffRL 29 28 0 27 Nov 2021
Deep Active Learning by Leveraging Training Dynamics Haonan Wang Wei Huang Ziwei Wu A. Margenot Hanghang Tong Jingrui He AI4CE 27 33 0 16 Oct 2021
On the Impact of Stable Ranks in Deep Nets B. Georgiev L. Franken Mayukh Mukherjee Georgios Arvanitidis 21 3 0 05 Oct 2021
Convergence of Deep ReLU Networks Yuesheng Xu Haizhang Zhang 37 26 0 27 Jul 2021
Random Neural Networks in the Infinite Width Limit as Gaussian Processes Boris Hanin BDL 32 43 0 04 Jul 2021
Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptation Haoxiang Wang Han Zhao Bo-wen Li 37 88 0 16 Jun 2021
The Future is Log-Gaussian: ResNets and Their Infinite-Depth-and-Width Limit at Initialization Mufan Li Mihai Nica Daniel M. Roy 30 33 0 07 Jun 2021
Priors in Bayesian Deep Learning: A Review Vincent Fortuin UQCV BDL 31 124 0 14 May 2021
Unsupervised Shape Completion via Deep Prior in the Neural Tangent Kernel Perspective Lei Chu Hao Pan Wenping Wang 3DPC 34 11 0 19 Apr 2021
Deep ReLU Networks Preserve Expected Length Boris Hanin Ryan Jeong David Rolnick 29 14 0 21 Feb 2021
Explaining Neural Scaling Laws Yasaman Bahri Ethan Dyer Jared Kaplan Jaehoon Lee Utkarsh Sharma 27 250 0 12 Feb 2021
Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning Zeyuan Allen-Zhu Yuanzhi Li FedML 60 355 0 17 Dec 2020
Tensor Programs II: Neural Tangent Kernel for Any Architecture Greg Yang 58 134 0 25 Jun 2020
Feature Purification: How Adversarial Training Performs Robust Deep Learning Zeyuan Allen-Zhu Yuanzhi Li MLT AAML 35 147 0 20 May 2020
Predicting the outputs of finite deep neural networks trained with noisy gradients Gadi Naveh Oded Ben-David H. Sompolinsky Zohar Ringel 19 20 0 02 Apr 2020
Scaling description of generalization with number of parameters in deep learning Mario Geiger Arthur Jacot S. Spigler Franck Gabriel Levent Sagun Stéphane dÁscoli Giulio Biroli Clément Hongler M. Wyart 49 195 0 06 Jan 2019