Neural Tangent Kernel: Convergence and Generalization in Neural Networks

20 June 2018

Papers citing "Neural Tangent Kernel: Convergence and Generalization in Neural Networks"

50 / 2,163 papers shown

Title
Mildly Overparametrized Neural Nets can Memorize Training Data Efficiently Rong Ge Runzhe Wang Haoyu Zhao TDI 18 20 0 26 Sep 2019
Asymptotics of Wide Networks from Feynman Diagrams Ethan Dyer Guy Gur-Ari 32 114 0 25 Sep 2019
Dynamics of Deep Neural Networks and Neural Tangent Hierarchy Jiaoyang Huang H. Yau 30 147 0 18 Sep 2019
Finite Depth and Width Corrections to the Neural Tangent Kernel Boris Hanin Mihai Nica MDE 30 149 0 13 Sep 2019
Additive function approximation in the brain K. Harris 32 13 0 05 Sep 2019
Neural Policy Gradient Methods: Global Optimality and Rates of Convergence Lingxiao Wang Qi Cai Zhuoran Yang Zhaoran Wang 25 236 0 29 Aug 2019
Deep Learning Theory Review: An Optimal Control and Dynamical Systems Perspective Guan-Horng Liu Evangelos A. Theodorou AI4CE 27 71 0 28 Aug 2019
On the Multiple Descent of Minimum-Norm Interpolants and Restricted Lower Isometry of Kernels Tengyuan Liang Alexander Rakhlin Xiyu Zhai 26 29 0 27 Aug 2019
Finite size corrections for neural network Gaussian processes J. Antognini BDL 30 30 0 27 Aug 2019
Effect of Activation Functions on the Training of Overparametrized Neural Nets A. Panigrahi Abhishek Shetty Navin Goyal 19 20 0 16 Aug 2019
The generalization error of random features regression: Precise asymptotics and double descent curve Song Mei Andrea Montanari 62 625 0 14 Aug 2019
A Fine-Grained Spectral Perspective on Neural Networks Greg Yang Hadi Salman 35 111 0 24 Jul 2019
Sparse Optimization on Measures with Over-parameterized Gradient Descent Lénaïc Chizat 21 92 0 24 Jul 2019
Order and Chaos: NTK views on DNN Normalization, Checkerboard and Boundary Artifacts Arthur Jacot Franck Gabriel François Ged Clément Hongler 19 23 0 11 Jul 2019
Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model Guodong Zhang Lala Li Zachary Nado James Martens Sushant Sachdeva George E. Dahl Christopher J. Shallue Roger C. Grosse 28 149 0 09 Jul 2019
Scaling Limit of Neural Networks with the Xavier Initialization and Convergence to a Global Minimum Justin A. Sirignano K. Spiliopoulos 19 14 0 09 Jul 2019
Weight-space symmetry in deep networks gives rise to permutation saddles, connected by equal-loss valleys across the loss landscape Johanni Brea Berfin Simsek Bernd Illing W. Gerstner 23 55 0 05 Jul 2019
On Symmetry and Initialization for Neural Networks Ido Nachum Amir Yehudayoff MLT 36 5 0 01 Jul 2019
Benign Overfitting in Linear Regression Peter L. Bartlett Philip M. Long Gábor Lugosi Alexander Tsigler MLT 8 763 0 26 Jun 2019
Neural Proximal/Trust Region Policy Optimization Attains Globally Optimal Policy Boyi Liu Qi Cai Zhuoran Yang Zhaoran Wang 30 108 0 25 Jun 2019
Limitations of Lazy Training of Two-layers Neural Networks Behrooz Ghorbani Song Mei Theodor Misiakiewicz Andrea Montanari MLT 24 143 0 21 Jun 2019
The Functional Neural Process Christos Louizos Xiahan Shi Klamer Schutte Max Welling BDL 38 77 0 19 Jun 2019
Disentangling feature and lazy training in deep neural networks Mario Geiger S. Spigler Arthur Jacot M. Wyart 17 17 0 19 Jun 2019
Convergence of Adversarial Training in Overparametrized Neural Networks Ruiqi Gao Tianle Cai Haochuan Li Liwei Wang Cho-Jui Hsieh Jason D. Lee AAML 18 107 0 19 Jun 2019
Gradient Dynamics of Shallow Univariate ReLU Networks Francis Williams Matthew Trager Claudio Silva Daniele Panozzo Denis Zorin Joan Bruna 19 79 0 18 Jun 2019
Dynamics of stochastic gradient descent for two-layer neural networks in the teacher-student setup Sebastian Goldt Madhu S. Advani Andrew M. Saxe Florent Krzakala Lenka Zdeborová MLT 27 140 0 18 Jun 2019
Meta-learning Pseudo-differential Operators with Deep Neural Networks Jordi Feliu-Fabà Yuwei Fan Lexing Ying 22 39 0 16 Jun 2019
Gradient Descent Maximizes the Margin of Homogeneous Neural Networks Kaifeng Lyu Jian Li 52 324 0 13 Jun 2019
Kernel and Rich Regimes in Overparametrized Models Blake E. Woodworth Suriya Gunasekar Pedro H. P. Savarese E. Moroshko Itay Golan Jason D. Lee Daniel Soudry Nathan Srebro 30 353 0 13 Jun 2019
Generalization Guarantees for Neural Networks via Harnessing the Low-rank Structure of the Jacobian Samet Oymak Zalan Fabian Mingchen Li Mahdi Soltanolkotabi MLT 21 88 0 12 Jun 2019
Learning Curves for Deep Neural Networks: A Gaussian Field Theory Perspective Omry Cohen Orit Malka Zohar Ringel AI4CE 23 21 0 12 Jun 2019
An Improved Analysis of Training Over-parameterized Deep Neural Networks Difan Zou Quanquan Gu 26 230 0 11 Jun 2019
Quadratic Suffices for Over-parametrization via Matrix Chernoff Bound Zhao Song Xin Yang 20 91 0 09 Jun 2019
The Normalization Method for Alleviating Pathological Sharpness in Wide Neural Networks Ryo Karakida S. Akaho S. Amari 27 40 0 07 Jun 2019
Approximate Inference Turns Deep Networks into Gaussian Processes Mohammad Emtiyaz Khan Alexander Immer Ehsan Abedi M. Korzepa UQCV BDL 42 122 0 05 Jun 2019
Global Optimality Guarantees For Policy Gradient Methods Jalaj Bhandari Daniel Russo 37 186 0 05 Jun 2019
Deep ReLU Networks Have Surprisingly Few Activation Patterns Boris Hanin David Rolnick 21 221 0 03 Jun 2019
A Mean Field Theory of Quantized Deep Networks: The Quantization-Depth Trade-Off Yaniv Blumenfeld D. Gilboa Daniel Soudry MQ 15 14 0 03 Jun 2019
A mean-field limit for certain deep neural networks Dyego Araújo R. Oliveira Daniel Yukimura AI4CE 19 69 0 01 Jun 2019
Exact Convergence Rates of the Neural Tangent Kernel in the Large Depth Limit Soufiane Hayou Arnaud Doucet Judith Rousseau 16 4 0 31 May 2019
What Can Neural Networks Reason About? Keyulu Xu Jingling Li Mozhi Zhang S. Du Ken-ichi Kawarabayashi Stefanie Jegelka NAI AI4CE 27 240 0 30 May 2019
Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks Yuan Cao Quanquan Gu MLT AI4CE 37 383 0 30 May 2019
Graph Neural Tangent Kernel: Fusing Graph Neural Networks with Graph Kernels S. Du Kangcheng Hou Barnabás Póczós Ruslan Salakhutdinov Ruosong Wang Keyulu Xu 23 271 0 30 May 2019
Norm-based generalisation bounds for multi-class convolutional neural networks Antoine Ledent Waleed Mustafa Yunwen Lei Marius Kloft 18 5 0 29 May 2019
Geometric Insights into the Convergence of Nonlinear TD Learning David Brandfonbrener Joan Bruna 17 15 0 29 May 2019
On the Inductive Bias of Neural Tangent Kernels A. Bietti Julien Mairal 28 253 0 29 May 2019
Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for Regression Problems Tianle Cai Ruiqi Gao Jikai Hou Siyu Chen Dong Wang Di He Zhihua Zhang Liwei Wang ODL 21 57 0 28 May 2019
Simple and Effective Regularization Methods for Training on Noisily Labeled Data with Generalization Guarantee Wei Hu Zhiyuan Li Dingli Yu NoLa 25 12 0 27 May 2019
Infinitely deep neural networks as diffusion processes Stefano Peluchetti Stefano Favaro ODL 16 31 0 27 May 2019
Scalable Training of Inference Networks for Gaussian-Process Models Jiaxin Shi Mohammad Emtiyaz Khan Jun Zhu BDL 27 18 0 27 May 2019