Deep Equals Shallow for ReLU Networks in Kernel Regimes

30 September 2020

Papers citing "Deep Equals Shallow for ReLU Networks in Kernel Regimes"

44 / 44 papers shown

Title
Spectral complexity of deep neural networks Simmaco Di Lillo Domenico Marinucci Michele Salvi Stefano Vigogna BDL 103 2 0 15 May 2024
Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHS Lin Chen Sheng Xu 122 94 0 22 Sep 2020
Generalized Leverage Score Sampling for Neural Networks Jason D. Lee Ruoqi Shen Zhao Song Mengdi Wang Zheng Yu 56 43 0 21 Sep 2020
On the Similarity between the Laplace and Neural Tangent Kernels Amnon Geifman A. Yadav Yoni Kasten Meirav Galun David Jacobs Ronen Basri 115 94 0 03 Jul 2020
Towards Understanding Hierarchical Learning: Benefits of Neural Representations Minshuo Chen Yu Bai Jason D. Lee T. Zhao Huan Wang Caiming Xiong R. Socher SSL 66 49 0 24 Jun 2020
Training (Overparametrized) Neural Networks in Near-Linear Time Jan van den Brand Binghui Peng Zhao Song Omri Weinstein ODL 55 83 0 20 Jun 2020
Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networks Z. Fan Zhichao Wang 105 73 0 25 May 2020
Frequency Bias in Neural Networks for Input of Non-Uniform Density Ronen Basri Meirav Galun Amnon Geifman David Jacobs Yoni Kasten S. Kritchman 75 184 0 10 Mar 2020
A Spectral Analysis of Dot-product Kernels M. Scetbon Zaïd Harchaoui 380 2 0 28 Feb 2020
Towards Understanding the Spectral Bias of Deep Learning Yuan Cao Zhiying Fang Yue Wu Ding-Xuan Zhou Quanquan Gu 95 218 0 03 Dec 2019
On the Multiple Descent of Minimum-Norm Interpolants and Restricted Lower Isometry of Kernels Tengyuan Liang Alexander Rakhlin Xiyu Zhai 61 29 0 27 Aug 2019
A Fine-Grained Spectral Perspective on Neural Networks Greg Yang Hadi Salman 76 113 0 24 Jul 2019
Quadratic Suffices for Over-parametrization via Matrix Chernoff Bound Zhao Song Xin Yang 60 91 0 09 Jun 2019
The Convergence Rate of Neural Networks for Learned Functions of Different Frequencies Ronen Basri David Jacobs Yoni Kasten S. Kritchman 64 218 0 02 Jun 2019
On the Inductive Bias of Neural Tangent Kernels A. Bietti Julien Mairal 83 257 0 29 May 2019
Linearized two-layers neural networks in high dimension Behrooz Ghorbani Song Mei Theodor Misiakiewicz Andrea Montanari MLT 54 243 0 27 Apr 2019
On Exact Computation with an Infinitely Wide Neural Net Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruslan Salakhutdinov Ruosong Wang 223 923 0 26 Apr 2019
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruosong Wang MLT 197 972 0 24 Jan 2019
On Lazy Training in Differentiable Programming Lénaïc Chizat Edouard Oyallon Francis R. Bach 111 833 0 19 Dec 2018
Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks Difan Zou Yuan Cao Dongruo Zhou Quanquan Gu ODL 185 448 0 21 Nov 2018
Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers Zeyuan Allen-Zhu Yuanzhi Li Yingyu Liang MLT 186 773 0 12 Nov 2018
A Convergence Theory for Deep Learning via Over-Parameterization Zeyuan Allen-Zhu Yuanzhi Li Zhao Song AI4CE ODL 247 1,463 0 09 Nov 2018
Gradient Descent Finds Global Minima of Deep Neural Networks S. Du Jason D. Lee Haochuan Li Liwei Wang Masayoshi Tomizuka ODL 201 1,135 0 09 Nov 2018
Gradient Descent Provably Optimizes Over-parameterized Neural Networks S. Du Xiyu Zhai Barnabás Póczós Aarti Singh MLT ODL 219 1,272 0 04 Oct 2018
Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data Yuanzhi Li Yingyu Liang MLT 216 652 0 03 Aug 2018
Neural Tangent Kernel: Convergence and Generalization in Neural Networks Arthur Jacot Franck Gabriel Clément Hongler 267 3,203 0 20 Jun 2018
On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport Lénaïc Chizat Francis R. Bach OT 204 735 0 24 May 2018
Gaussian Process Behaviour in Wide Deep Neural Networks A. G. Matthews Mark Rowland Jiri Hron Richard Turner Zoubin Ghahramani BDL 144 559 0 30 Apr 2018
A Mean Field View of the Landscape of Two-Layers Neural Networks Song Mei Andrea Montanari Phan-Minh Nguyen MLT 91 858 0 18 Apr 2018
To understand deep learning we need to understand kernel learning M. Belkin Siyuan Ma Soumik Mandal 60 419 0 05 Feb 2018
Deep Neural Networks as Gaussian Processes Jaehoon Lee Yasaman Bahri Roman Novak S. Schoenholz Jeffrey Pennington Jascha Narain Sohl-Dickstein UQCV BDL 122 1,093 0 01 Nov 2017
Nonparametric regression using deep neural networks with ReLU activation function Johannes Schmidt-Hieber 225 810 0 22 Aug 2017
Depth Separation for Neural Networks Amit Daniely MDE 37 74 0 27 Feb 2017
Diverse Neural Network Learns True Target Functions Bo Xie Yingyu Liang Le Song 169 138 0 09 Nov 2016
Error bounds for approximations with deep ReLU networks Dmitry Yarotsky 195 1,227 0 03 Oct 2016
Deep vs. shallow networks : An approximation theory perspective H. Mhaskar T. Poggio 165 341 0 10 Aug 2016
Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity Amit Daniely Roy Frostig Y. Singer 163 343 0 18 Feb 2016
Benefits of depth in neural networks Matus Telgarsky 354 608 0 14 Feb 2016
Generalization Properties of Learning with Random Features Alessandro Rudi Lorenzo Rosasco MLT 68 331 0 14 Feb 2016
The Power of Depth for Feedforward Neural Networks Ronen Eldan Ohad Shamir 216 732 0 12 Dec 2015
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification Kaiming He Xinming Zhang Shaoqing Ren Jian Sun VLM 323 18,613 0 06 Feb 2015
Breaking the Curse of Dimensionality with Convex Neural Networks Francis R. Bach 182 706 0 30 Dec 2014
Sharp analysis of low-rank kernel matrix approximations Francis R. Bach 158 282 0 09 Aug 2012
The spectrum of kernel random matrices N. Karoui 154 224 0 04 Jan 2010