v1v2v3 (latest)

Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation

13 February 2019

Papers citing "Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation"

50 / 57 papers shown

Title
Conditional Temporal Neural Processes with Covariance Loss Boseon Yoo Jiwoo Lee Janghoon Ju Seijun Chung Soyeon Kim Jaesik Choi 135 15 0 01 Apr 2025
On the Cone Effect in the Learning Dynamics Zhanpeng Zhou Yongyi Yang Jie Ren Mahito Sugiyama Junchi Yan 88 0 0 20 Mar 2025
Equivariant Neural Tangent Kernels Philipp Misof Pan Kessel Jan E. Gerken 176 0 0 10 Jun 2024
Neural Networks and Quantum Field Theory James Halverson Anindita Maiti Keegan Stoner 74 77 0 19 Aug 2020
Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes Greg Yang 116 199 0 28 Oct 2019
A Mean Field Theory of Batch Normalization Greg Yang Jeffrey Pennington Vinay Rao Jascha Narain Sohl-Dickstein S. Schoenholz 72 178 0 21 Feb 2019
Generalization Error Bounds of Gradient Descent for Learning Over-parameterized Deep ReLU Networks Yuan Cao Quanquan Gu ODL MLT AI4CE 79 157 0 04 Feb 2019
Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks Difan Zou Yuan Cao Dongruo Zhou Quanquan Gu ODL 187 448 0 21 Nov 2018
Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers Zeyuan Allen-Zhu Yuanzhi Li Yingyu Liang MLT 188 773 0 12 Nov 2018
A Convergence Theory for Deep Learning via Over-Parameterization Zeyuan Allen-Zhu Yuanzhi Li Zhao Song AI4CE ODL 264 1,463 0 09 Nov 2018
Gradient Descent Finds Global Minima of Deep Neural Networks S. Du Jason D. Lee Haochuan Li Liwei Wang Masayoshi Tomizuka ODL 203 1,135 0 09 Nov 2018
Exploration by Random Network Distillation Yuri Burda Harrison Edwards Amos Storkey Oleg Klimov 159 1,332 0 30 Oct 2018
On the Convergence Rate of Training Recurrent Neural Networks Zeyuan Allen-Zhu Yuanzhi Li Zhao Song 169 191 0 29 Oct 2018
A Gaussian Process perspective on Convolutional Neural Networks Anastasia Borovykh 77 19 0 25 Oct 2018
Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes Roman Novak Lechao Xiao Jaehoon Lee Yasaman Bahri Greg Yang Jiri Hron Daniel A. Abolafia Jeffrey Pennington Jascha Narain Sohl-Dickstein UQCV BDL 65 309 0 11 Oct 2018
Deep convolutional Gaussian processes Kenneth Blomqvist Samuel Kaski Markus Heinonen BDL 67 61 0 06 Oct 2018
Gradient Descent Provably Optimizes Over-parameterized Neural Networks S. Du Xiyu Zhai Barnabás Póczós Aarti Singh MLT ODL 221 1,272 0 04 Oct 2018
Fisher Information and Natural Gradient Learning of Random Deep Networks S. Amari Ryo Karakida Masafumi Oizumi 62 36 0 22 Aug 2018
Deep Convolutional Networks as shallow Gaussian Processes Adrià Garriga-Alonso C. Rasmussen Laurence Aitchison BDL UQCV 109 271 0 16 Aug 2018
Large-Scale Study of Curiosity-Driven Learning Yuri Burda Harrison Edwards Deepak Pathak Amos Storkey Trevor Darrell Alexei A. Efros LRM 69 704 0 13 Aug 2018
Neural Tangent Kernel: Convergence and Generalization in Neural Networks Arthur Jacot Franck Gabriel Clément Hongler 267 3,203 0 20 Jun 2018
Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables Signal Propagation in Recurrent Neural Networks Minmin Chen Jeffrey Pennington S. Schoenholz SyDa AI4CE 55 116 0 14 Jun 2018
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks Lechao Xiao Yasaman Bahri Jascha Narain Sohl-Dickstein S. Schoenholz Jeffrey Pennington 301 353 0 14 Jun 2018
Randomized Prior Functions for Deep Reinforcement Learning Ian Osband John Aslanides Albin Cassirer UQCV BDL 74 379 0 08 Jun 2018
Deep Gaussian Processes with Convolutional Kernels Vinayak Kumar Vaibhav Singh P. K. Srijith Andreas C. Damianou BDL GP 74 30 0 05 Jun 2018
Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach Ryo Karakida S. Akaho S. Amari FedML 150 145 0 04 Jun 2018
Entropy and mutual information in models of deep neural networks Marylou Gabrié Andre Manoel Clément Luneau Jean Barbier N. Macris Florent Krzakala Lenka Zdeborová 71 180 0 24 May 2018
Gaussian Process Behaviour in Wide Deep Neural Networks A. G. Matthews Mark Rowland Jiri Hron Richard Turner Zoubin Ghahramani BDL 144 559 0 30 Apr 2018
How to Start Training: The Effect of Initialization and Architecture Boris Hanin David Rolnick 68 255 0 05 Mar 2018
Mean Field Residual Networks: On the Edge of Chaos Greg Yang S. Schoenholz 64 192 0 24 Dec 2017
Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice Jeffrey Pennington S. Schoenholz Surya Ganguli ODL 43 252 0 13 Nov 2017
Deep Neural Networks as Gaussian Processes Jaehoon Lee Yasaman Bahri Roman Novak S. Schoenholz Jeffrey Pennington Jascha Narain Sohl-Dickstein UQCV BDL 128 1,093 0 01 Nov 2017
Additivity of Information in Multilayer Networks via Additive Gaussian Noise Transforms Galen Reeves 40 33 0 12 Oct 2017
Convolutional Gaussian Processes Mark van der Wilk C. Rasmussen J. Hensman BDL 71 132 0 06 Sep 2017
Exploring the Function Space of Deep-Learning Machines Yue Liu D. Saad PINN 59 25 0 04 Aug 2017
Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks John Bradshaw A. G. Matthews Zoubin Ghahramani BDL AAML 103 171 0 08 Jul 2017
Inference in Deep Networks in High Dimensions A. Fletcher S. Rangan BDL 105 69 0 20 Jun 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 701 131,652 0 12 Jun 2017
Deep Information Propagation S. Schoenholz Justin Gilmer Surya Ganguli Jascha Narain Sohl-Dickstein 82 367 0 04 Nov 2016
Stochastic Variational Deep Kernel Learning A. Wilson Zhiting Hu Ruslan Salakhutdinov Eric Xing BDL 106 267 0 01 Nov 2016
Densely Connected Convolutional Networks Gao Huang Zhuang Liu Laurens van der Maaten Kilian Q. Weinberger PINN 3DV 772 36,813 0 25 Aug 2016
Layer Normalization Jimmy Lei Ba J. Kiros Geoffrey E. Hinton 413 10,494 0 21 Jul 2016
Exponential expressivity in deep neural networks through transient chaos Ben Poole Subhaneil Lahiri M. Raghu Jascha Narain Sohl-Dickstein Surya Ganguli 90 592 0 16 Jun 2016
Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity Amit Daniely Roy Frostig Y. Singer 166 343 0 18 Feb 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 194,020 0 10 Dec 2015
Deep Kernel Learning A. Wilson Zhiting Hu Ruslan Salakhutdinov Eric Xing BDL 248 886 0 06 Nov 2015
Steps Toward Deep Kernel Methods from Infinite Neural Networks Tamir Hazan Tommi Jaakkola 80 83 0 20 Aug 2015
Deep Neural Networks with Random Gaussian Weights: A Universal Classification Strategy? Raja Giryes Guillermo Sapiro A. Bronstein 104 187 0 30 Apr 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 463 43,305 0 11 Feb 2015
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification Kaiming He Xinming Zhang Shaoqing Ren Jian Sun VLM 326 18,625 0 06 Feb 2015