On Exact Computation with an Infinitely Wide Neural Net

26 April 2019

Papers citing "On Exact Computation with an Infinitely Wide Neural Net"

50 / 266 papers shown

Title
How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks Keyulu Xu Mozhi Zhang Jingling Li S. Du Ken-ichi Kawarabayashi Stefanie Jegelka MLT 25 306 0 24 Sep 2020
Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don't E. Weinan Chao Ma Stephan Wojtowytsch Lei Wu AI4CE 24 133 0 22 Sep 2020
Tensor Programs III: Neural Matrix Laws Greg Yang 19 44 0 22 Sep 2020
Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHS Lin Chen Sheng Xu 32 93 0 22 Sep 2020
Generalized Leverage Score Sampling for Neural Networks Jason D. Lee Ruoqi Shen Zhao Song Mengdi Wang Zheng Yu 21 42 0 21 Sep 2020
Deep Networks and the Multiple Manifold Problem Sam Buchanan D. Gilboa John N. Wright 166 39 0 25 Aug 2020
Obtaining Adjustable Regularization for Free via Iterate Averaging Jingfeng Wu Vladimir Braverman Lin F. Yang 30 2 0 15 Aug 2020
Multiple Descent: Design Your Own Generalization Curve Lin Chen Yifei Min M. Belkin Amin Karbasi DRL 35 61 0 03 Aug 2020
When and why PINNs fail to train: A neural tangent kernel perspective Sizhuang He Xinling Yu P. Perdikaris 33 880 0 28 Jul 2020
Phase diagram for two-layer ReLU neural networks at infinite-width limit Yaoyu Zhang Zhi-Qin John Xu Zheng Ma Tao Luo 27 59 0 15 Jul 2020
Explicit Regularisation in Gaussian Noise Injections A. Camuto M. Willetts Umut Simsekli Stephen J. Roberts Chris Holmes 25 55 0 14 Jul 2020
On the Similarity between the Laplace and Neural Tangent Kernels Amnon Geifman A. Yadav Yoni Kasten Meirav Galun David Jacobs Ronen Basri 23 89 0 03 Jul 2020
Beyond Signal Propagation: Is Feature Diversity Necessary in Deep Neural Network Initialization? Yaniv Blumenfeld D. Gilboa Daniel Soudry ODL 30 13 0 02 Jul 2020
A Revision of Neural Tangent Kernel-based Approaches for Neural Networks Kyungsu Kim A. Lozano Eunho Yang AAML 40 0 0 02 Jul 2020
Associative Memory in Iterated Overparameterized Sigmoid Autoencoders Yibo Jiang Cengiz Pehlevan 19 13 0 30 Jun 2020
Tensor Programs II: Neural Tangent Kernel for Any Architecture Greg Yang 58 135 0 25 Jun 2020
Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networks Abdulkadir Canatar Blake Bordelon Cengiz Pehlevan 27 181 0 23 Jun 2020
Training (Overparametrized) Neural Networks in Near-Linear Time Jan van den Brand Binghui Peng Zhao Song Omri Weinstein ODL 29 82 0 20 Jun 2020
An analytic theory of shallow networks dynamics for hinge loss classification Franco Pellegrini Giulio Biroli 35 19 0 19 Jun 2020
Kernel Alignment Risk Estimator: Risk Prediction from Training Data Arthur Jacot Berfin cSimcsek Francesco Spadaro Clément Hongler Franck Gabriel 22 67 0 17 Jun 2020
On the training dynamics of deep networks with $L_2$ regularization Aitor Lewkowycz Guy Gur-Ari 44 53 0 15 Jun 2020
Can Temporal-Difference and Q-Learning Learn Representation? A Mean-Field Theory Yufeng Zhang Qi Cai Zhuoran Yang Yongxin Chen Zhaoran Wang OOD MLT 153 11 0 08 Jun 2020
Halting Time is Predictable for Large Models: A Universality Property and Average-case Analysis Courtney Paquette B. V. Merrienboer Elliot Paquette Fabian Pedregosa 31 25 0 08 Jun 2020
Coresets via Bilevel Optimization for Continual Learning and Streaming Zalan Borsos Mojmír Mutný Andreas Krause CLL 38 226 0 06 Jun 2020
Is deeper better? It depends on locality of relevant features Takashi Mori Masahito Ueda OOD 25 4 0 26 May 2020
Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networks Z. Fan Zhichao Wang 44 71 0 25 May 2020
Feature Purification: How Adversarial Training Performs Robust Deep Learning Zeyuan Allen-Zhu Yuanzhi Li MLT AAML 39 147 0 20 May 2020
Modularizing Deep Learning via Pairwise Learning With Kernels Shiyu Duan Shujian Yu José C. Príncipe MoMe 27 20 0 12 May 2020
Provable Training of a ReLU Gate with an Iterative Non-Gradient Algorithm Sayar Karmakar Anirbit Mukherjee 14 7 0 08 May 2020
Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyond Fanghui Liu Xiaolin Huang Yudong Chen Johan A. K. Suykens BDL 44 172 0 23 Apr 2020
Predicting the outputs of finite deep neural networks trained with noisy gradients Gadi Naveh Oded Ben-David H. Sompolinsky Zohar Ringel 19 21 0 02 Apr 2020
On Infinite-Width Hypernetworks Etai Littwin Tomer Galanti Lior Wolf Greg Yang 11 11 0 27 Mar 2020
A Mean-field Analysis of Deep ResNet and Beyond: Towards Provable Optimization Via Overparameterization From Depth Yiping Lu Chao Ma Yulong Lu Jianfeng Lu Lexing Ying MLT 39 78 0 11 Mar 2020
Frequency Bias in Neural Networks for Input of Non-Uniform Density Ronen Basri Meirav Galun Amnon Geifman David Jacobs Yoni Kasten S. Kritchman 45 183 0 10 Mar 2020
Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observations Aditya Golatkar Alessandro Achille Stefano Soatto MU OOD 24 189 0 05 Mar 2020
The large learning rate phase of deep learning: the catapult mechanism Aitor Lewkowycz Yasaman Bahri Ethan Dyer Jascha Narain Sohl-Dickstein Guy Gur-Ari ODL 159 236 0 04 Mar 2020
Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regime Stéphane dÁscoli Maria Refinetti Giulio Biroli Florent Krzakala 98 152 0 02 Mar 2020
Stable behaviour of infinitely wide deep neural networks Stefano Favaro S. Fortini Stefano Peluchetti BDL 19 28 0 01 Mar 2020
Deep Randomized Neural Networks Claudio Gallicchio Simone Scardapane OOD 45 61 0 27 Feb 2020
Convex Geometry and Duality of Over-parameterized Neural Networks Tolga Ergen Mert Pilanci MLT 42 54 0 25 Feb 2020
Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-layer Networks Mert Pilanci Tolga Ergen 29 116 0 24 Feb 2020
Over-parameterized Adversarial Training: An Analysis Overcoming the Curse of Dimensionality Yi Zhang Orestis Plevrakis S. Du Xingguo Li Zhao Song Sanjeev Arora 29 51 0 16 Feb 2020
Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networks Blake Bordelon Abdulkadir Canatar Cengiz Pehlevan 149 201 0 07 Feb 2020
On the infinite width limit of neural networks with a standard parameterization Jascha Narain Sohl-Dickstein Roman Novak S. Schoenholz Jaehoon Lee 32 47 0 21 Jan 2020
Any Target Function Exists in a Neighborhood of Any Sufficiently Wide Random Network: A Geometrical Perspective S. Amari 27 12 0 20 Jan 2020
Revisiting Landscape Analysis in Deep Neural Networks: Eliminating Decreasing Paths to Infinity Shiyu Liang Ruoyu Sun R. Srikant 37 19 0 31 Dec 2019
Deep Graph Similarity Learning: A Survey Guixiang Ma Nesreen Ahmed Theodore L. Willke Philip S. Yu GNN 21 77 0 25 Dec 2019
Optimization for deep learning: theory and algorithms Ruoyu Sun ODL 27 168 0 19 Dec 2019
Neural Tangents: Fast and Easy Infinite Neural Networks in Python Roman Novak Lechao Xiao Jiri Hron Jaehoon Lee Alexander A. Alemi Jascha Narain Sohl-Dickstein S. Schoenholz 38 225 0 05 Dec 2019
Towards Understanding the Spectral Bias of Deep Learning Yuan Cao Zhiying Fang Yue Wu Ding-Xuan Zhou Quanquan Gu 41 215 0 03 Dec 2019