Mean Field Limit of the Learning Dynamics of Multilayer Neural Networks

7 February 2019

Papers citing "Mean Field Limit of the Learning Dynamics of Multilayer Neural Networks"

48 / 48 papers shown

Title
Mean-Field Analysis for Learning Subspace-Sparse Polynomials with Gaussian Input Ziang Chen Rong Ge MLT 115 1 0 10 Jan 2025
A Mean Field Theory of Batch Normalization Greg Yang Jeffrey Pennington Vinay Rao Jascha Narain Sohl-Dickstein S. Schoenholz 72 179 0 21 Feb 2019
Scaling description of generalization with number of parameters in deep learning Mario Geiger Arthur Jacot S. Spigler Franck Gabriel Levent Sagun Stéphane dÁscoli Giulio Biroli Clément Hongler Matthieu Wyart 83 196 0 06 Jan 2019
Analysis of a Two-Layer Neural Network via Displacement Convexity Adel Javanmard Marco Mondelli Andrea Montanari MLT 87 57 0 05 Jan 2019
On Lazy Training in Differentiable Programming Lénaïc Chizat Edouard Oyallon Francis R. Bach 111 839 0 19 Dec 2018
Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks Difan Zou Yuan Cao Dongruo Zhou Quanquan Gu ODL 190 448 0 21 Nov 2018
Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers Zeyuan Allen-Zhu Yuanzhi Li Yingyu Liang MLT 201 773 0 12 Nov 2018
A Convergence Theory for Deep Learning via Over-Parameterization Zeyuan Allen-Zhu Yuanzhi Li Zhao Song AI4CE ODL 264 1,466 0 09 Nov 2018
Gradient Descent Finds Global Minima of Deep Neural Networks S. Du Jason D. Lee Haochuan Li Liwei Wang Masayoshi Tomizuka ODL 209 1,135 0 09 Nov 2018
A jamming transition from under- to over-parametrization affects loss landscape and generalization S. Spigler Mario Geiger Stéphane dÁscoli Levent Sagun Giulio Biroli Matthieu Wyart 64 150 0 22 Oct 2018
Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel Colin Wei Jason D. Lee Qiang Liu Tengyu Ma 214 244 0 12 Oct 2018
Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes Roman Novak Lechao Xiao Jaehoon Lee Yasaman Bahri Greg Yang Jiri Hron Daniel A. Abolafia Jeffrey Pennington Jascha Narain Sohl-Dickstein UQCV BDL 68 310 0 11 Oct 2018
Gradient Descent Provably Optimizes Over-parameterized Neural Networks S. Du Xiyu Zhai Barnabás Póczós Aarti Singh MLT ODL 227 1,275 0 04 Oct 2018
The jamming transition as a paradigm to understand the loss landscape of deep neural networks Mario Geiger S. Spigler Stéphane dÁscoli Levent Sagun Marco Baity-Jesi Giulio Biroli Matthieu Wyart 51 143 0 25 Sep 2018
Mean Field Analysis of Neural Networks: A Central Limit Theorem Justin A. Sirignano K. Spiliopoulos MLT 75 194 0 28 Aug 2018
Deep Convolutional Networks as shallow Gaussian Processes Adrià Garriga-Alonso C. Rasmussen Laurence Aitchison BDL UQCV 109 271 0 16 Aug 2018
Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data Yuanzhi Li Yingyu Liang MLT 219 652 0 03 Aug 2018
Neural Tangent Kernel: Convergence and Generalization in Neural Networks Arthur Jacot Franck Gabriel Clément Hongler 271 3,213 0 20 Jun 2018
Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables Signal Propagation in Recurrent Neural Networks Minmin Chen Jeffrey Pennington S. Schoenholz SyDa AI4CE 57 116 0 14 Jun 2018
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks Lechao Xiao Yasaman Bahri Jascha Narain Sohl-Dickstein S. Schoenholz Jeffrey Pennington 304 354 0 14 Jun 2018
On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport Lénaïc Chizat Francis R. Bach OT 212 736 0 24 May 2018
Gaussian Process Behaviour in Wide Deep Neural Networks A. G. Matthews Mark Rowland Jiri Hron Richard Turner Zoubin Ghahramani BDL 152 559 0 30 Apr 2018
The loss landscape of overparameterized neural networks Y. Cooper 49 77 0 26 Apr 2018
A Mean Field View of the Landscape of Two-Layers Neural Networks Song Mei Andrea Montanari Phan-Minh Nguyen MLT 101 861 0 18 Apr 2018
How to Start Training: The Effect of Initialization and Architecture Boris Hanin David Rolnick 73 255 0 05 Mar 2018
On the Power of Over-parametrization in Neural Networks with Quadratic Activation S. Du Jason D. Lee 183 272 0 03 Mar 2018
Spurious Valleys in Two-layer Neural Network Optimization Landscapes Luca Venturi Afonso S. Bandeira Joan Bruna 60 74 0 18 Feb 2018
Small nonlinearities in activation functions create bad local minima in neural networks Chulhee Yun S. Sra Ali Jadbabaie ODL 74 95 0 10 Feb 2018
Which Neural Net Architectures Give Rise To Exploding and Vanishing Gradients? Boris Hanin 65 254 0 11 Jan 2018
Mean Field Residual Networks: On the Edge of Chaos Greg Yang S. Schoenholz 67 194 0 24 Dec 2017
Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice Jeffrey Pennington S. Schoenholz Surya Ganguli ODL 43 253 0 13 Nov 2017
Deep Neural Networks as Gaussian Processes Jaehoon Lee Yasaman Bahri Roman Novak S. Schoenholz Jeffrey Pennington Jascha Narain Sohl-Dickstein UQCV BDL 131 1,097 0 01 Nov 2017
Theoretical insights into the optimization landscape of over-parameterized shallow neural networks Mahdi Soltanolkotabi Adel Javanmard Jason D. Lee 175 422 0 16 Jul 2017
Recovery Guarantees for One-hidden-layer Neural Networks Kai Zhong Zhao Song Prateek Jain Peter L. Bartlett Inderjit S. Dhillon MLT 175 337 0 10 Jun 2017
The loss surface of deep and wide neural networks Quynh N. Nguyen Matthias Hein ODL 164 285 0 26 Apr 2017
Opening the Black Box of Deep Neural Networks via Information Ravid Shwartz-Ziv Naftali Tishby AI4CE 105 1,416 0 02 Mar 2017
Topology and Geometry of Half-Rectified Network Optimization C. Freeman Joan Bruna 218 235 0 04 Nov 2016
Deep vs. shallow networks : An approximation theory perspective H. Mhaskar T. Poggio 165 341 0 10 Aug 2016
The Landscape of Empirical Risk for Non-convex Losses Song Mei Yu Bai Andrea Montanari 117 313 0 22 Jul 2016
Exponential expressivity in deep neural networks through transient chaos Ben Poole Subhaneil Lahiri M. Raghu Jascha Narain Sohl-Dickstein Surya Ganguli 90 592 0 16 Jun 2016
No bad local minima: Data independent training error guarantees for multilayer neural networks Daniel Soudry Y. Carmon 188 235 0 26 May 2016
Understanding Deep Convolutional Networks S. Mallat FAtt AI4CE 175 641 0 19 Jan 2016
On the Quality of the Initial Basin in Overspecified Neural Networks Itay Safran Ohad Shamir 78 127 0 13 Nov 2015
Steps Toward Deep Kernel Methods from Infinite Neural Networks Tamir Hazan Tommi Jaakkola 80 83 0 20 Aug 2015
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 261 1,200 0 30 Nov 2014
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 1.7K 100,479 0 04 Sep 2014
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 1.7K 39,590 0 01 Sep 2014
Provable Bounds for Learning Some Deep Representations Sanjeev Arora Aditya Bhaskara Rong Ge Tengyu Ma BDL 97 335 0 23 Oct 2013