Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers

12 November 2018

Papers citing "Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers"

50 / 498 papers shown

Title
Student Specialization in Deep ReLU Networks With Finite Width and Input Dimension Yuandong Tian MLT 22 8 0 30 Sep 2019
A Constructive Prediction of the Generalization Error Across Scales Jonathan S. Rosenfeld Amir Rosenfeld Yonatan Belinkov Nir Shavit 36 205 0 27 Sep 2019
Polylogarithmic width suffices for gradient descent to achieve arbitrarily small test error with shallow ReLU networks Ziwei Ji Matus Telgarsky 27 177 0 26 Sep 2019
Mildly Overparametrized Neural Nets can Memorize Training Data Efficiently Rong Ge Runzhe Wang Haoyu Zhao TDI 18 20 0 26 Sep 2019
Wider Networks Learn Better Features D. Gilboa Guy Gur-Ari 13 7 0 25 Sep 2019
Dynamics of Deep Neural Networks and Neural Tangent Hierarchy Jiaoyang Huang H. Yau 22 146 0 18 Sep 2019
Auxiliary Learning for Deep Multi-task Learning Yifan Liu Bohan Zhuang Chunhua Shen Hao Chen Wei Yin MoE 30 10 0 05 Sep 2019
Neural Policy Gradient Methods: Global Optimality and Rates of Convergence Lingxiao Wang Qi Cai Zhuoran Yang Zhaoran Wang 16 236 0 29 Aug 2019
Deep Learning Theory Review: An Optimal Control and Dynamical Systems Perspective Guan-Horng Liu Evangelos A. Theodorou AI4CE 19 71 0 28 Aug 2019
Theoretical Issues in Deep Networks: Approximation, Optimization and Generalization T. Poggio Andrzej Banburski Q. Liao ODL 31 161 0 25 Aug 2019
Effect of Activation Functions on the Training of Overparametrized Neural Nets A. Panigrahi Abhishek Shetty Navin Goyal 16 20 0 16 Aug 2019
The generalization error of random features regression: Precise asymptotics and double descent curve Song Mei Andrea Montanari 60 626 0 14 Aug 2019
Visualizing the PHATE of Neural Networks Scott A. Gigante Adam S. Charles Smita Krishnaswamy Gal Mishne 36 37 0 07 Aug 2019
Fast generalization error bound of deep learning without scale invariance of activation functions Y. Terada Ryoma Hirose MLT 11 6 0 25 Jul 2019
A Fine-Grained Spectral Perspective on Neural Networks Greg Yang Hadi Salman 30 110 0 24 Jul 2019
Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks Yuanzhi Li Colin Wei Tengyu Ma 6 291 0 10 Jul 2019
Non-Structured DNN Weight Pruning -- Is It Beneficial in Any Platform? Xiaolong Ma Sheng Lin Shaokai Ye Zhezhi He Linfeng Zhang ... Deliang Fan Xuehai Qian X. Lin Kaisheng Ma Yanzhi Wang MQ 27 92 0 03 Jul 2019
On Symmetry and Initialization for Neural Networks Ido Nachum Amir Yehudayoff MLT 28 5 0 01 Jul 2019
Empirical Study of the Benefits of Overparameterization in Learning Latent Variable Models Rares-Darius Buhai Yoni Halpern Yoon Kim Andrej Risteski David Sontag BDL 14 5 0 28 Jun 2019
Neural Proximal/Trust Region Policy Optimization Attains Globally Optimal Policy Boyi Liu Qi Cai Zhuoran Yang Zhaoran Wang 30 108 0 25 Jun 2019
ID3 Learns Juntas for Smoothed Product Distributions Alon Brutzkus Amit Daniely Eran Malach 20 20 0 20 Jun 2019
Dynamics of stochastic gradient descent for two-layer neural networks in the teacher-student setup Sebastian Goldt Madhu S. Advani Andrew M. Saxe Florent Krzakala Lenka Zdeborová MLT 27 140 0 18 Jun 2019
Approximation power of random neural networks Bolton Bailey Ziwei Ji Matus Telgarsky Ruicheng Xian 18 6 0 18 Jun 2019
Kernel and Rich Regimes in Overparametrized Models Blake E. Woodworth Suriya Gunasekar Pedro H. P. Savarese E. Moroshko Itay Golan J. Lee Daniel Soudry Nathan Srebro 30 353 0 13 Jun 2019
Generalization Guarantees for Neural Networks via Harnessing the Low-rank Structure of the Jacobian Samet Oymak Zalan Fabian Mingchen Li Mahdi Soltanolkotabi MLT 21 88 0 12 Jun 2019
Learning Curves for Deep Neural Networks: A Gaussian Field Theory Perspective Omry Cohen Orit Malka Zohar Ringel AI4CE 21 21 0 12 Jun 2019
Decoupling Gating from Linearity Jonathan Fiat Eran Malach Shai Shalev-Shwartz 17 28 0 12 Jun 2019
Semi-flat minima and saddle points by embedding neural networks to overparameterization Kenji Fukumizu Shoichiro Yamaguchi Yoh-ichi Mototake Mirai Tanaka 3DPC 19 23 0 12 Jun 2019
An Improved Analysis of Training Over-parameterized Deep Neural Networks Difan Zou Quanquan Gu 21 230 0 11 Jun 2019
The Generalization-Stability Tradeoff In Neural Network Pruning Brian Bartoldson Ari S. Morcos Adrian Barbu G. Erlebacher 24 72 0 09 Jun 2019
Quadratic Suffices for Over-parametrization via Matrix Chernoff Bound Zhao Song Xin Yang 15 91 0 09 Jun 2019
One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers Ari S. Morcos Haonan Yu Michela Paganini Yuandong Tian 16 228 0 06 Jun 2019
Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP Haonan Yu Sergey Edunov Yuandong Tian Ari S. Morcos 24 148 0 06 Jun 2019
Towards Task and Architecture-Independent Generalization Gap Predictors Scott Yak J. Gonzalvo Hanna Mazzawi UQCV AI4CE 4 27 0 04 Jun 2019
What Can Neural Networks Reason About? Keyulu Xu Jingling Li Mozhi Zhang S. Du Ken-ichi Kawarabayashi Stefanie Jegelka NAI AI4CE 27 240 0 30 May 2019
Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks Yuan Cao Quanquan Gu MLT AI4CE 17 383 0 30 May 2019
On the Generalization Gap in Reparameterizable Reinforcement Learning Huan Wang Stephan Zheng Caiming Xiong R. Socher 9 39 0 29 May 2019
Norm-based generalisation bounds for multi-class convolutional neural networks Antoine Ledent Waleed Mustafa Yunwen Lei Marius Kloft 18 5 0 29 May 2019
On the Inductive Bias of Neural Tangent Kernels A. Bietti Julien Mairal 28 252 0 29 May 2019
Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for Regression Problems Tianle Cai Ruiqi Gao Jikai Hou Siyu Chen Dong Wang Di He Zhihua Zhang Liwei Wang ODL 21 57 0 28 May 2019
Implicit Rugosity Regularization via Data Augmentation Daniel LeJeune Randall Balestriero Hamid Javadi Richard G. Baraniuk 12 4 0 28 May 2019
SGD on Neural Networks Learns Functions of Increasing Complexity Preetum Nakkiran Gal Kaplun Dimitris Kalimeris Tristan Yang Benjamin L. Edelman Fred Zhang Boaz Barak MLT 14 236 0 28 May 2019
Abstraction Mechanisms Predict Generalization in Deep Neural Networks Alex Gain H. Siegelmann AI4CE 19 6 0 27 May 2019
Quantifying the generalization error in deep learning in terms of data distribution and neural network smoothness Pengzhan Jin Lu Lu Yifa Tang George Karniadakis 11 60 0 27 May 2019
Simple and Effective Regularization Methods for Training on Noisily Labeled Data with Generalization Guarantee Wei Hu Zhiyuan Li Dingli Yu NoLa 22 12 0 27 May 2019
Temporal-difference learning with nonlinear function approximation: lazy training and mean field regimes Andrea Agazzi Jianfeng Lu 19 8 0 27 May 2019
On Learning Over-parameterized Neural Networks: A Functional Approximation Perspective Lili Su Pengkun Yang MLT 27 53 0 26 May 2019
Empirical Risk Minimization in the Interpolating Regime with Application to Neural Network Learning Nicole Mücke Ingo Steinwart AI4CE 16 2 0 25 May 2019
What Can ResNet Learn Efficiently, Going Beyond Kernels? Zeyuan Allen-Zhu Yuanzhi Li 24 183 0 24 May 2019
Neural Temporal-Difference and Q-Learning Provably Converge to Global Optima Qi Cai Zhuoran Yang Jason D. Lee Zhaoran Wang 34 29 0 24 May 2019