Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers

12 November 2018

Papers citing "Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers"

48 / 498 papers shown

Title
Gradient Descent can Learn Less Over-parameterized Two-layer Neural Networks on Classification Problems Atsushi Nitanda Geoffrey Chinot Taiji Suzuki MLT 16 33 0 23 May 2019
Consensus-based Interpretable Deep Neural Networks with Application to Mortality Prediction Shaeke Salman S. N. Payrovnaziri Xiuwen Liu Pablo Rengifo-Moreno Zhe He 19 0 0 14 May 2019
Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentation Colin Wei Tengyu Ma 25 109 0 09 May 2019
Accelerated Target Updates for Q-learning Bowen Weng Huaqing Xiong Wei Zhang 6 0 0 07 May 2019
On Exact Computation with an Infinitely Wide Neural Net Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruslan Salakhutdinov Ruosong Wang 44 905 0 26 Apr 2019
The Impact of Neural Network Overparameterization on Gradient Confusion and Stochastic Gradient Descent Karthik A. Sankararaman Soham De Zheng Xu Yifan Jiang Tom Goldstein ODL 24 103 0 15 Apr 2019
Analysis of the Gradient Descent Algorithm for a Deep Neural Network Model with Skip-connections E. Weinan Chao Ma Qingcan Wang Lei Wu MLT 35 22 0 10 Apr 2019
A Comparative Analysis of the Optimization and Generalization Property of Two-layer Neural Network and Random Feature Models Under Gradient Descent Dynamics E. Weinan Chao Ma Lei Wu MLT 19 121 0 08 Apr 2019
On the Power and Limitations of Random Features for Understanding Neural Networks Gilad Yehudai Ohad Shamir MLT 26 181 0 01 Apr 2019
Adversarial Robustness vs Model Compression, or Both? Shaokai Ye Kaidi Xu Sijia Liu Jan-Henrik Lambrechts Huan Zhang Aojun Zhou Kaisheng Ma Yanzhi Wang Xue Lin AAML 17 163 0 29 Mar 2019
Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks Mingchen Li Mahdi Soltanolkotabi Samet Oymak NoLa 47 351 0 27 Mar 2019
Harmless interpolation of noisy data in regression Vidya Muthukumar Kailas Vodrahalli Vignesh Subramanian A. Sahai 23 203 0 21 Mar 2019
Stabilize Deep ResNet with A Sharp Scaling Factor $τ$ Huishuai Zhang Da Yu Mingyang Yi Wei Chen Tie-Yan Liu 32 8 0 17 Mar 2019
Theory III: Dynamics and Generalization in Deep Networks Andrzej Banburski Q. Liao Brando Miranda Lorenzo Rosasco Fernanda De La Torre Jack Hidary T. Poggio AI4CE 32 3 0 12 Mar 2019
A Priori Estimates of the Population Risk for Residual Networks E. Weinan Chao Ma Qingcan Wang UQCV 37 61 0 06 Mar 2019
Implicit Regularization in Over-parameterized Neural Networks M. Kubo Ryotaro Banno Hidetaka Manabe Masataka Minoji 19 23 0 05 Mar 2019
Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation Greg Yang 11 282 0 13 Feb 2019
Uniform convergence may be unable to explain generalization in deep learning Vaishnavh Nagarajan J. Zico Kolter MoMe AI4CE 17 309 0 13 Feb 2019
Identity Crisis: Memorization and Generalization under Extreme Overparameterization Chiyuan Zhang Samy Bengio Moritz Hardt Michael C. Mozer Y. Singer 14 88 0 13 Feb 2019
Towards moderate overparameterization: global convergence guarantees for training shallow neural networks Samet Oymak Mahdi Soltanolkotabi 19 319 0 12 Feb 2019
Mean Field Limit of the Learning Dynamics of Multilayer Neural Networks Phan-Minh Nguyen AI4CE 24 72 0 07 Feb 2019
Generalization Error Bounds of Gradient Descent for Learning Over-parameterized Deep ReLU Networks Yuan Cao Quanquan Gu ODL MLT AI4CE 25 155 0 04 Feb 2019
Can SGD Learn Recurrent Neural Networks with Provable Generalization? Zeyuan Allen-Zhu Yuanzhi Li MLT LRM 11 57 0 04 Feb 2019
On Generalization Error Bounds of Noisy Gradient Methods for Non-Convex Learning Jian Li Xuanyuan Luo Mingda Qiao 21 85 0 02 Feb 2019
Compressing GANs using Knowledge Distillation Angeline Aguinaldo Ping Yeh-Chiang Alex Gain Ameya D. Patil Kolten Pearson S. Feizi GAN 11 83 0 01 Feb 2019
Depth creates no more spurious local minima Li Zhang 10 19 0 28 Jan 2019
Generalisation dynamics of online learning in over-parameterised neural networks Sebastian Goldt Madhu S. Advani Andrew M. Saxe Florent Krzakala Lenka Zdeborová 33 14 0 25 Jan 2019
Orthogonal Statistical Learning Dylan J. Foster Vasilis Syrgkanis 16 167 0 25 Jan 2019
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruosong Wang MLT 55 961 0 24 Jan 2019
On Connected Sublevel Sets in Deep Learning Quynh N. Nguyen 19 102 0 22 Jan 2019
Overfitting Mechanism and Avoidance in Deep Neural Networks Shaeke Salman Xiuwen Liu 9 138 0 19 Jan 2019
A Theoretical Analysis of Deep Q-Learning Jianqing Fan Zhuoran Yang Yuchen Xie Zhaoran Wang 23 595 0 01 Jan 2019
Greedy Layerwise Learning Can Scale to ImageNet Eugene Belilovsky Michael Eickenberg Edouard Oyallon 9 180 0 29 Dec 2018
Overparameterized Nonlinear Learning: Gradient Descent Takes the Shortest Path? Samet Oymak Mahdi Soltanolkotabi ODL 6 176 0 25 Dec 2018
On Lazy Training in Differentiable Programming Lénaïc Chizat Edouard Oyallon Francis R. Bach 46 806 0 19 Dec 2018
ExpandNets: Linear Over-parameterization to Train Compact Convolutional Networks Shuxuan Guo J. Álvarez Mathieu Salzmann 21 77 0 26 Nov 2018
On a Sparse Shortcut Topology of Artificial Neural Networks Fenglei Fan Dayang Wang Hengtao Guo Qikui Zhu Pingkun Yan Ge Wang Hengyong Yu 38 22 0 22 Nov 2018
Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks Difan Zou Yuan Cao Dongruo Zhou Quanquan Gu ODL 33 446 0 21 Nov 2018
A Convergence Theory for Deep Learning via Over-Parameterization Zeyuan Allen-Zhu Yuanzhi Li Zhao Song AI4CE ODL 27 1,447 0 09 Nov 2018
Gradient Descent Finds Global Minima of Deep Neural Networks S. Du J. Lee Haochuan Li Liwei Wang Masayoshi Tomizuka ODL 44 1,122 0 09 Nov 2018
ReLeQ: A Reinforcement Learning Approach for Deep Quantization of Neural Networks Ahmed T. Elthakeb Prannoy Pilligundla Fatemehsadat Mireshghallah Amir Yazdanbakhsh H. Esmaeilzadeh MQ 55 68 0 05 Nov 2018
On the Convergence Rate of Training Recurrent Neural Networks Zeyuan Allen-Zhu Yuanzhi Li Zhao Song 23 191 0 29 Oct 2018
Learning Two-layer Neural Networks with Symmetric Inputs Rong Ge Rohith Kuditipudi Zhize Li Xiang Wang OOD MLT 36 57 0 16 Oct 2018
A Priori Estimates of the Population Risk for Two-layer Neural Networks Weinan E Chao Ma Lei Wu 29 130 0 15 Oct 2018
Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel Colin Wei J. Lee Qiang Liu Tengyu Ma 23 245 0 12 Oct 2018
Learning One-hidden-layer ReLU Networks via Gradient Descent Xiao Zhang Yaodong Yu Lingxiao Wang Quanquan Gu MLT 28 134 0 20 Jun 2018
On Tighter Generalization Bound for Deep Neural Networks: CNNs, ResNets, and Beyond Xingguo Li Junwei Lu Zhaoran Wang Jarvis Haupt T. Zhao 27 78 0 13 Jun 2018
Size-Independent Sample Complexity of Neural Networks Noah Golowich Alexander Rakhlin Ohad Shamir 23 544 0 18 Dec 2017