Gradient Descent Provably Optimizes Over-parameterized Neural Networks

4 October 2018

Aarti Singh

Papers citing "Gradient Descent Provably Optimizes Over-parameterized Neural Networks"

50 / 468 papers shown

Title
Towards Understanding the Spectral Bias of Deep Learning Yuan Cao Zhiying Fang Yue Wu Ding-Xuan Zhou Quanquan Gu 41 215 0 03 Dec 2019
Neural Contextual Bandits with UCB-based Exploration Dongruo Zhou Lihong Li Quanquan Gu 36 15 0 11 Nov 2019
Enhanced Convolutional Neural Tangent Kernels Zhiyuan Li Ruosong Wang Dingli Yu S. Du Wei Hu Ruslan Salakhutdinov Sanjeev Arora 21 131 0 03 Nov 2019
Global Convergence of Gradient Descent for Deep Linear Residual Networks Lei Wu Qingcan Wang Chao Ma ODL AI4CE 28 22 0 02 Nov 2019
Denoising and Regularization via Exploiting the Structural Bias of Convolutional Generators Reinhard Heckel Mahdi Soltanolkotabi DiffM 37 81 0 31 Oct 2019
Capacity, Bandwidth, and Compositionality in Emergent Language Learning Cinjon Resnick Abhinav Gupta Jakob N. Foerster Andrew M. Dai Kyunghyun Cho 31 51 0 24 Oct 2019
The Local Elasticity of Neural Networks Hangfeng He Weijie J. Su 40 44 0 15 Oct 2019
Algorithm-Dependent Generalization Bounds for Overparameterized Deep Residual Networks Spencer Frei Yuan Cao Quanquan Gu ODL 9 31 0 07 Oct 2019
Harnessing the Power of Infinitely Wide Deep Nets on Small-data Tasks Sanjeev Arora S. Du Zhiyuan Li Ruslan Salakhutdinov Ruosong Wang Dingli Yu AAML 19 161 0 03 Oct 2019
Beyond Linearization: On Quadratic and Higher-Order Approximation of Wide Neural Networks Yu Bai J. Lee 24 116 0 03 Oct 2019
Overparameterized Neural Networks Implement Associative Memory Adityanarayanan Radhakrishnan M. Belkin Caroline Uhler BDL 35 71 0 26 Sep 2019
Towards Understanding the Transferability of Deep Representations Hong Liu Mingsheng Long Jianmin Wang Michael I. Jordan 30 25 0 26 Sep 2019
Asymptotics of Wide Networks from Feynman Diagrams Ethan Dyer Guy Gur-Ari 29 113 0 25 Sep 2019
Sample Efficient Policy Gradient Methods with Recursive Variance Reduction Pan Xu F. Gao Quanquan Gu 31 83 0 18 Sep 2019
Stochastic AUC Maximization with Deep Neural Networks Mingrui Liu Zhuoning Yuan Yiming Ying Tianbao Yang 17 103 0 28 Aug 2019
Adaptative Inference Cost With Convolutional Neural Mixture Models Adria Ruiz Jakob Verbeek VLM 30 22 0 19 Aug 2019
The generalization error of random features regression: Precise asymptotics and double descent curve Song Mei Andrea Montanari 60 626 0 14 Aug 2019
Hessian based analysis of SGD for Deep Nets: Dynamics and Generalization Xinyan Li Qilong Gu Yingxue Zhou Tiancong Chen A. Banerjee ODL 42 51 0 24 Jul 2019
Sparse Optimization on Measures with Over-parameterized Gradient Descent Lénaïc Chizat 21 92 0 24 Jul 2019
Benign Overfitting in Linear Regression Peter L. Bartlett Philip M. Long Gábor Lugosi Alexander Tsigler MLT 8 763 0 26 Jun 2019
Algorithmic Guarantees for Inverse Imaging with Untrained Network Priors Gauri Jagatap C. Hegde 26 70 0 20 Jun 2019
Kernel and Rich Regimes in Overparametrized Models Blake E. Woodworth Suriya Gunasekar Pedro H. P. Savarese E. Moroshko Itay Golan J. Lee Daniel Soudry Nathan Srebro 30 353 0 13 Jun 2019
Generalization Guarantees for Neural Networks via Harnessing the Low-rank Structure of the Jacobian Samet Oymak Zalan Fabian Mingchen Li Mahdi Soltanolkotabi MLT 21 88 0 12 Jun 2019
One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers Ari S. Morcos Haonan Yu Michela Paganini Yuandong Tian 16 228 0 06 Jun 2019
Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks Yuan Cao Quanquan Gu MLT AI4CE 34 383 0 30 May 2019
Graph Neural Tangent Kernel: Fusing Graph Neural Networks with Graph Kernels S. Du Kangcheng Hou Barnabás Póczós Ruslan Salakhutdinov Ruosong Wang Keyulu Xu 23 270 0 30 May 2019
Generalization bounds for deep convolutional neural networks Philip M. Long Hanie Sedghi MLT 42 89 0 29 May 2019
Norm-based generalisation bounds for multi-class convolutional neural networks Antoine Ledent Waleed Mustafa Yunwen Lei Marius Kloft 18 5 0 29 May 2019
Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for Regression Problems Tianle Cai Ruiqi Gao Jikai Hou Siyu Chen Dong Wang Di He Zhihua Zhang Liwei Wang ODL 21 57 0 28 May 2019
Enhancing Adversarial Defense by k-Winners-Take-All Chang Xiao Peilin Zhong Changxi Zheng AAML 24 97 0 25 May 2019
What Can ResNet Learn Efficiently, Going Beyond Kernels? Zeyuan Allen-Zhu Yuanzhi Li 24 183 0 24 May 2019
Gradient Descent can Learn Less Over-parameterized Two-layer Neural Networks on Classification Problems Atsushi Nitanda Geoffrey Chinot Taiji Suzuki MLT 16 33 0 23 May 2019
A type of generalization error induced by initialization in deep neural networks Yaoyu Zhang Zhi-Qin John Xu Tao Luo Zheng Ma 9 50 0 19 May 2019
Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentation Colin Wei Tengyu Ma 25 109 0 09 May 2019
Rethinking Arithmetic for Deep Neural Networks George A. Constantinides 34 4 0 07 May 2019
Linearized two-layers neural networks in high dimension Behrooz Ghorbani Song Mei Theodor Misiakiewicz Andrea Montanari MLT 18 241 0 27 Apr 2019
On Exact Computation with an Infinitely Wide Neural Net Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruslan Salakhutdinov Ruosong Wang 47 905 0 26 Apr 2019
Analysis of the Gradient Descent Algorithm for a Deep Neural Network Model with Skip-connections E. Weinan Chao Ma Qingcan Wang Lei Wu MLT 37 22 0 10 Apr 2019
Correlation Congruence for Knowledge Distillation Baoyun Peng Xiao Jin Jiaheng Liu Shunfeng Zhou Yichao Wu Yu Liu Dongsheng Li Zhaoning Zhang 63 507 0 03 Apr 2019
Convergence rates for the stochastic gradient descent method for non-convex objective functions Benjamin J. Fehrman Benjamin Gess Arnulf Jentzen 19 101 0 02 Apr 2019
On the Power and Limitations of Random Features for Understanding Neural Networks Gilad Yehudai Ohad Shamir MLT 28 181 0 01 Apr 2019
Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks Mingchen Li Mahdi Soltanolkotabi Samet Oymak NoLa 47 351 0 27 Mar 2019
Surprises in High-Dimensional Ridgeless Least Squares Interpolation Trevor Hastie Andrea Montanari Saharon Rosset R. Tibshirani 31 728 0 19 Mar 2019
Mean Field Analysis of Deep Neural Networks Justin A. Sirignano K. Spiliopoulos 22 82 0 11 Mar 2019
A Priori Estimates of the Population Risk for Residual Networks E. Weinan Chao Ma Qingcan Wang UQCV 37 61 0 06 Mar 2019
LipschitzLR: Using theoretically computed adaptive learning rates for fast convergence Rahul Yedida Snehanshu Saha Tejas Prashanth ODL 25 12 0 20 Feb 2019
Understanding over-parameterized deep networks by geometrization Xiao Dong Ling Zhou GNN AI4CE 21 7 0 11 Feb 2019
Stiffness: A New Perspective on Generalization in Neural Networks Stanislav Fort Pawel Krzysztof Nowak Stanislaw Jastrzebski S. Narayanan 24 94 0 28 Jan 2019
Dynamical Isometry and a Mean Field Theory of LSTMs and GRUs D. Gilboa B. Chang Minmin Chen Greg Yang S. Schoenholz Ed H. Chi Jeffrey Pennington 34 40 0 25 Jan 2019
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruosong Wang MLT 55 961 0 24 Jan 2019