v1v2v3 (latest)

Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks

21 November 2018

Quanquan Gu

Papers citing "Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks"

45 / 45 papers shown

Title
Towards Understanding the Optimization Mechanisms in Deep Learning Binchuan Qi Wei Gong Li Li 95 0 0 29 Mar 2025
Extended convexity and smoothness and their applications in deep learning Binchuan Qi Wei Gong Li Li 105 0 0 08 Oct 2024
Measuring and Controlling Solution Degeneracy across Task-Trained Recurrent Neural Networks Ann Huang Satpreet H. Singh Flavio Martinelli Kanaka Rajan 88 0 0 04 Oct 2024
Loss Gradient Gaussian Width based Generalization and Optimization Guarantees A. Banerjee Qiaobo Li Yingxue Zhou 149 0 0 11 Jun 2024
Fundamental Limits of Deep Learning-Based Binary Classifiers Trained with Hinge Loss T. Getu Georges Kaddoum M. Bennis 82 1 0 13 Sep 2023
Algorithms and SQ Lower Bounds for PAC Learning One-Hidden-Layer ReLU Networks Ilias Diakonikolas D. Kane Vasilis Kontonis Nikos Zarifis 74 66 0 22 Jun 2020
Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers Zeyuan Allen-Zhu Yuanzhi Li Yingyu Liang MLT 207 775 0 12 Nov 2018
A Convergence Theory for Deep Learning via Over-Parameterization Zeyuan Allen-Zhu Yuanzhi Li Zhao Song AI4CE ODL 281 1,469 0 09 Nov 2018
Gradient Descent Finds Global Minima of Deep Neural Networks S. Du Jason D. Lee Haochuan Li Liwei Wang Masayoshi Tomizuka ODL 261 1,136 0 09 Nov 2018
On the Convergence Rate of Training Recurrent Neural Networks Zeyuan Allen-Zhu Yuanzhi Li Zhao Song 207 192 0 29 Oct 2018
A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks Sanjeev Arora Nadav Cohen Noah Golowich Wei Hu 141 293 0 04 Oct 2018
Gradient Descent Provably Optimizes Over-parameterized Neural Networks S. Du Xiyu Zhai Barnabás Póczós Aarti Singh MLT ODL 245 1,276 0 04 Oct 2018
Gradient descent aligns the layers of deep linear networks Ziwei Ji Matus Telgarsky 123 257 0 04 Oct 2018
Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data Yuanzhi Li Yingyu Liang MLT 222 653 0 03 Aug 2018
Learning One-hidden-layer ReLU Networks via Gradient Descent Xiao Zhang Yaodong Yu Lingxiao Wang Quanquan Gu MLT 129 135 0 20 Jun 2018
Stochastic Gradient Descent on Separable Data: Exact Convergence with a Fixed Learning Rate Mor Shpigel Nacson Nathan Srebro Daniel Soudry FedML MLT 90 102 0 05 Jun 2018
On the Power of Over-parametrization in Neural Networks with Quadratic Activation S. Du Jason D. Lee 186 272 0 03 Mar 2018
On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization Sanjeev Arora Nadav Cohen Elad Hazan 122 487 0 19 Feb 2018
Gradient descent with identity initialization efficiently learns positive definite linear transformations by deep residual networks Peter L. Bartlett D. Helmbold Philip M. Long 126 117 0 16 Feb 2018
Stronger generalization bounds for deep nets via a compression approach Sanjeev Arora Rong Ge Behnam Neyshabur Yi Zhang MLT AI4CE 102 643 0 14 Feb 2018
Optimal approximation of continuous functions by very deep ReLU networks Dmitry Yarotsky 209 294 0 10 Feb 2018
Spurious Local Minima are Common in Two-Layer ReLU Neural Networks Itay Safran Ohad Shamir 184 265 0 24 Dec 2017
Size-Independent Sample Complexity of Neural Networks Noah Golowich Alexander Rakhlin Ohad Shamir 161 551 0 18 Dec 2017
Gradient Descent Learns One-hidden-layer CNN: Don't be Afraid of Spurious Local Minima S. Du Jason D. Lee Yuandong Tian Barnabás Póczós Aarti Singh MLT 157 236 0 03 Dec 2017
Learning Non-overlapping Convolutional Neural Networks with Multiple Kernels Kai Zhong Zhao Song Inderjit S. Dhillon 77 75 0 08 Nov 2017
Approximating Continuous Functions by ReLU Nets of Minimal Width Boris Hanin Mark Sellke 107 239 0 31 Oct 2017
The Implicit Bias of Gradient Descent on Separable Data Daniel Soudry Elad Hoffer Mor Shpigel Nacson Suriya Gunasekar Nathan Srebro 180 924 0 27 Oct 2017
SGD Learns Over-parameterized Networks that Provably Generalize on Linearly Separable Data Alon Brutzkus Amir Globerson Eran Malach Shai Shalev-Shwartz MLT 156 279 0 27 Oct 2017
When is a Convolutional Filter Easy To Learn? S. Du Jason D. Lee Yuandong Tian MLT 60 130 0 18 Sep 2017
The Expressive Power of Neural Networks: A View from the Width Zhou Lu Hongming Pu Feicheng Wang Zhiqiang Hu Liwei Wang 102 899 0 08 Sep 2017
Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activations Boris Hanin 73 356 0 09 Aug 2017
A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks Behnam Neyshabur Srinadh Bhojanapalli Nathan Srebro 92 610 0 29 Jul 2017
Exploring Generalization in Deep Learning Behnam Neyshabur Srinadh Bhojanapalli David A. McAllester Nathan Srebro FAtt 165 1,259 0 27 Jun 2017
Spectrally-normalized margin bounds for neural networks Peter L. Bartlett Dylan J. Foster Matus Telgarsky ODL 220 1,225 0 26 Jun 2017
Convergence Analysis of Two-layer Neural Networks with ReLU Activation Yuanzhi Li Yang Yuan MLT 166 654 0 28 May 2017
An Analytical Formula of Population Gradient for two-layered ReLU network and its Applications in Convergence and Critical Point Analysis Yuandong Tian MLT 218 217 0 02 Mar 2017
Globally Optimal Gradient Descent for a ConvNet with Gaussian Inputs Alon Brutzkus Amir Globerson MLT 176 313 0 26 Feb 2017
Identity Matters in Deep Learning Moritz Hardt Tengyu Ma OOD 99 399 0 14 Nov 2016
Why Deep Neural Networks for Function Approximation? Shiyu Liang R. Srikant 143 385 0 13 Oct 2016
Error bounds for approximations with deep ReLU networks Dmitry Yarotsky 207 1,236 0 03 Oct 2016
Deep Learning without Poor Local Minima Kenji Kawaguchi ODL 232 927 0 23 May 2016
Benefits of depth in neural networks Matus Telgarsky 385 609 0 14 Feb 2016
Representation Benefits of Deep Feedforward Networks Matus Telgarsky 88 242 0 27 Sep 2015
Norm-Based Capacity Control in Neural Networks Behnam Neyshabur Ryota Tomioka Nathan Srebro 292 591 0 27 Feb 2015
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 265 1,202 0 30 Nov 2014