On the Global Convergence of Training Deep Linear ResNets

On the Global Convergence of Training Deep Linear ResNets

2 March 2020

Quanquan Gu

ArXiv (abs)PDF HTML

Papers citing "On the Global Convergence of Training Deep Linear ResNets"

13 / 13 papers shown

Title
Width Provably Matters in Optimization for Deep Linear Neural Networks S. Du Wei Hu 79 95 0 24 Jan 2019
On Lazy Training in Differentiable Programming Lénaïc Chizat Edouard Oyallon Francis R. Bach 111 840 0 19 Dec 2018
Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks Difan Zou Yuan Cao Dongruo Zhou Quanquan Gu ODL 215 448 0 21 Nov 2018
Gradient Descent Provably Optimizes Over-parameterized Neural Networks S. Du Xiyu Zhai Barnabás Póczós Aarti Singh MLT ODL 245 1,276 0 04 Oct 2018
Exponential Convergence Time of Gradient Descent for One-Dimensional Deep Linear Neural Networks Ohad Shamir 89 47 0 23 Sep 2018
Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data Yuanzhi Li Yingyu Liang MLT 222 653 0 03 Aug 2018
Implicit Bias of Gradient Descent on Linear Convolutional Networks Suriya Gunasekar Jason D. Lee Daniel Soudry Nathan Srebro MDE 130 414 0 01 Jun 2018
On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization Sanjeev Arora Nadav Cohen Elad Hazan 120 487 0 19 Feb 2018
SGD Learns Over-parameterized Networks that Provably Generalize on Linearly Separable Data Alon Brutzkus Amir Globerson Eran Malach Shai Shalev-Shwartz MLT 156 279 0 27 Oct 2017
Recovery Guarantees for One-hidden-layer Neural Networks Kai Zhong Zhao Song Prateek Jain Peter L. Bartlett Inderjit S. Dhillon MLT 183 337 0 10 Jun 2017
Depth Creates No Bad Local Minima Haihao Lu Kenji Kawaguchi ODL FAtt 78 121 0 27 Feb 2017
Identity Matters in Deep Learning Moritz Hardt Tengyu Ma OOD 99 399 0 14 Nov 2016
Topology and Geometry of Half-Rectified Network Optimization C. Freeman Joan Bruna 233 235 0 04 Nov 2016