Learning One-hidden-layer ReLU Networks via Gradient Descent

20 June 2018

Quanquan Gu

Papers citing "Learning One-hidden-layer ReLU Networks via Gradient Descent"

50 / 56 papers shown

Title
Nonparametric Learning of Two-Layer ReLU Residual Units Zhunxuan Wang Linyun He Chunchuan Lyu Shay B. Cohen MLT OffRL 77 1 0 17 Aug 2020
Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks Difan Zou Yuan Cao Dongruo Zhou Quanquan Gu ODL 96 448 0 21 Nov 2018
Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers Zeyuan Allen-Zhu Yuanzhi Li Yingyu Liang MLT 94 769 0 12 Nov 2018
A Convergence Theory for Deep Learning via Over-Parameterization Zeyuan Allen-Zhu Yuanzhi Li Zhao Song AI4CE ODL 141 1,457 0 09 Nov 2018
Gradient Descent Finds Global Minima of Deep Neural Networks S. Du Jason D. Lee Haochuan Li Liwei Wang Masayoshi Tomizuka ODL 98 1,133 0 09 Nov 2018
Learning Two-layer Neural Networks with Symmetric Inputs Rong Ge Rohith Kuditipudi Zhize Li Xiang Wang OOD MLT 82 57 0 16 Oct 2018
Learning One-hidden-layer Neural Networks under General Input Distributions Weihao Gao Ashok Vardhan Makkuva Sewoong Oh Pramod Viswanath MLT 47 28 0 09 Oct 2018
Gradient Descent Provably Optimizes Over-parameterized Neural Networks S. Du Xiyu Zhai Barnabás Póczós Aarti Singh MLT ODL 114 1,261 0 04 Oct 2018
Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data Yuanzhi Li Yingyu Liang MLT 96 652 0 03 Aug 2018
On the Power of Over-parametrization in Neural Networks with Quadratic Activation S. Du Jason D. Lee 93 268 0 03 Mar 2018
Optimal approximation of continuous functions by very deep ReLU networks Dmitry Yarotsky 87 293 0 10 Feb 2018
Spurious Local Minima are Common in Two-Layer ReLU Neural Networks Itay Safran Ohad Shamir 101 263 0 24 Dec 2017
Gradient Descent Learns One-hidden-layer CNN: Don't be Afraid of Spurious Local Minima S. Du Jason D. Lee Yuandong Tian Barnabás Póczós Aarti Singh MLT 117 236 0 03 Dec 2017
Lower bounds over Boolean inputs for deep neural networks with ReLU gates Anirbit Mukherjee A. Basu 36 21 0 08 Nov 2017
Learning One-hidden-layer Neural Networks with Landscape Design Rong Ge Jason D. Lee Tengyu Ma MLT 86 260 0 01 Nov 2017
Approximating Continuous Functions by ReLU Nets of Minimal Width Boris Hanin Mark Sellke 72 232 0 31 Oct 2017
When is a Convolutional Filter Easy To Learn? S. Du Jason D. Lee Yuandong Tian MLT 43 130 0 18 Sep 2017
The Expressive Power of Neural Networks: A View from the Width Zhou Lu Hongming Pu Feicheng Wang Zhiqiang Hu Liwei Wang 56 886 0 08 Sep 2017
Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activations Boris Hanin 34 354 0 09 Aug 2017
Theoretical insights into the optimization landscape of over-parameterized shallow neural networks Mahdi Soltanolkotabi Adel Javanmard Jason D. Lee 82 417 0 16 Jul 2017
Global optimality conditions for deep neural networks Chulhee Yun S. Sra Ali Jadbabaie 134 118 0 08 Jul 2017
Recovery Guarantees for One-hidden-layer Neural Networks Kai Zhong Zhao Song Prateek Jain Peter L. Bartlett Inderjit S. Dhillon MLT 84 336 0 10 Jun 2017
Weight Sharing is Crucial to Succesful Optimization Shai Shalev-Shwartz Ohad Shamir Shaked Shammah 59 12 0 02 Jun 2017
Convergence Analysis of Two-layer Neural Networks with ReLU Activation Yuanzhi Li Yang Yuan MLT 55 649 0 28 May 2017
Learning ReLUs via Gradient Descent Mahdi Soltanolkotabi MLT 56 181 0 10 May 2017
The loss surface of deep and wide neural networks Quynh N. Nguyen Matthias Hein ODL 72 284 0 26 Apr 2017
Failures of Gradient-Based Deep Learning Shai Shalev-Shwartz Ohad Shamir Shaked Shammah ODL UQCV 61 200 0 23 Mar 2017
How to Escape Saddle Points Efficiently Chi Jin Rong Ge Praneeth Netrapalli Sham Kakade Michael I. Jordan ODL 118 834 0 02 Mar 2017
An Analytical Formula of Population Gradient for two-layered ReLU network and its Applications in Convergence and Critical Point Analysis Yuandong Tian MLT 72 216 0 02 Mar 2017
Globally Optimal Gradient Descent for a ConvNet with Gaussian Inputs Alon Brutzkus Amir Globerson MLT 63 313 0 26 Feb 2017
Provable learning of Noisy-or Networks Sanjeev Arora Rong Ge Tengyu Ma Andrej Risteski 44 26 0 28 Dec 2016
Reliably Learning the ReLU in Polynomial Time Surbhi Goel Varun Kanade Adam R. Klivans J. Thaler 53 126 0 30 Nov 2016
Identity Matters in Deep Learning Moritz Hardt Tengyu Ma OOD 43 398 0 14 Nov 2016
Understanding deep learning requires rethinking generalization Chiyuan Zhang Samy Bengio Moritz Hardt Benjamin Recht Oriol Vinyals HAI 226 4,612 0 10 Nov 2016
Diverse Neural Network Learns True Target Functions Bo Xie Yingyu Liang Le Song 96 137 0 09 Nov 2016
Error bounds for approximations with deep ReLU networks Dmitry Yarotsky 102 1,226 0 03 Oct 2016
Distribution-Specific Hardness of Learning Neural Networks Ohad Shamir 40 116 0 05 Sep 2016
The Landscape of Empirical Risk for Non-convex Losses Song Mei Yu Bai Andrea Montanari 26 312 0 22 Jul 2016
Exponential expressivity in deep neural networks through transient chaos Ben Poole Subhaneil Lahiri M. Raghu Jascha Narain Sohl-Dickstein Surya Ganguli 71 587 0 16 Jun 2016
On the Expressive Power of Deep Neural Networks M. Raghu Ben Poole Jon M. Kleinberg Surya Ganguli Jascha Narain Sohl-Dickstein 42 780 0 16 Jun 2016
No bad local minima: Data independent training error guarantees for multilayer neural networks Daniel Soudry Y. Carmon 88 235 0 26 May 2016
Deep Learning without Poor Local Minima Kenji Kawaguchi ODL 99 919 0 23 May 2016
Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity Amit Daniely Roy Frostig Y. Singer 60 343 0 18 Feb 2016
Benefits of depth in neural networks Matus Telgarsky 266 605 0 14 Feb 2016
Learning Halfspaces and Neural Networks with Random Initialization Yuchen Zhang Jason D. Lee Martin J. Wainwright Michael I. Jordan 34 35 0 25 Nov 2015
Expressiveness of Rectifier Networks Xingyuan Pan Vivek Srikumar OffRL 39 46 0 18 Nov 2015
On the Quality of the Initial Basin in Overspecified Neural Networks Itay Safran Ohad Shamir 39 127 0 13 Nov 2015
On the Expressive Power of Deep Learning: A Tensor Analysis Nadav Cohen Or Sharir Amnon Shashua 52 469 0 16 Sep 2015
Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decomposition Rong Ge Furong Huang Chi Jin Yang Yuan 105 1,056 0 06 Mar 2015
Provable Methods for Training Neural Networks with Sparse Connectivity Hanie Sedghi Anima Anandkumar 38 64 0 08 Dec 2014