Learning One-hidden-layer ReLU Networks via Gradient Descent

20 June 2018

Quanquan Gu

Papers citing "Learning One-hidden-layer ReLU Networks via Gradient Descent"

50 / 56 papers shown

Title
Nonparametric Learning of Two-Layer ReLU Residual Units Zhunxuan Wang Linyun He Chunchuan Lyu Shay B. Cohen MLT OffRL 55 1 0 17 Aug 2020
Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks Difan Zou Yuan Cao Dongruo Zhou Quanquan Gu ODL 66 448 0 21 Nov 2018
Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers Zeyuan Allen-Zhu Yuanzhi Li Yingyu Liang MLT 72 769 0 12 Nov 2018
A Convergence Theory for Deep Learning via Over-Parameterization Zeyuan Allen-Zhu Yuanzhi Li Zhao Song AI4CE ODL 101 1,457 0 09 Nov 2018
Gradient Descent Finds Global Minima of Deep Neural Networks S. Du Jason D. Lee Haochuan Li Liwei Wang Masayoshi Tomizuka ODL 76 1,133 0 09 Nov 2018
Learning Two-layer Neural Networks with Symmetric Inputs Rong Ge Rohith Kuditipudi Zhize Li Xiang Wang OOD MLT 58 57 0 16 Oct 2018
Learning One-hidden-layer Neural Networks under General Input Distributions Weihao Gao Ashok Vardhan Makkuva Sewoong Oh Pramod Viswanath MLT 45 28 0 09 Oct 2018
Gradient Descent Provably Optimizes Over-parameterized Neural Networks S. Du Xiyu Zhai Barnabás Póczós Aarti Singh MLT ODL 92 1,261 0 04 Oct 2018
Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data Yuanzhi Li Yingyu Liang MLT 66 652 0 03 Aug 2018
On the Power of Over-parametrization in Neural Networks with Quadratic Activation S. Du Jason D. Lee 86 268 0 03 Mar 2018
Optimal approximation of continuous functions by very deep ReLU networks Dmitry Yarotsky 46 293 0 10 Feb 2018
Spurious Local Minima are Common in Two-Layer ReLU Neural Networks Itay Safran Ohad Shamir 88 263 0 24 Dec 2017
Gradient Descent Learns One-hidden-layer CNN: Don't be Afraid of Spurious Local Minima S. Du Jason D. Lee Yuandong Tian Barnabás Póczós Aarti Singh MLT 107 236 0 03 Dec 2017
Lower bounds over Boolean inputs for deep neural networks with ReLU gates Anirbit Mukherjee A. Basu 36 21 0 08 Nov 2017
Learning One-hidden-layer Neural Networks with Landscape Design Rong Ge Jason D. Lee Tengyu Ma MLT 53 260 0 01 Nov 2017
Approximating Continuous Functions by ReLU Nets of Minimal Width Boris Hanin Mark Sellke 64 232 0 31 Oct 2017
When is a Convolutional Filter Easy To Learn? S. Du Jason D. Lee Yuandong Tian MLT 34 130 0 18 Sep 2017
The Expressive Power of Neural Networks: A View from the Width Zhou Lu Hongming Pu Feicheng Wang Zhiqiang Hu Liwei Wang 46 886 0 08 Sep 2017
Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activations Boris Hanin 24 354 0 09 Aug 2017
Theoretical insights into the optimization landscape of over-parameterized shallow neural networks Mahdi Soltanolkotabi Adel Javanmard Jason D. Lee 60 417 0 16 Jul 2017
Global optimality conditions for deep neural networks Chulhee Yun S. Sra Ali Jadbabaie 132 118 0 08 Jul 2017
Recovery Guarantees for One-hidden-layer Neural Networks Kai Zhong Zhao Song Prateek Jain Peter L. Bartlett Inderjit S. Dhillon MLT 60 336 0 10 Jun 2017
Weight Sharing is Crucial to Succesful Optimization Shai Shalev-Shwartz Ohad Shamir Shaked Shammah 59 12 0 02 Jun 2017
Convergence Analysis of Two-layer Neural Networks with ReLU Activation Yuanzhi Li Yang Yuan MLT 25 649 0 28 May 2017
Learning ReLUs via Gradient Descent Mahdi Soltanolkotabi MLT 48 181 0 10 May 2017
The loss surface of deep and wide neural networks Quynh N. Nguyen Matthias Hein ODL 70 284 0 26 Apr 2017
Failures of Gradient-Based Deep Learning Shai Shalev-Shwartz Ohad Shamir Shaked Shammah ODL UQCV 53 200 0 23 Mar 2017
How to Escape Saddle Points Efficiently Chi Jin Rong Ge Praneeth Netrapalli Sham Kakade Michael I. Jordan ODL 90 834 0 02 Mar 2017
An Analytical Formula of Population Gradient for two-layered ReLU network and its Applications in Convergence and Critical Point Analysis Yuandong Tian MLT 40 216 0 02 Mar 2017
Globally Optimal Gradient Descent for a ConvNet with Gaussian Inputs Alon Brutzkus Amir Globerson MLT 33 313 0 26 Feb 2017
Provable learning of Noisy-or Networks Sanjeev Arora Rong Ge Tengyu Ma Andrej Risteski 44 26 0 28 Dec 2016
Reliably Learning the ReLU in Polynomial Time Surbhi Goel Varun Kanade Adam R. Klivans J. Thaler 49 126 0 30 Nov 2016
Identity Matters in Deep Learning Moritz Hardt Tengyu Ma OOD 34 398 0 14 Nov 2016
Understanding deep learning requires rethinking generalization Chiyuan Zhang Samy Bengio Moritz Hardt Benjamin Recht Oriol Vinyals HAI 190 4,612 0 10 Nov 2016
Diverse Neural Network Learns True Target Functions Bo Xie Yingyu Liang Le Song 69 137 0 09 Nov 2016
Error bounds for approximations with deep ReLU networks Dmitry Yarotsky 63 1,226 0 03 Oct 2016
Distribution-Specific Hardness of Learning Neural Networks Ohad Shamir 32 116 0 05 Sep 2016
The Landscape of Empirical Risk for Non-convex Losses Song Mei Yu Bai Andrea Montanari 18 312 0 22 Jul 2016
Exponential expressivity in deep neural networks through transient chaos Ben Poole Subhaneil Lahiri M. Raghu Jascha Narain Sohl-Dickstein Surya Ganguli 60 587 0 16 Jun 2016
On the Expressive Power of Deep Neural Networks M. Raghu Ben Poole Jon M. Kleinberg Surya Ganguli Jascha Narain Sohl-Dickstein 38 780 0 16 Jun 2016
No bad local minima: Data independent training error guarantees for multilayer neural networks Daniel Soudry Y. Carmon 63 235 0 26 May 2016
Deep Learning without Poor Local Minima Kenji Kawaguchi ODL 65 919 0 23 May 2016
Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity Amit Daniely Roy Frostig Y. Singer 32 343 0 18 Feb 2016
Benefits of depth in neural networks Matus Telgarsky 227 605 0 14 Feb 2016
Learning Halfspaces and Neural Networks with Random Initialization Yuchen Zhang Jason D. Lee Martin J. Wainwright Michael I. Jordan 34 35 0 25 Nov 2015
Expressiveness of Rectifier Networks Xingyuan Pan Vivek Srikumar OffRL 34 46 0 18 Nov 2015
On the Quality of the Initial Basin in Overspecified Neural Networks Itay Safran Ohad Shamir 35 127 0 13 Nov 2015
On the Expressive Power of Deep Learning: A Tensor Analysis Nadav Cohen Or Sharir Amnon Shashua 48 469 0 16 Sep 2015
Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decomposition Rong Ge Furong Huang Chi Jin Yang Yuan 95 1,056 0 06 Mar 2015
Provable Methods for Training Neural Networks with Sparse Connectivity Hanie Sedghi Anima Anandkumar 32 64 0 08 Dec 2014