v1v2 (latest)

Gradient Descent Learns One-hidden-layer CNN: Don't be Afraid of Spurious Local Minima

3 December 2017

Aarti Singh

Papers citing "Gradient Descent Learns One-hidden-layer CNN: Don't be Afraid of Spurious Local Minima"

50 / 102 papers shown

Title
Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias Stéphane dÁscoli Levent Sagun Joan Bruna Giulio Biroli 98 37 0 16 Jun 2019
Quadratic Suffices for Over-parametrization via Matrix Chernoff Bound Zhao Song Xin Yang 82 91 0 09 Jun 2019
Recursive Sketches for Modular Deep Learning Badih Ghazi Rina Panigrahy Joshua R. Wang 85 20 0 29 May 2019
On the Learning Dynamics of Two-layer Nonlinear Convolutional Neural Networks Ting Yu Junzhao Zhang Zhanxing Zhu MLT 46 5 0 24 May 2019
Convergence Analyses of Online ADAM Algorithm in Convex Setting and Two-Layer ReLU Neural Network Biyi Fang Diego Klabjan 131 9 0 22 May 2019
Fine-grained Optimization of Deep Neural Networks Mete Ozay ODL 64 2 0 22 May 2019
Budgeted Training: Rethinking Deep Neural Network Training Under Resource Constraints Mengtian Li Ersin Yumer Deva Ramanan 72 49 0 12 May 2019
On Stationary-Point Hitting Time and Ergodicity of Stochastic Gradient Langevin Dynamics Xi Chen S. Du Xin T. Tong 79 33 0 30 Apr 2019
Image Reconstruction: From Sparsity to Data-adaptive Methods and Machine Learning S. Ravishankar J. C. Ye Jeffrey A. Fessler 130 246 0 04 Apr 2019
Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets Penghang Yin J. Lyu Shuai Zhang Stanley Osher Y. Qi Jack Xin MQ LLMSV 177 320 0 13 Mar 2019
Generalization Error Bounds of Gradient Descent for Learning Over-parameterized Deep ReLU Networks Yuan Cao Quanquan Gu ODL MLT AI4CE 174 159 0 04 Feb 2019
Passed & Spurious: Descent Algorithms and Local Minima in Spiked Matrix-Tensor Models Stefano Sarao Mannelli Florent Krzakala Pierfrancesco Urbani Lenka Zdeborová 114 50 0 01 Feb 2019
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruosong Wang MLT 245 974 0 24 Jan 2019
Width Provably Matters in Optimization for Deep Linear Neural Networks S. Du Wei Hu 130 95 0 24 Jan 2019
Understanding Geometry of Encoder-Decoder CNNs J. C. Ye Woon Kyoung Sung 3DV AI4CE 97 74 0 22 Jan 2019
Convex Relaxations of Convolutional Neural Nets Burak Bartan Mert Pilanci 110 5 0 31 Dec 2018
Non-attracting Regions of Local Minima in Deep and Wide Neural Networks Henning Petzka C. Sminchisescu 98 10 0 16 Dec 2018
Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks Difan Zou Yuan Cao Dongruo Zhou Quanquan Gu ODL 290 448 0 21 Nov 2018
A Convergence Theory for Deep Learning via Over-Parameterization Zeyuan Allen-Zhu Yuanzhi Li Zhao Song AI4CE ODL 331 1,472 0 09 Nov 2018
Gradient Descent Finds Global Minima of Deep Neural Networks S. Du Jason D. Lee Haochuan Li Liwei Wang Masayoshi Tomizuka ODL 373 1,137 0 09 Nov 2018
On the Convergence Rate of Training Recurrent Neural Networks Zeyuan Allen-Zhu Yuanzhi Li Zhao Song 264 193 0 29 Oct 2018
Subgradient Descent Learns Orthogonal Dictionaries Yu Bai Qijia Jiang Ju Sun 167 51 0 25 Oct 2018
Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity Chulhee Yun S. Sra Ali Jadbabaie 166 118 0 17 Oct 2018
Learning Two-layer Neural Networks with Symmetric Inputs Rong Ge Rohith Kuditipudi Zhize Li Xiang Wang OOD MLT 203 59 0 16 Oct 2018
Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel Colin Wei Jason D. Lee Qiang Liu Tengyu Ma 287 245 0 12 Oct 2018
Why do Larger Models Generalize Better? A Theoretical Perspective via the XOR Problem Alon Brutzkus Amir Globerson MLT 67 7 0 06 Oct 2018
A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks Sanjeev Arora Nadav Cohen Noah Golowich Wei Hu 186 293 0 04 Oct 2018
Gradient Descent Provably Optimizes Over-parameterized Neural Networks S. Du Xiyu Zhai Barnabás Póczós Aarti Singh MLT ODL 474 1,275 0 04 Oct 2018
A theoretical framework for deep locally connected ReLU network Yuandong Tian PINN 64 10 0 28 Sep 2018
On the loss landscape of a class of deep neural networks with no bad local valleys Quynh N. Nguyen Mahesh Chandra Mukkamala Matthias Hein 99 87 0 27 Sep 2018
Global Convergence of Stochastic Gradient Hamiltonian Monte Carlo for Non-Convex Stochastic Optimization: Non-Asymptotic Performance Bounds and Momentum-Based Acceleration Xuefeng Gao Mert Gurbuzbalaban Lingjiong Zhu 105 60 0 12 Sep 2018
Blended Coarse Gradient Descent for Full Quantization of Deep Neural Networks Penghang Yin Shuai Zhang J. Lyu Stanley Osher Y. Qi Jack Xin MQ 113 62 0 15 Aug 2018
Collapse of Deep and Narrow Neural Nets Lu Lu Yanhui Su George Karniadakis ODL 98 156 0 15 Aug 2018
Learning ReLU Networks on Linearly Separable Data: Algorithm, Optimality, and Generalization G. Wang G. Giannakis Jie Chen MLT 83 132 0 14 Aug 2018
ResNet with one-neuron hidden layers is a Universal Approximator Hongzhou Lin Stefanie Jegelka 128 229 0 28 Jun 2018
Learning One-hidden-layer ReLU Networks via Gradient Descent Xiao Zhang Yaodong Yu Lingxiao Wang Quanquan Gu MLT 133 135 0 20 Jun 2018
Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balanced S. Du Wei Hu Jason D. Lee MLT 190 243 0 04 Jun 2018
Provably convergent acceleration in factored gradient descent with applications in matrix sensing Tayo Ajayi David Mildebrath Anastasios Kyrillidis Shashanka Ubaru Georgios Kollias K. Bouchard 193 1 0 01 Jun 2018
Understanding Generalization and Optimization Performance of Deep CNNs Pan Zhou Jiashi Feng MLT 129 50 0 28 May 2018
Nonlinear Inductive Matrix Completion based on One-layer Neural Networks Kai Zhong Zhao Song Prateek Jain Inderjit S. Dhillon 23 6 0 26 May 2018
How Many Samples are Needed to Estimate a Convolutional or Recurrent Neural Network? S. Du Yining Wang Xiyu Zhai Sivaraman Balakrishnan Ruslan Salakhutdinov Aarti Singh SSL 91 57 0 21 May 2018
Improved Learning of One-hidden-layer Convolutional Neural Networks with Overlaps S. Du Surbhi Goel MLT 99 17 0 20 May 2018
End-to-end Learning of a Convolutional Neural Network via Deep Tensor Decomposition Samet Oymak Mahdi Soltanolkotabi 87 12 0 16 May 2018
Are ResNets Provably Better than Linear Predictors? Ohad Shamir 80 54 0 18 Apr 2018
On the Power of Over-parametrization in Neural Networks with Quadratic Activation S. Du Jason D. Lee 204 272 0 03 Mar 2018
Breaking the gridlock in Mixture-of-Experts: Consistent and Efficient Algorithms Ashok Vardhan Makkuva Sewoong Oh Sreeram Kannan Pramod Viswanath MoE 49 19 0 21 Feb 2018
Guaranteed Recovery of One-Hidden-Layer Neural Networks via Cross Entropy H. Fu Yuejie Chi Yingbin Liang FedML 109 39 0 18 Feb 2018
Spurious Valleys in Two-layer Neural Network Optimization Landscapes Luca Venturi Afonso S. Bandeira Joan Bruna 97 75 0 18 Feb 2018
Small nonlinearities in activation functions create bad local minima in neural networks Chulhee Yun S. Sra Ali Jadbabaie ODL 129 95 0 10 Feb 2018
Learning One Convolutional Layer with Overlapping Patches Surbhi Goel Adam R. Klivans Raghu Meka MLT 90 81 0 07 Feb 2018