Recovery Guarantees for One-hidden-layer Neural Networks

10 June 2017

Inderjit S. Dhillon

Papers citing "Recovery Guarantees for One-hidden-layer Neural Networks"

50 / 223 papers shown

Title
Superpolynomial Lower Bounds for Learning One-Layer Neural Networks using Gradient Descent Surbhi Goel Aravind Gollakota Zhihan Jin Sushrut Karmalkar Adam R. Klivans MLT ODL 84 72 0 22 Jun 2020
Training (Overparametrized) Neural Networks in Near-Linear Time Jan van den Brand Binghui Peng Zhao Song Omri Weinstein ODL 91 83 0 20 Jun 2020
The Effects of Mild Over-parameterization on the Optimization Landscape of Shallow ReLU Neural Networks Itay Safran Gilad Yehudai Ohad Shamir 148 35 0 01 Jun 2020
Feature Purification: How Adversarial Training Performs Robust Deep Learning Zeyuan Allen-Zhu Yuanzhi Li MLT AAML 122 151 0 20 May 2020
$Orthant Based Proximal Stochastic Gradient Method for $\ell_1$-Regularized Optimization$ Orthant Based Proximal Stochastic Gradient Method for $\ell_1$ -Regularized Optimization Tianyi Chen Tianyu Ding Bo Ji Guanyi Wang Jing Tian Yixin Shi Sheng Yi Xiao Tu Zhihui Zhu 65 29 0 07 Apr 2020
Piecewise linear activations substantially shape the loss surfaces of neural networks Fengxiang He Bohan Wang Dacheng Tao ODL 91 30 0 27 Mar 2020
Symmetry & critical points for a model shallow neural network Yossi Arjevani M. Field 107 13 0 23 Mar 2020
On the Global Convergence of Training Deep Linear ResNets Difan Zou Philip M. Long Quanquan Gu 78 39 0 02 Mar 2020
Semiparametric Nonlinear Bipartite Graph Representation Learning with Provable Guarantees Sen Na Yuwei Luo Zhuoran Yang Zhaoran Wang Mladen Kolar 68 7 0 02 Mar 2020
Convergence of End-to-End Training in Deep Unsupervised Contrastive Learning Zixin Wen SSL 68 3 0 17 Feb 2020
A Generalized Neural Tangent Kernel Analysis for Two-layer Neural Networks Zixiang Chen Yuan Cao Quanquan Gu Tong Zhang MLT 84 10 0 10 Feb 2020
Avoiding Spurious Local Minima in Deep Quadratic Networks A. Kazemipour Brett W. Larsen S. Druckmann ODL 62 6 0 31 Dec 2019
Revisiting Landscape Analysis in Deep Neural Networks: Eliminating Decreasing Paths to Infinity Shiyu Liang Ruoyu Sun R. Srikant 83 20 0 31 Dec 2019
On the Principle of Least Symmetry Breaking in Shallow ReLU Models Yossi Arjevani M. Field 65 8 0 26 Dec 2019
Optimization for deep learning: theory and algorithms Ruoyu Sun ODL 137 169 0 19 Dec 2019
Stationary Points of Shallow Neural Networks with Quadratic Activation Function D. Gamarnik Eren C. Kizildag Ilias Zadik 45 14 0 03 Dec 2019
Benefits of Jointly Training Autoencoders: An Improved Neural Tangent Kernel Analysis THANH VAN NGUYEN Raymond K. W. Wong Chinmay Hegde 84 13 0 27 Nov 2019
Tight Sample Complexity of Learning One-hidden-layer Convolutional Neural Networks Yuan Cao Quanquan Gu MLT 83 19 0 12 Nov 2019
Time/Accuracy Tradeoffs for Learning a ReLU with respect to Gaussian Marginals Surbhi Goel Sushrut Karmalkar Adam R. Klivans 95 53 0 04 Nov 2019
Sub-Optimal Local Minima Exist for Neural Networks with Almost All Non-Linear Activations Tian Ding Dawei Li Ruoyu Sun 95 13 0 04 Nov 2019
Denoising and Regularization via Exploiting the Structural Bias of Convolutional Generators Reinhard Heckel Mahdi Soltanolkotabi DiffM 106 83 0 31 Oct 2019
Improved Zeroth-Order Variance Reduced Algorithms and Analysis for Nonconvex Optimization Kaiyi Ji Zhe Wang Yi Zhou Yingbin Liang 78 77 0 27 Oct 2019
History-Gradient Aided Batch Size Adaptation for Variance Reduced Algorithms Kaiyi Ji Zhe Wang Bowen Weng Yi Zhou Wei Zhang Yingbin Liang ODL 60 5 0 21 Oct 2019
Student Specialization in Deep ReLU Networks With Finite Width and Input Dimension Yuandong Tian MLT 60 8 0 30 Sep 2019
Towards Understanding the Importance of Noise in Training Neural Networks Mo Zhou Tianyi Liu Yan Li Dachao Lin Enlu Zhou T. Zhao MLT 92 26 0 07 Sep 2019
Learning Distributions Generated by One-Layer ReLU Networks Shanshan Wu A. Dimakis Sujay Sanghavi 84 22 0 04 Sep 2019
Max-Affine Regression: Provable, Tractable, and Near-Optimal Statistical Estimation Avishek Ghosh A. Pananjady Adityanand Guntuboyina Kannan Ramchandran 68 26 0 21 Jun 2019
Limitations of Lazy Training of Two-layers Neural Networks Behrooz Ghorbani Song Mei Theodor Misiakiewicz Andrea Montanari MLT 62 143 0 21 Jun 2019
Gradient Descent Maximizes the Margin of Homogeneous Neural Networks Kaifeng Lyu Jian Li 114 336 0 13 Jun 2019
An Improved Analysis of Training Over-parameterized Deep Neural Networks Difan Zou Quanquan Gu 80 235 0 11 Jun 2019
Quadratic Suffices for Over-parametrization via Matrix Chernoff Bound Zhao Song Xin Yang 75 91 0 09 Jun 2019
Learning in Gated Neural Networks Ashok Vardhan Makkuva Sewoong Oh Sreeram Kannan Pramod Viswanath 58 11 0 06 Jun 2019
Recursive Sketches for Modular Deep Learning Badih Ghazi Rina Panigrahy Joshua R. Wang 78 20 0 29 May 2019
On Learning Over-parameterized Neural Networks: A Functional Approximation Perspective Lili Su Pengkun Yang MLT 80 54 0 26 May 2019
What Can ResNet Learn Efficiently, Going Beyond Kernels? Zeyuan Allen-Zhu Yuanzhi Li 418 183 0 24 May 2019
Gradient Descent can Learn Less Over-parameterized Two-layer Neural Networks on Classification Problems Atsushi Nitanda Geoffrey Chinot Taiji Suzuki MLT 105 34 0 23 May 2019
A Selective Overview of Deep Learning Jianqing Fan Cong Ma Yiqiao Zhong BDL VLM 210 135 0 10 Apr 2019
Every Local Minimum Value is the Global Minimum Value of Induced Model in Non-convex Machine Learning Kenji Kawaguchi Jiaoyang Huang L. Kaelbling AAML 98 18 0 07 Apr 2019
Recovering the Lowest Layer of Deep Networks with High Threshold Activations Surbhi Goel Rina Panigrahy FedML 19 0 0 21 Mar 2019
Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets Penghang Yin J. Lyu Shuai Zhang Stanley Osher Y. Qi Jack Xin MQ LLMSV 154 320 0 13 Mar 2019
Theory III: Dynamics and Generalization in Deep Networks Andrzej Banburski Q. Liao Alycia Lee Lorenzo Rosasco Fernanda De La Torre Jack Hidary T. Poggio AI4CE 74 3 0 12 Mar 2019
Towards moderate overparameterization: global convergence guarantees for training shallow neural networks Samet Oymak Mahdi Soltanolkotabi 79 323 0 12 Feb 2019
Mean Field Limit of the Learning Dynamics of Multilayer Neural Networks Phan-Minh Nguyen AI4CE 82 72 0 07 Feb 2019
Generalization Error Bounds of Gradient Descent for Learning Over-parameterized Deep ReLU Networks Yuan Cao Quanquan Gu ODL MLT AI4CE 158 159 0 04 Feb 2019
Can SGD Learn Recurrent Neural Networks with Provable Generalization? Zeyuan Allen-Zhu Yuanzhi Li MLT LRM 181 57 0 04 Feb 2019
DANTE: Deep AlterNations for Training nEural networks Vaibhav Sinha Sneha Kudugunta Adepu Ravi Sankar Surya Teja Chavali Purushottam Kar V. Balasubramanian 27 3 0 01 Feb 2019
Sharp Analysis for Nonconvex SGD Escaping from Saddle Points Cong Fang Zhouchen Lin Tong Zhang 92 104 0 01 Feb 2019
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruosong Wang MLT 245 974 0 24 Jan 2019
Width Provably Matters in Optimization for Deep Linear Neural Networks S. Du Wei Hu 117 95 0 24 Jan 2019
Analysis of a Two-Layer Neural Network via Displacement Convexity Adel Javanmard Marco Mondelli Andrea Montanari MLT 119 57 0 05 Jan 2019