Gradient Descent Learns One-hidden-layer CNN: Don't be Afraid of Spurious Local Minima

3 December 2017

Aarti Singh

Papers citing "Gradient Descent Learns One-hidden-layer CNN: Don't be Afraid of Spurious Local Minima"

50 / 64 papers shown

Title
System Identification and Control Using Lyapunov-Based Deep Neural Networks without Persistent Excitation: A Concurrent Learning Approach Rebecca G. Hart Omkar Sudhir Patil Zachary I. Bell Warren E. Dixon 14 0 0 15 May 2025
Analysis of the rate of convergence of an over-parametrized convolutional neural network image classifier learned by gradient descent Michael Kohler A. Krzyżak Benjamin Walter 36 1 0 13 May 2024
$How does promoting the minority fraction affect generalization? A theoretical study of the one-hidden-layer neural network on group imbalance$ How does promoting the minority fraction affect generalization? A theoretical study of the one-hidden-layer neural network on group imbalance Hongkang Li Shuai Zhang Yihua Zhang Meng Wang Sijia Liu Pin-Yu Chen 46 4 0 12 Mar 2024
Analysis of the expected $L_2$ error of an over-parametrized deep neural network estimate learned by gradient descent without regularization Selina Drews Michael Kohler 36 3 0 24 Nov 2023
Over-Parameterization Exponentially Slows Down Gradient Descent for Learning a Single Neuron Weihang Xu S. Du 37 16 0 20 Feb 2023
Annihilation of Spurious Minima in Two-Layer ReLU Networks Yossi Arjevani M. Field 16 8 0 12 Oct 2022
Towards Theoretically Inspired Neural Initialization Optimization Yibo Yang Hong Wang Haobo Yuan Zhouchen Lin 29 9 0 12 Oct 2022
Explicitising The Implicit Intrepretability of Deep Neural Networks Via Duality Chandrashekar Lakshminarayanan Ashutosh Kumar Singh A. Rajkumar AI4CE 26 1 0 01 Mar 2022
Understanding Deep Contrastive Learning via Coordinate-wise Optimization Yuandong Tian 52 34 0 29 Jan 2022
Mode connectivity in the loss landscape of parameterized quantum circuits Kathleen E. Hamilton E. Lynn R. Pooser 29 3 0 09 Nov 2021
Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Pruned Neural Networks Shuai Zhang Meng Wang Sijia Liu Pin-Yu Chen Jinjun Xiong UQCV MLT 31 13 0 12 Oct 2021
Analytic Study of Families of Spurious Minima in Two-Layer ReLU Neural Networks: A Tale of Symmetry II Yossi Arjevani M. Field 28 18 0 21 Jul 2021
Continual Learning in the Teacher-Student Setup: Impact of Task Similarity Sebastian Lee Sebastian Goldt Andrew M. Saxe CLL 32 73 0 09 Jul 2021
Neural Active Learning with Performance Guarantees Pranjal Awasthi Christoph Dann Claudio Gentile Ayush Sekhari Zhilei Wang 29 22 0 06 Jun 2021
From Local Pseudorandom Generators to Hardness of Learning Amit Daniely Gal Vardi 109 30 0 20 Jan 2021
Learning Graph Neural Networks with Approximate Gradient Descent Qunwei Li Shaofeng Zou Leon Wenliang Zhong GNN 32 1 0 07 Dec 2020
Align, then memorise: the dynamics of learning with feedback alignment Maria Refinetti Stéphane dÁscoli Ruben Ohana Sebastian Goldt 26 36 0 24 Nov 2020
Computational Separation Between Convolutional and Fully-Connected Networks Eran Malach Shai Shalev-Shwartz 24 26 0 03 Oct 2020
Generalized Leverage Score Sampling for Neural Networks J. Lee Ruoqi Shen Zhao Song Mengdi Wang Zheng Yu 21 42 0 21 Sep 2020
Nonparametric Learning of Two-Layer ReLU Residual Units Zhunxuan Wang Linyun He Chunchuan Lyu Shay B. Cohen MLT OffRL 33 1 0 17 Aug 2020
From Boltzmann Machines to Neural Networks and Back Again Surbhi Goel Adam R. Klivans Frederic Koehler 19 5 0 25 Jul 2020
The Gaussian equivalence of generative models for learning with shallow neural networks Sebastian Goldt Bruno Loureiro Galen Reeves Florent Krzakala M. Mézard Lenka Zdeborová BDL 41 100 0 25 Jun 2020
Understanding and Improving Information Transfer in Multi-Task Learning Sen Wu Hongyang R. Zhang Christopher Ré 18 154 0 02 May 2020
Orthogonal Over-Parameterized Training Weiyang Liu Rongmei Lin Zhen Liu James M. Rehg Liam Paull Li Xiong Le Song Adrian Weller 32 41 0 09 Apr 2020
Symmetry & critical points for a model shallow neural network Yossi Arjevani M. Field 34 13 0 23 Mar 2020
An Optimization and Generalization Analysis for Max-Pooling Networks Alon Brutzkus Amir Globerson MLT AI4CE 16 4 0 22 Feb 2020
Replica Exchange for Non-Convex Optimization Jing-rong Dong Xin T. Tong 27 21 0 23 Jan 2020
Thresholds of descending algorithms in inference problems Stefano Sarao Mannelli Lenka Zdeborova AI4CE 24 4 0 02 Jan 2020
Optimization for deep learning: theory and algorithms Ruoyu Sun ODL 27 168 0 19 Dec 2019
Naive Gabor Networks for Hyperspectral Image Classification Chenying Liu Jun Li Lin He Antonio J. Plaza Shutao Li Bo Li 37 45 0 09 Dec 2019
Theoretical Issues in Deep Networks: Approximation, Optimization and Generalization T. Poggio Andrzej Banburski Q. Liao ODL 31 161 0 25 Aug 2019
Hessian based analysis of SGD for Deep Nets: Dynamics and Generalization Xinyan Li Qilong Gu Yingxue Zhou Tiancong Chen A. Banerjee ODL 42 51 0 24 Jul 2019
Recursive Sketches for Modular Deep Learning Badih Ghazi Rina Panigrahy Joshua R. Wang 13 20 0 29 May 2019
Fine-grained Optimization of Deep Neural Networks Mete Ozay ODL 16 1 0 22 May 2019
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruosong Wang MLT 55 961 0 24 Jan 2019
Width Provably Matters in Optimization for Deep Linear Neural Networks S. Du Wei Hu 23 94 0 24 Jan 2019
Understanding Geometry of Encoder-Decoder CNNs J. C. Ye Woon Kyoung Sung 3DV AI4CE 11 72 0 22 Jan 2019
Convex Relaxations of Convolutional Neural Nets Burak Bartan Mert Pilanci 20 5 0 31 Dec 2018
Non-attracting Regions of Local Minima in Deep and Wide Neural Networks Henning Petzka C. Sminchisescu 29 9 0 16 Dec 2018
Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks Difan Zou Yuan Cao Dongruo Zhou Quanquan Gu ODL 33 446 0 21 Nov 2018
Gradient Descent Finds Global Minima of Deep Neural Networks S. Du J. Lee Haochuan Li Liwei Wang Masayoshi Tomizuka ODL 44 1,125 0 09 Nov 2018
On the Convergence Rate of Training Recurrent Neural Networks Zeyuan Allen-Zhu Yuanzhi Li Zhao Song 26 191 0 29 Oct 2018
Subgradient Descent Learns Orthogonal Dictionaries Yu Bai Qijia Jiang Ju Sun 20 51 0 25 Oct 2018
Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity Chulhee Yun S. Sra Ali Jadbabaie 28 117 0 17 Oct 2018
Learning Two-layer Neural Networks with Symmetric Inputs Rong Ge Rohith Kuditipudi Zhize Li Xiang Wang OOD MLT 36 57 0 16 Oct 2018
Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel Colin Wei J. Lee Qiang Liu Tengyu Ma 26 245 0 12 Oct 2018
Why do Larger Models Generalize Better? A Theoretical Perspective via the XOR Problem Alon Brutzkus Amir Globerson MLT 13 7 0 06 Oct 2018
A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks Sanjeev Arora Nadav Cohen Noah Golowich Wei Hu 27 281 0 04 Oct 2018
Gradient Descent Provably Optimizes Over-parameterized Neural Networks S. Du Xiyu Zhai Barnabás Póczós Aarti Singh MLT ODL 56 1,252 0 04 Oct 2018
A theoretical framework for deep locally connected ReLU network Yuandong Tian PINN 25 10 0 28 Sep 2018