Theoretical insights into the optimization landscape of over-parameterized shallow neural networks

16 July 2017

Papers citing "Theoretical insights into the optimization landscape of over-parameterized shallow neural networks"

47 / 97 papers shown

Title
The Interpolation Phase Transition in Neural Networks: Memorization and Generalization under Lazy Training Andrea Montanari Yiqiao Zhong 47 95 0 25 Jul 2020
Quantitative Propagation of Chaos for SGD in Wide Neural Networks Valentin De Bortoli Alain Durmus Xavier Fontaine Umut Simsekli 22 25 0 13 Jul 2020
The Gaussian equivalence of generative models for learning with shallow neural networks Sebastian Goldt Bruno Loureiro Galen Reeves Florent Krzakala M. Mézard Lenka Zdeborová BDL 41 100 0 25 Jun 2020
Non-convergence of stochastic gradient descent in the training of deep neural networks Patrick Cheridito Arnulf Jentzen Florian Rossmannek 14 37 0 12 Jun 2020
Probably Approximately Correct Constrained Learning Luiz F. O. Chamon Alejandro Ribeiro 13 37 0 09 Jun 2020
Feature Purification: How Adversarial Training Performs Robust Deep Learning Zeyuan Allen-Zhu Yuanzhi Li MLT AAML 27 147 0 20 May 2020
Compressive sensing with un-trained neural networks: Gradient descent finds the smoothest approximation Reinhard Heckel Mahdi Soltanolkotabi 6 79 0 07 May 2020
Symmetry & critical points for a model shallow neural network Yossi Arjevani M. Field 26 13 0 23 Mar 2020
On Interpretability of Artificial Neural Networks: A Survey Fenglei Fan Jinjun Xiong Mengzhou Li Ge Wang AAML AI4CE 38 300 0 08 Jan 2020
Revisiting Landscape Analysis in Deep Neural Networks: Eliminating Decreasing Paths to Infinity Shiyu Liang Ruoyu Sun R. Srikant 25 19 0 31 Dec 2019
Convergence and sample complexity of gradient methods for the model-free linear quadratic regulator problem Hesameddin Mohammadi A. Zare Mahdi Soltanolkotabi M. Jovanović 32 121 0 26 Dec 2019
Denoising and Regularization via Exploiting the Structural Bias of Convolutional Generators Reinhard Heckel Mahdi Soltanolkotabi DiffM 27 81 0 31 Oct 2019
The Local Elasticity of Neural Networks Hangfeng He Weijie J. Su 26 44 0 15 Oct 2019
Beyond Linearization: On Quadratic and Higher-Order Approximation of Wide Neural Networks Yu Bai J. Lee 16 116 0 03 Oct 2019
Towards Scalable Koopman Operator Learning: Convergence Rates and A Distributed Learning Algorithm Zhiyuan Liu Guohui Ding Lijun Chen Enoch Yeung 12 3 0 30 Sep 2019
Generating Accurate Pseudo-labels in Semi-Supervised Learning and Avoiding Overconfident Predictions via Hermite Polynomial Activations Vishnu Suresh Lokhande Songwong Tasneeyapant Abhay Venkatesh Sathya Ravi Vikas Singh 11 29 0 12 Sep 2019
Neural ODEs as the Deep Limit of ResNets with constant weights B. Avelin K. Nystrom ODL 34 31 0 28 Jun 2019
Greedy Shallow Networks: An Approach for Constructing and Training Neural Networks Anton Dereventsov Armenak Petrosyan Clayton Webster 15 9 0 24 May 2019
What Can ResNet Learn Efficiently, Going Beyond Kernels? Zeyuan Allen-Zhu Yuanzhi Li 24 183 0 24 May 2019
T-Net: Parametrizing Fully Convolutional Nets with a Single High-Order Tensor Jean Kossaifi Adrian Bulat Georgios Tzimiropoulos M. Pantic 14 67 0 04 Apr 2019
Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks Mingchen Li Mahdi Soltanolkotabi Samet Oymak NoLa 28 351 0 27 Mar 2019
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruosong Wang MLT 35 962 0 24 Jan 2019
Width Provably Matters in Optimization for Deep Linear Neural Networks S. Du Wei Hu 16 93 0 24 Jan 2019
Analysis of a Two-Layer Neural Network via Displacement Convexity Adel Javanmard Marco Mondelli Andrea Montanari MLT 40 57 0 05 Jan 2019
Reconciling modern machine learning practice and the bias-variance trade-off M. Belkin Daniel J. Hsu Siyuan Ma Soumik Mandal 36 1,610 0 28 Dec 2018
Gradient Descent Finds Global Minima of Deep Neural Networks S. Du J. Lee Haochuan Li Liwei Wang M. Tomizuka ODL 21 1,122 0 09 Nov 2018
Implicit Regularization of Stochastic Gradient Descent in Natural Language Processing: Observations and Implications Deren Lei Zichen Sun Yijun Xiao William Yang Wang 33 14 0 01 Nov 2018
On the Convergence Rate of Training Recurrent Neural Networks Zeyuan Allen-Zhu Yuanzhi Li Zhao-quan Song 18 191 0 29 Oct 2018
Fast and Faster Convergence of SGD for Over-Parameterized Models and an Accelerated Perceptron Sharan Vaswani Francis R. Bach Mark W. Schmidt 30 296 0 16 Oct 2018
Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel Colin Wei J. Lee Qiang Liu Tengyu Ma 18 243 0 12 Oct 2018
Interpretable Convolutional Neural Networks via Feedforward Design C.-C. Jay Kuo Min Zhang Siyang Li Jiali Duan Yueru Chen 30 155 0 05 Oct 2018
Gradient Descent Provably Optimizes Over-parameterized Neural Networks S. Du Xiyu Zhai Barnabás Póczós Aarti Singh MLT ODL 33 1,251 0 04 Oct 2018
Learning ReLU Networks on Linearly Separable Data: Algorithm, Optimality, and Generalization G. Wang G. Giannakis Jie Chen MLT 22 131 0 14 Aug 2018
Learning One-hidden-layer ReLU Networks via Gradient Descent Xiao Zhang Yaodong Yu Lingxiao Wang Quanquan Gu MLT 28 134 0 20 Jun 2018
Data augmentation instead of explicit regularization Alex Hernández-García Peter König 30 141 0 11 Jun 2018
Stochastic Gradient/Mirror Descent: Minimax Optimality and Implicit Regularization Navid Azizan B. Hassibi 16 61 0 04 Jun 2018
On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport Lénaïc Chizat Francis R. Bach OT 31 723 0 24 May 2018
End-to-end Learning of a Convolutional Neural Network via Deep Tensor Decomposition Samet Oymak Mahdi Soltanolkotabi 19 12 0 16 May 2018
A Mean Field View of the Landscape of Two-Layers Neural Networks Song Mei Andrea Montanari Phan-Minh Nguyen MLT 23 849 0 18 Apr 2018
Learning Compact Neural Networks with Regularization Samet Oymak MLT 35 39 0 05 Feb 2018
Fix your classifier: the marginal value of training the last weight layer Elad Hoffer Itay Hubara Daniel Soudry 27 101 0 14 Jan 2018
Visualizing the Loss Landscape of Neural Nets Hao Li Zheng Xu Gavin Taylor Christoph Studer Tom Goldstein 72 1,843 0 28 Dec 2017
Spurious Local Minima are Common in Two-Layer ReLU Neural Networks Itay Safran Ohad Shamir 29 261 0 24 Dec 2017
SGD Learns Over-parameterized Networks that Provably Generalize on Linearly Separable Data Alon Brutzkus Amir Globerson Eran Malach Shai Shalev-Shwartz MLT 37 276 0 27 Oct 2017
On Data-Driven Saak Transform C.-C. Jay Kuo Yueru Chen AI4TS 13 93 0 11 Oct 2017
Benefits of depth in neural networks Matus Telgarsky 142 602 0 14 Feb 2016
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 179 1,185 0 30 Nov 2014