Gradient Descent Provably Optimizes Over-parameterized Neural Networks

4 October 2018

Aarti Singh

Papers citing "Gradient Descent Provably Optimizes Over-parameterized Neural Networks"

50 / 302 papers shown

Title
Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHS Lin Chen Sheng Xu 30 93 0 22 Sep 2020
Generalized Leverage Score Sampling for Neural Networks J. Lee Ruoqi Shen Zhao Song Mengdi Wang Zheng Yu 21 42 0 21 Sep 2020
GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training Tianle Cai Shengjie Luo Keyulu Xu Di He Tie-Yan Liu Liwei Wang GNN 32 158 0 07 Sep 2020
Predicting Training Time Without Training L. Zancato Alessandro Achille Avinash Ravichandran Rahul Bhotika Stefano Soatto 26 24 0 28 Aug 2020
Deep Networks and the Multiple Manifold Problem Sam Buchanan D. Gilboa John N. Wright 166 39 0 25 Aug 2020
Adversarial Training and Provable Robustness: A Tale of Two Objectives Jiameng Fan Wenchao Li AAML 23 20 0 13 Aug 2020
The Interpolation Phase Transition in Neural Networks: Memorization and Generalization under Lazy Training Andrea Montanari Yiqiao Zhong 49 95 0 25 Jul 2020
Geometric compression of invariant manifolds in neural nets J. Paccolat Leonardo Petrini Mario Geiger Kevin Tyloo M. Wyart MLT 55 34 0 22 Jul 2020
Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy E. Moroshko Suriya Gunasekar Blake E. Woodworth J. Lee Nathan Srebro Daniel Soudry 35 85 0 13 Jul 2020
A Revision of Neural Tangent Kernel-based Approaches for Neural Networks Kyungsu Kim A. Lozano Eunho Yang AAML 32 0 0 02 Jul 2020
Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theory Tao Luo Haizhao Yang 32 73 0 28 Jun 2020
Tensor Programs II: Neural Tangent Kernel for Any Architecture Greg Yang 58 134 0 25 Jun 2020
Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descent Mehdi Abbana Bennani Thang Doan Masashi Sugiyama CLL 50 61 0 21 Jun 2020
Training (Overparametrized) Neural Networks in Near-Linear Time Jan van den Brand Binghui Peng Zhao Song Omri Weinstein ODL 29 82 0 20 Jun 2020
An analytic theory of shallow networks dynamics for hinge loss classification Franco Pellegrini Giulio Biroli 35 19 0 19 Jun 2020
Exploring Weight Importance and Hessian Bias in Model Pruning Mingchen Li Yahya Sattar Christos Thrampoulidis Samet Oymak 28 3 0 19 Jun 2020
Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains Matthew Tancik Pratul P. Srinivasan B. Mildenhall Sara Fridovich-Keil N. Raghavan Utkarsh Singhal R. Ramamoorthi Jonathan T. Barron Ren Ng 60 2,344 0 18 Jun 2020
Kernel Alignment Risk Estimator: Risk Prediction from Training Data Arthur Jacot Berfin cSimcsek Francesco Spadaro Clément Hongler Franck Gabriel 22 67 0 17 Jun 2020
Directional Pruning of Deep Neural Networks Shih-Kang Chao Zhanyu Wang Yue Xing Guang Cheng ODL 21 33 0 16 Jun 2020
Optimization and Generalization Analysis of Transduction through Gradient Boosting and Application to Multi-scale Graph Neural Networks Kenta Oono Taiji Suzuki AI4CE 37 31 0 15 Jun 2020
Non-convergence of stochastic gradient descent in the training of deep neural networks Patrick Cheridito Arnulf Jentzen Florian Rossmannek 14 37 0 12 Jun 2020
H3DNet: 3D Object Detection Using Hybrid Geometric Primitives Zaiwei Zhang Bo Sun Haitao Yang Qi-Xing Huang 3DPC 20 195 0 10 Jun 2020
Can Temporal-Difference and Q-Learning Learn Representation? A Mean-Field Theory Yufeng Zhang Qi Cai Zhuoran Yang Yongxin Chen Zhaoran Wang OOD MLT 108 11 0 08 Jun 2020
An Overview of Neural Network Compression James OÑeill AI4CE 45 98 0 05 Jun 2020
Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networks Z. Fan Zhichao Wang 44 71 0 25 May 2020
Can Shallow Neural Networks Beat the Curse of Dimensionality? A mean field training perspective Stephan Wojtowytsch E. Weinan MLT 26 48 0 21 May 2020
Feature Purification: How Adversarial Training Performs Robust Deep Learning Zeyuan Allen-Zhu Yuanzhi Li MLT AAML 37 147 0 20 May 2020
Learning the gravitational force law and other analytic functions Atish Agarwala Abhimanyu Das Rina Panigrahy Qiuyi Zhang MLT 16 0 0 15 May 2020
Compressive sensing with un-trained neural networks: Gradient descent finds the smoothest approximation Reinhard Heckel Mahdi Soltanolkotabi 8 79 0 07 May 2020
Analysis of Knowledge Transfer in Kernel Regime Arman Rahbar Ashkan Panahi Chiranjib Bhattacharyya Devdatt Dubhashi M. Chehreghani 23 3 0 30 Mar 2020
Symmetry & critical points for a model shallow neural network Yossi Arjevani M. Field 34 13 0 23 Mar 2020
Critical Point-Finding Methods Reveal Gradient-Flat Regions of Deep Network Losses Charles G. Frye James B. Simon Neha S. Wadia A. Ligeralde M. DeWeese K. Bouchard ODL 16 2 0 23 Mar 2020
Frequency Bias in Neural Networks for Input of Non-Uniform Density Ronen Basri Meirav Galun Amnon Geifman David Jacobs Yoni Kasten S. Kritchman 39 183 0 10 Mar 2020
Overall error analysis for the training of deep neural networks via stochastic gradient descent with random initialisation Arnulf Jentzen Timo Welti 22 15 0 03 Mar 2020
Loss landscapes and optimization in over-parameterized non-linear systems and neural networks Chaoyue Liu Libin Zhu M. Belkin ODL 17 247 0 29 Feb 2020
Convex Geometry and Duality of Over-parameterized Neural Networks Tolga Ergen Mert Pilanci MLT 42 54 0 25 Feb 2020
Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-layer Networks Mert Pilanci Tolga Ergen 26 116 0 24 Feb 2020
An Optimization and Generalization Analysis for Max-Pooling Networks Alon Brutzkus Amir Globerson MLT AI4CE 16 4 0 22 Feb 2020
Generalisation error in learning with random features and the hidden manifold model Federica Gerace Bruno Loureiro Florent Krzakala M. Mézard Lenka Zdeborová 25 165 0 21 Feb 2020
Learning Parities with Neural Networks Amit Daniely Eran Malach 24 76 0 18 Feb 2020
Over-parameterized Adversarial Training: An Analysis Overcoming the Curse of Dimensionality Yi Zhang Orestis Plevrakis S. Du Xingguo Li Zhao Song Sanjeev Arora 21 51 0 16 Feb 2020
Estimating Uncertainty Intervals from Collaborating Networks Tianhui Zhou Yitong Li Yuan Wu David Carlson UQCV 30 15 0 12 Feb 2020
Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss Lénaïc Chizat Francis R. Bach MLT 39 327 0 11 Feb 2020
Proving the Lottery Ticket Hypothesis: Pruning is All You Need Eran Malach Gilad Yehudai Shai Shalev-Shwartz Ohad Shamir 64 271 0 03 Feb 2020
Memory capacity of neural networks with threshold and ReLU activations Roman Vershynin 31 21 0 20 Jan 2020
Neural Tangents: Fast and Easy Infinite Neural Networks in Python Roman Novak Lechao Xiao Jiri Hron Jaehoon Lee Alexander A. Alemi Jascha Narain Sohl-Dickstein S. Schoenholz 38 224 0 05 Dec 2019
Towards Understanding the Spectral Bias of Deep Learning Yuan Cao Zhiying Fang Yue Wu Ding-Xuan Zhou Quanquan Gu 41 214 0 03 Dec 2019
Neural Contextual Bandits with UCB-based Exploration Dongruo Zhou Lihong Li Quanquan Gu 36 15 0 11 Nov 2019
Enhanced Convolutional Neural Tangent Kernels Zhiyuan Li Ruosong Wang Dingli Yu S. Du Wei Hu Ruslan Salakhutdinov Sanjeev Arora 16 131 0 03 Nov 2019
Global Convergence of Gradient Descent for Deep Linear Residual Networks Lei Wu Qingcan Wang Chao Ma ODL AI4CE 28 22 0 02 Nov 2019