v1v2 (latest)

Gradient Descent Provably Optimizes Over-parameterized Neural Networks

4 October 2018

Aarti Singh

Papers citing "Gradient Descent Provably Optimizes Over-parameterized Neural Networks"

50 / 882 papers shown

Title
A General Framework for Analyzing Stochastic Dynamics in Learning Algorithms Chi-Ning Chou Juspreet Singh Sandhu Mien Brabeeba Wang Tiancheng Yu 69 4 0 11 Jun 2020
The Hidden Convex Optimization Landscape of Two-Layer ReLU Neural Networks: an Exact Characterization of the Optimal Solutions Yifei Wang Jonathan Lacotte Mert Pilanci MLT 91 27 0 10 Jun 2020
H3DNet: 3D Object Detection Using Hybrid Geometric Primitives Zaiwei Zhang Bo Sun Haitao Yang Qi-Xing Huang 3DPC 112 200 0 10 Jun 2020
Banach Space Representer Theorems for Neural Networks and Ridge Splines Rahul Parhi Robert D. Nowak 33 7 0 10 Jun 2020
Can Temporal-Difference and Q-Learning Learn Representation? A Mean-Field Theory Yufeng Zhang Qi Cai Zhuoran Yang Yongxin Chen Zhaoran Wang OOD MLT 360 11 0 08 Jun 2020
An Overview of Neural Network Compression James OÑeill AI4CE 160 100 0 05 Jun 2020
Hardness of Learning Neural Networks with Natural Weights Amit Daniely Gal Vardi 69 19 0 05 Jun 2020
The Effects of Mild Over-parameterization on the Optimization Landscape of Shallow ReLU Neural Networks Itay Safran Gilad Yehudai Ohad Shamir 148 35 0 01 Jun 2020
On the Convergence of Gradient Descent Training for Two-layer ReLU-networks in the Mean Field Regime Stephan Wojtowytsch MLT 128 51 0 27 May 2020
Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networks Z. Fan Zhichao Wang 115 74 0 25 May 2020
Can Shallow Neural Networks Beat the Curse of Dimensionality? A mean field training perspective Stephan Wojtowytsch E. Weinan MLT 72 51 0 21 May 2020
Kolmogorov Width Decay and Poor Approximators in Machine Learning: Shallow Neural Networks, Random Feature Models and Neural Tangent Kernels E. Weinan Stephan Wojtowytsch 173 31 0 21 May 2020
Feature Purification: How Adversarial Training Performs Robust Deep Learning Zeyuan Allen-Zhu Yuanzhi Li MLT AAML 122 151 0 20 May 2020
Model Repair: Robust Recovery of Over-Parameterized Statistical Models Chao Gao John D. Lafferty 44 6 0 20 May 2020
Scalable Privacy-Preserving Distributed Learning D. Froelicher J. Troncoso-Pastoriza Apostolos Pyrgelis Sinem Sav João Sá Sousa Jean-Philippe Bossuat Jean-Pierre Hubaux FedML 97 70 0 19 May 2020
Learning the gravitational force law and other analytic functions Atish Agarwala Abhimanyu Das Rina Panigrahy Qiuyi Zhang MLT 43 0 0 15 May 2020
RSO: A Gradient Free Sampling Based Approach For Training Deep Neural Networks Rohun Tripathi Bharat Singh 40 6 0 12 May 2020
Compressive sensing with un-trained neural networks: Gradient descent finds the smoothest approximation Reinhard Heckel Mahdi Soltanolkotabi 75 81 0 07 May 2020
Generalization Error of Generalized Linear Models in High Dimensions M Motavali Emami Mojtaba Sahraee-Ardakan Parthe Pandit S. Rangan A. Fletcher AI4CE 59 39 0 01 May 2020
Optimization in Machine Learning: A Distribution Space Approach Yongqiang Cai Qianxiao Li Zuowei Shen 32 1 0 18 Apr 2020
Machine-learning-based methods for output only structural modal identification Dawei Liu Zhiyi Tang Y. Bao Hui Li 6 47 0 16 Apr 2020
On the Neural Tangent Kernel of Deep Networks with Orthogonal Initialization Wei Huang Weitao Du R. Xu 74 38 0 13 Apr 2020
Mehler's Formula, Branching Process, and Compositional Kernels of Deep Neural Networks Tengyuan Liang Hai Tran-Bach 48 11 0 09 Apr 2020
Analysis of Knowledge Transfer in Kernel Regime Arman Rahbar Ashkan Panahi Chiranjib Bhattacharyya Devdatt Dubhashi M. Chehreghani 68 3 0 30 Mar 2020
Memorizing Gaussians with no over-parameterizaion via gradient decent on neural networks Amit Daniely VLM MLT 43 15 0 28 Mar 2020
Piecewise linear activations substantially shape the loss surfaces of neural networks Fengxiang He Bohan Wang Dacheng Tao ODL 91 30 0 27 Mar 2020
Volumization as a Natural Generalization of Weight Decay Liu Ziyin Zihao Wang M. Yamada Masahito Ueda AI4CE 20 0 0 25 Mar 2020
Symmetry & critical points for a model shallow neural network Yossi Arjevani M. Field 107 13 0 23 Mar 2020
Neural Networks and Polynomial Regression. Demystifying the Overparametrization Phenomena Matt Emschwiller D. Gamarnik Eren C. Kizildag Ilias Zadik 77 9 0 23 Mar 2020
Critical Point-Finding Methods Reveal Gradient-Flat Regions of Deep Network Losses Charles G. Frye James B. Simon Neha S. Wadia A. Ligeralde M. DeWeese K. Bouchard ODL 55 2 0 23 Mar 2020
Steepest Descent Neural Architecture Optimization: Escaping Local Optimum with Signed Neural Splitting Lemeng Wu Mao Ye Qi Lei Jason D. Lee Qiang Liu 88 15 0 23 Mar 2020
On Alignment in Deep Linear Neural Networks Adityanarayanan Radhakrishnan Eshaan Nichani D. Bernstein Caroline Uhler 47 2 0 13 Mar 2020
Frequency Bias in Neural Networks for Input of Non-Uniform Density Ronen Basri Meirav Galun Amnon Geifman David Jacobs Yoni Kasten S. Kritchman 92 186 0 10 Mar 2020
Neural Kernels Without Tangents Vaishaal Shankar Alex Fang Wenshuo Guo Sara Fridovich-Keil Ludwig Schmidt Jonathan Ragan-Kelley Benjamin Recht 64 91 0 04 Mar 2020
Good Subnetworks Provably Exist: Pruning via Greedy Forward Selection Mao Ye Chengyue Gong Lizhen Nie Denny Zhou Adam R. Klivans Qiang Liu 113 111 0 03 Mar 2020
Overall error analysis for the training of deep neural networks via stochastic gradient descent with random initialisation Arnulf Jentzen Timo Welti 65 17 0 03 Mar 2020
On the Global Convergence of Training Deep Linear ResNets Difan Zou Philip M. Long Quanquan Gu 78 39 0 02 Mar 2020
Loss landscapes and optimization in over-parameterized non-linear systems and neural networks Chaoyue Liu Libin Zhu M. Belkin ODL 133 266 0 29 Feb 2020
Training BatchNorm and Only BatchNorm: On the Expressive Power of Random Features in CNNs Jonathan Frankle D. Schwab Ari S. Morcos 117 143 0 29 Feb 2020
Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers Zhuohan Li Eric Wallace Sheng Shen Kevin Lin Kurt Keutzer Dan Klein Joseph E. Gonzalez 138 151 0 26 Feb 2020
Convex Geometry and Duality of Over-parameterized Neural Networks Tolga Ergen Mert Pilanci MLT 138 56 0 25 Feb 2020
Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-layer Networks Mert Pilanci Tolga Ergen 101 118 0 24 Feb 2020
De-randomized PAC-Bayes Margin Bounds: Applications to Non-convex and Non-smooth Predictors A. Banerjee Tiancong Chen Yingxue Zhou BDL 86 8 0 23 Feb 2020
An Optimization and Generalization Analysis for Max-Pooling Networks Alon Brutzkus Amir Globerson MLT AI4CE 46 4 0 22 Feb 2020
Revealing the Structure of Deep Neural Networks via Convex Duality Tolga Ergen Mert Pilanci MLT 102 72 0 22 Feb 2020
Generalisation error in learning with random features and the hidden manifold model Federica Gerace Bruno Loureiro Florent Krzakala M. Mézard Lenka Zdeborová 87 172 0 21 Feb 2020
Robust Pruning at Initialization Soufiane Hayou Jean-François Ton Arnaud Doucet Yee Whye Teh 43 47 0 19 Feb 2020
Global Convergence of Deep Networks with One Wide Layer Followed by Pyramidal Topology Quynh N. Nguyen Marco Mondelli ODL AI4CE 83 70 0 18 Feb 2020
Learning Parities with Neural Networks Amit Daniely Eran Malach 104 78 0 18 Feb 2020
Over-parameterized Adversarial Training: An Analysis Overcoming the Curse of Dimensionality Yi Zhang Orestis Plevrakis S. Du Xingguo Li Zhao Song Sanjeev Arora 127 53 0 16 Feb 2020