ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 1810.02054
  4. Cited By
Gradient Descent Provably Optimizes Over-parameterized Neural Networks
v1v2 (latest)

Gradient Descent Provably Optimizes Over-parameterized Neural Networks

4 October 2018
S. Du
Xiyu Zhai
Barnabás Póczós
Aarti Singh
    MLTODL
ArXiv (abs)PDFHTML

Papers citing "Gradient Descent Provably Optimizes Over-parameterized Neural Networks"

50 / 882 papers shown
Title
A General Framework for Analyzing Stochastic Dynamics in Learning
  Algorithms
A General Framework for Analyzing Stochastic Dynamics in Learning Algorithms
Chi-Ning Chou
Juspreet Singh Sandhu
Mien Brabeeba Wang
Tiancheng Yu
69
4
0
11 Jun 2020
The Hidden Convex Optimization Landscape of Two-Layer ReLU Neural
  Networks: an Exact Characterization of the Optimal Solutions
The Hidden Convex Optimization Landscape of Two-Layer ReLU Neural Networks: an Exact Characterization of the Optimal Solutions
Yifei Wang
Jonathan Lacotte
Mert Pilanci
MLT
91
27
0
10 Jun 2020
H3DNet: 3D Object Detection Using Hybrid Geometric Primitives
H3DNet: 3D Object Detection Using Hybrid Geometric Primitives
Zaiwei Zhang
Bo Sun
Haitao Yang
Qi-Xing Huang
3DPC
112
202
0
10 Jun 2020
Banach Space Representer Theorems for Neural Networks and Ridge Splines
Banach Space Representer Theorems for Neural Networks and Ridge Splines
Rahul Parhi
Robert D. Nowak
33
7
0
10 Jun 2020
Can Temporal-Difference and Q-Learning Learn Representation? A
  Mean-Field Theory
Can Temporal-Difference and Q-Learning Learn Representation? A Mean-Field Theory
Yufeng Zhang
Qi Cai
Zhuoran Yang
Yongxin Chen
Zhaoran Wang
OODMLT
360
11
0
08 Jun 2020
An Overview of Neural Network Compression
An Overview of Neural Network Compression
James OÑeill
AI4CE
160
100
0
05 Jun 2020
Hardness of Learning Neural Networks with Natural Weights
Hardness of Learning Neural Networks with Natural Weights
Amit Daniely
Gal Vardi
69
19
0
05 Jun 2020
The Effects of Mild Over-parameterization on the Optimization Landscape
  of Shallow ReLU Neural Networks
The Effects of Mild Over-parameterization on the Optimization Landscape of Shallow ReLU Neural Networks
Itay Safran
Gilad Yehudai
Ohad Shamir
148
35
0
01 Jun 2020
On the Convergence of Gradient Descent Training for Two-layer
  ReLU-networks in the Mean Field Regime
On the Convergence of Gradient Descent Training for Two-layer ReLU-networks in the Mean Field Regime
Stephan Wojtowytsch
MLT
128
51
0
27 May 2020
Spectra of the Conjugate Kernel and Neural Tangent Kernel for
  linear-width neural networks
Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networks
Z. Fan
Zhichao Wang
115
74
0
25 May 2020
Can Shallow Neural Networks Beat the Curse of Dimensionality? A mean
  field training perspective
Can Shallow Neural Networks Beat the Curse of Dimensionality? A mean field training perspective
Stephan Wojtowytsch
E. Weinan
MLT
72
51
0
21 May 2020
Kolmogorov Width Decay and Poor Approximators in Machine Learning:
  Shallow Neural Networks, Random Feature Models and Neural Tangent Kernels
Kolmogorov Width Decay and Poor Approximators in Machine Learning: Shallow Neural Networks, Random Feature Models and Neural Tangent Kernels
E. Weinan
Stephan Wojtowytsch
173
31
0
21 May 2020
Feature Purification: How Adversarial Training Performs Robust Deep
  Learning
Feature Purification: How Adversarial Training Performs Robust Deep Learning
Zeyuan Allen-Zhu
Yuanzhi Li
MLTAAML
122
151
0
20 May 2020
Model Repair: Robust Recovery of Over-Parameterized Statistical Models
Model Repair: Robust Recovery of Over-Parameterized Statistical Models
Chao Gao
John D. Lafferty
44
6
0
20 May 2020
Scalable Privacy-Preserving Distributed Learning
Scalable Privacy-Preserving Distributed Learning
D. Froelicher
J. Troncoso-Pastoriza
Apostolos Pyrgelis
Sinem Sav
João Sá Sousa
Jean-Philippe Bossuat
Jean-Pierre Hubaux
FedML
97
70
0
19 May 2020
Learning the gravitational force law and other analytic functions
Learning the gravitational force law and other analytic functions
Atish Agarwala
Abhimanyu Das
Rina Panigrahy
Qiuyi Zhang
MLT
43
0
0
15 May 2020
RSO: A Gradient Free Sampling Based Approach For Training Deep Neural
  Networks
RSO: A Gradient Free Sampling Based Approach For Training Deep Neural Networks
Rohun Tripathi
Bharat Singh
40
6
0
12 May 2020
Compressive sensing with un-trained neural networks: Gradient descent
  finds the smoothest approximation
Compressive sensing with un-trained neural networks: Gradient descent finds the smoothest approximation
Reinhard Heckel
Mahdi Soltanolkotabi
75
81
0
07 May 2020
Generalization Error of Generalized Linear Models in High Dimensions
Generalization Error of Generalized Linear Models in High Dimensions
M Motavali Emami
Mojtaba Sahraee-Ardakan
Parthe Pandit
S. Rangan
A. Fletcher
AI4CE
59
39
0
01 May 2020
Optimization in Machine Learning: A Distribution Space Approach
Optimization in Machine Learning: A Distribution Space Approach
Yongqiang Cai
Qianxiao Li
Zuowei Shen
32
1
0
18 Apr 2020
Machine-learning-based methods for output only structural modal
  identification
Machine-learning-based methods for output only structural modal identification
Dawei Liu
Zhiyi Tang
Y. Bao
Hui Li
6
47
0
16 Apr 2020
On the Neural Tangent Kernel of Deep Networks with Orthogonal
  Initialization
On the Neural Tangent Kernel of Deep Networks with Orthogonal Initialization
Wei Huang
Weitao Du
R. Xu
74
38
0
13 Apr 2020
Mehler's Formula, Branching Process, and Compositional Kernels of Deep
  Neural Networks
Mehler's Formula, Branching Process, and Compositional Kernels of Deep Neural Networks
Tengyuan Liang
Hai Tran-Bach
48
11
0
09 Apr 2020
Analysis of Knowledge Transfer in Kernel Regime
Analysis of Knowledge Transfer in Kernel Regime
Arman Rahbar
Ashkan Panahi
Chiranjib Bhattacharyya
Devdatt Dubhashi
M. Chehreghani
68
3
0
30 Mar 2020
Memorizing Gaussians with no over-parameterizaion via gradient decent on
  neural networks
Memorizing Gaussians with no over-parameterizaion via gradient decent on neural networks
Amit Daniely
VLMMLT
43
15
0
28 Mar 2020
Piecewise linear activations substantially shape the loss surfaces of
  neural networks
Piecewise linear activations substantially shape the loss surfaces of neural networks
Fengxiang He
Bohan Wang
Dacheng Tao
ODL
91
30
0
27 Mar 2020
Volumization as a Natural Generalization of Weight Decay
Volumization as a Natural Generalization of Weight Decay
Liu Ziyin
Zihao Wang
M. Yamada
Masahito Ueda
AI4CE
20
0
0
25 Mar 2020
Symmetry & critical points for a model shallow neural network
Symmetry & critical points for a model shallow neural network
Yossi Arjevani
M. Field
107
13
0
23 Mar 2020
Neural Networks and Polynomial Regression. Demystifying the
  Overparametrization Phenomena
Neural Networks and Polynomial Regression. Demystifying the Overparametrization Phenomena
Matt Emschwiller
D. Gamarnik
Eren C. Kizildag
Ilias Zadik
77
9
0
23 Mar 2020
Critical Point-Finding Methods Reveal Gradient-Flat Regions of Deep
  Network Losses
Critical Point-Finding Methods Reveal Gradient-Flat Regions of Deep Network Losses
Charles G. Frye
James B. Simon
Neha S. Wadia
A. Ligeralde
M. DeWeese
K. Bouchard
ODL
55
2
0
23 Mar 2020
Steepest Descent Neural Architecture Optimization: Escaping Local
  Optimum with Signed Neural Splitting
Steepest Descent Neural Architecture Optimization: Escaping Local Optimum with Signed Neural Splitting
Lemeng Wu
Mao Ye
Qi Lei
Jason D. Lee
Qiang Liu
88
15
0
23 Mar 2020
On Alignment in Deep Linear Neural Networks
On Alignment in Deep Linear Neural Networks
Adityanarayanan Radhakrishnan
Eshaan Nichani
D. Bernstein
Caroline Uhler
47
2
0
13 Mar 2020
Frequency Bias in Neural Networks for Input of Non-Uniform Density
Frequency Bias in Neural Networks for Input of Non-Uniform Density
Ronen Basri
Meirav Galun
Amnon Geifman
David Jacobs
Yoni Kasten
S. Kritchman
92
186
0
10 Mar 2020
Neural Kernels Without Tangents
Neural Kernels Without Tangents
Vaishaal Shankar
Alex Fang
Wenshuo Guo
Sara Fridovich-Keil
Ludwig Schmidt
Jonathan Ragan-Kelley
Benjamin Recht
64
91
0
04 Mar 2020
Good Subnetworks Provably Exist: Pruning via Greedy Forward Selection
Good Subnetworks Provably Exist: Pruning via Greedy Forward Selection
Mao Ye
Chengyue Gong
Lizhen Nie
Denny Zhou
Adam R. Klivans
Qiang Liu
113
111
0
03 Mar 2020
Overall error analysis for the training of deep neural networks via
  stochastic gradient descent with random initialisation
Overall error analysis for the training of deep neural networks via stochastic gradient descent with random initialisation
Arnulf Jentzen
Timo Welti
65
17
0
03 Mar 2020
On the Global Convergence of Training Deep Linear ResNets
On the Global Convergence of Training Deep Linear ResNets
Difan Zou
Philip M. Long
Quanquan Gu
78
39
0
02 Mar 2020
Loss landscapes and optimization in over-parameterized non-linear
  systems and neural networks
Loss landscapes and optimization in over-parameterized non-linear systems and neural networks
Chaoyue Liu
Libin Zhu
M. Belkin
ODL
133
266
0
29 Feb 2020
Training BatchNorm and Only BatchNorm: On the Expressive Power of Random
  Features in CNNs
Training BatchNorm and Only BatchNorm: On the Expressive Power of Random Features in CNNs
Jonathan Frankle
D. Schwab
Ari S. Morcos
117
143
0
29 Feb 2020
Train Large, Then Compress: Rethinking Model Size for Efficient Training
  and Inference of Transformers
Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers
Zhuohan Li
Eric Wallace
Sheng Shen
Kevin Lin
Kurt Keutzer
Dan Klein
Joseph E. Gonzalez
138
151
0
26 Feb 2020
Convex Geometry and Duality of Over-parameterized Neural Networks
Convex Geometry and Duality of Over-parameterized Neural Networks
Tolga Ergen
Mert Pilanci
MLT
138
56
0
25 Feb 2020
Neural Networks are Convex Regularizers: Exact Polynomial-time Convex
  Optimization Formulations for Two-layer Networks
Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-layer Networks
Mert Pilanci
Tolga Ergen
101
118
0
24 Feb 2020
De-randomized PAC-Bayes Margin Bounds: Applications to Non-convex and
  Non-smooth Predictors
De-randomized PAC-Bayes Margin Bounds: Applications to Non-convex and Non-smooth Predictors
A. Banerjee
Tiancong Chen
Yingxue Zhou
BDL
86
8
0
23 Feb 2020
An Optimization and Generalization Analysis for Max-Pooling Networks
An Optimization and Generalization Analysis for Max-Pooling Networks
Alon Brutzkus
Amir Globerson
MLTAI4CE
46
4
0
22 Feb 2020
Revealing the Structure of Deep Neural Networks via Convex Duality
Revealing the Structure of Deep Neural Networks via Convex Duality
Tolga Ergen
Mert Pilanci
MLT
102
72
0
22 Feb 2020
Generalisation error in learning with random features and the hidden
  manifold model
Generalisation error in learning with random features and the hidden manifold model
Federica Gerace
Bruno Loureiro
Florent Krzakala
M. Mézard
Lenka Zdeborová
87
172
0
21 Feb 2020
Robust Pruning at Initialization
Robust Pruning at Initialization
Soufiane Hayou
Jean-François Ton
Arnaud Doucet
Yee Whye Teh
43
47
0
19 Feb 2020
Global Convergence of Deep Networks with One Wide Layer Followed by
  Pyramidal Topology
Global Convergence of Deep Networks with One Wide Layer Followed by Pyramidal Topology
Quynh N. Nguyen
Marco Mondelli
ODLAI4CE
83
70
0
18 Feb 2020
Learning Parities with Neural Networks
Learning Parities with Neural Networks
Amit Daniely
Eran Malach
104
78
0
18 Feb 2020
Over-parameterized Adversarial Training: An Analysis Overcoming the
  Curse of Dimensionality
Over-parameterized Adversarial Training: An Analysis Overcoming the Curse of Dimensionality
Yi Zhang
Orestis Plevrakis
S. Du
Xingguo Li
Zhao Song
Sanjeev Arora
127
53
0
16 Feb 2020
Previous
123...131415161718
Next