Gradient Descent Provably Optimizes Over-parameterized Neural Networks

4 October 2018

Aarti Singh

Papers citing "Gradient Descent Provably Optimizes Over-parameterized Neural Networks"

50 / 302 papers shown

Title
Dynamic Programming in Rank Space: Scaling Structured Inference with Low-Rank HMMs and PCFGs Aaron Courville Wei Liu Kewei Tu 21 8 0 01 May 2022
Beyond the Quadratic Approximation: the Multiscale Structure of Neural Network Loss Landscapes Chao Ma D. Kunin Lei Wu Lexing Ying 25 27 0 24 Apr 2022
Theory of Graph Neural Networks: Representation and Learning Stefanie Jegelka GNN AI4CE 33 68 0 16 Apr 2022
On Convergence Lemma and Convergence Stability for Piecewise Analytic Functions Xiaotie Deng Hanyu Li Ningyuan Li 15 0 0 04 Apr 2022
Convergence of gradient descent for deep neural networks S. Chatterjee ODL 21 20 0 30 Mar 2022
Random matrix analysis of deep neural network weight matrices M. Thamm Max Staats B. Rosenow 35 12 0 28 Mar 2022
On the (Non-)Robustness of Two-Layer Neural Networks in Different Learning Regimes Elvis Dohmatob A. Bietti AAML 35 13 0 22 Mar 2022
The Spectral Bias of Polynomial Neural Networks Moulik Choraria L. Dadi Grigorios G. Chrysos Julien Mairal V. Cevher 24 18 0 27 Feb 2022
Sparse Neural Additive Model: Interpretable Deep Learning with Feature Selection via Group Sparsity Shiyun Xu Zhiqi Bu Pratik Chaudhari Ian Barnett 27 21 0 25 Feb 2022
Benefit of Interpolation in Nearest Neighbor Algorithms Yue Xing Qifan Song Guang Cheng 14 28 0 23 Feb 2022
Random Feature Amplification: Feature Learning and Generalization in Neural Networks Spencer Frei Niladri S. Chatterji Peter L. Bartlett MLT 30 29 0 15 Feb 2022
Noise Regularizes Over-parameterized Rank One Matrix Recovery, Provably Tianyi Liu Yan Li Enlu Zhou Tuo Zhao 38 1 0 07 Feb 2022
Demystify Optimization and Generalization of Over-parameterized PAC-Bayesian Learning Wei Huang Chunrui Liu Yilan Chen Tianyu Liu R. Xu BDL MLT 19 2 0 04 Feb 2022
Improved Overparametrization Bounds for Global Convergence of Stochastic Gradient Descent for Shallow Neural Networks Bartlomiej Polaczyk J. Cyranka ODL 33 3 0 28 Jan 2022
How does unlabeled data improve generalization in self-training? A one-hidden-layer theoretical analysis Shuai Zhang Hao Wu Sijia Liu Pin-Yu Chen Jinjun Xiong SSL MLT 41 22 0 21 Jan 2022
Largest Eigenvalues of the Conjugate Kernel of Single-Layered Neural Networks L. Benigni Sandrine Péché 42 8 0 13 Jan 2022
Implicit Bias of MSE Gradient Optimization in Underparameterized Neural Networks Benjamin Bowman Guido Montúfar 28 11 0 12 Jan 2022
A Theoretical View of Linear Backpropagation and Its Convergence Ziang Li Yiwen Guo Haodi Liu Changshui Zhang AAML 18 3 0 21 Dec 2021
Rethinking Influence Functions of Neural Networks in the Over-parameterized Regime Rui Zhang Shihua Zhang TDI 27 21 0 15 Dec 2021
Training Multi-Layer Over-Parametrized Neural Network in Subquadratic Time Zhao Song Licheng Zhang Ruizhe Zhang 26 64 0 14 Dec 2021
SCORE: Approximating Curvature Information under Self-Concordant Regularization Adeyemi Damilare Adeoye Alberto Bemporad 20 4 0 14 Dec 2021
Provable Continual Learning via Sketched Jacobian Approximations Reinhard Heckel CLL 20 9 0 09 Dec 2021
Understanding Square Loss in Training Overparametrized Neural Network Classifiers Tianyang Hu Jun Wang Wei Cao Zhenguo Li UQCV AAML 41 19 0 07 Dec 2021
On the Convergence of Shallow Neural Network Training with Randomly Masked Neurons Fangshuo Liao Anastasios Kyrillidis 46 16 0 05 Dec 2021
Fast Graph Neural Tangent Kernel via Kronecker Sketching Shunhua Jiang Yunze Man Zhao Song Zheng Yu Danyang Zhuo 29 6 0 04 Dec 2021
Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models Tri Dao Beidi Chen Kaizhao Liang Jiaming Yang Zhao Song Atri Rudra Christopher Ré 33 75 0 30 Nov 2021
Offline Neural Contextual Bandits: Pessimism, Optimization and Generalization Thanh Nguyen-Tang Sunil R. Gupta A. Nguyen Svetha Venkatesh OffRL 29 28 0 27 Nov 2021
Learning with convolution and pooling operations in kernel methods Theodor Misiakiewicz Song Mei MLT 15 29 0 16 Nov 2021
On the Equivalence between Neural Network and Support Vector Machine Yilan Chen Wei Huang Lam M. Nguyen Tsui-Wei Weng AAML 25 18 0 11 Nov 2021
Subquadratic Overparameterization for Shallow Neural Networks Chaehwan Song Ali Ramezani-Kebrya Thomas Pethick Armin Eftekhari V. Cevher 30 31 0 02 Nov 2021
Quantifying Epistemic Uncertainty in Deep Learning Ziyi Huang H. Lam Haofeng Zhang UQCV BDL UD PER 24 12 0 23 Oct 2021
Pro-KD: Progressive Distillation by Following the Footsteps of the Teacher Mehdi Rezagholizadeh A. Jafari Puneeth Salad Pranav Sharma Ali Saheb Pasand A. Ghodsi 79 18 0 16 Oct 2021
Provable Regret Bounds for Deep Online Learning and Control Xinyi Chen Edgar Minasyan Jason D. Lee Elad Hazan 36 6 0 15 Oct 2021
AIR-Net: Adaptive and Implicit Regularization Neural Network for Matrix Completion Zhemin Li Tao Sun Hongxia Wang Bao Wang 50 6 0 12 Oct 2021
Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Pruned Neural Networks Shuai Zhang Meng Wang Sijia Liu Pin-Yu Chen Jinjun Xiong UQCV MLT 31 13 0 12 Oct 2021
Imitating Deep Learning Dynamics via Locally Elastic Stochastic Differential Equations Jiayao Zhang Hua Wang Weijie J. Su 35 7 0 11 Oct 2021
Deep Bayesian inference for seismic imaging with tasks Ali Siahkoohi G. Rizzuti Felix J. Herrmann BDL UQCV 40 21 0 10 Oct 2021
Does Preprocessing Help Training Over-parameterized Neural Networks? Zhao Song Shuo Yang Ruizhe Zhang 38 49 0 09 Oct 2021
New Insights into Graph Convolutional Networks using Neural Tangent Kernels Mahalakshmi Sabanayagam P. Esser D. Ghoshdastidar 26 6 0 08 Oct 2021
On the Global Convergence of Gradient Descent for multi-layer ResNets in the mean-field regime Zhiyan Ding Shi Chen Qin Li S. Wright MLT AI4CE 41 11 0 06 Oct 2021
Efficient and Private Federated Learning with Partially Trainable Networks Hakim Sidahmed Zheng Xu Ankush Garg Yuan Cao Mingqing Chen FedML 49 13 0 06 Oct 2021
The Role of Lookahead and Approximate Policy Evaluation in Reinforcement Learning with Linear Value Function Approximation Anna Winnicki Joseph Lubars Michael Livesay R. Srikant 31 3 0 28 Sep 2021
Theory of overparametrization in quantum neural networks Martín Larocca Nathan Ju Diego García-Martín Patrick J. Coles M. Cerezo 43 188 0 23 Sep 2021
Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networks Zhichao Wang Yizhe Zhu 35 18 0 20 Sep 2021
NASI: Label- and Data-agnostic Neural Architecture Search at Initialization Yao Shu Shaofeng Cai Zhongxiang Dai Beng Chin Ooi K. H. Low 22 43 0 02 Sep 2021
Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularization Difan Zou Yuan Cao Yuanzhi Li Quanquan Gu MLT AI4CE 47 38 0 25 Aug 2021
Fast Sketching of Polynomial Kernels of Polynomial Degree Zhao Song David P. Woodruff Zheng Yu Lichen Zhang 21 40 0 21 Aug 2021
A proof of convergence for the gradient descent optimization method with random initializations in the training of neural networks with ReLU activation for piecewise linear target functions Arnulf Jentzen Adrian Riekert 33 13 0 10 Aug 2021
Towards General Function Approximation in Zero-Sum Markov Games Baihe Huang Jason D. Lee Zhaoran Wang Zhuoran Yang 33 47 0 30 Jul 2021
Convergence analysis for gradient flows in the training of artificial neural networks with ReLU activation Arnulf Jentzen Adrian Riekert 27 23 0 09 Jul 2021