Gradient Descent Finds Global Minima of Deep Neural Networks

9 November 2018

Papers citing "Gradient Descent Finds Global Minima of Deep Neural Networks"

50 / 763 papers shown

Title
Exploring Ordinality in Text Classification: A Comparative Study of Explicit and Implicit Techniques Siva Rajesh Kasa Aniket Goel Karan Gupta Sumegh Roychowdhury Anish Bhanushali Nikhil Pattisapu Prasanna Srinivasa Murthy 41 1 0 20 May 2024
Approximation and Gradient Descent Training with Neural Networks G. Welper 38 1 0 19 May 2024
Cross-domain Learning Framework for Tracking Users in RIS-aided Multi-band ISAC Systems with Sparse Labeled Data Jingzhi Hu Dusit Niyato Jun Luo 19 2 0 10 May 2024
An Improved Finite-time Analysis of Temporal Difference Learning with Deep Neural Networks Zhifa Ke Zaiwen Wen Junyu Zhang 34 0 0 07 May 2024
Physics-Informed Neural Networks: Minimizing Residual Loss with Wide Networks and Effective Activations Nima Hosseini Dashtbayaz G. Farhani Boyu Wang Charles Ling 28 1 0 02 May 2024
Neural Dynamic Data Valuation Zhangyong Liang Huanhuan Gao Ji Zhang TDI 48 1 0 30 Apr 2024
Regularized Gauss-Newton for Optimizing Overparameterized Neural Networks Adeyemi Damilare Adeoye Philipp Christian Petersen Alberto Bemporad 28 1 0 23 Apr 2024
The Positivity of the Neural Tangent Kernel Luís Carvalho Joao L. Costa José Mourao Gonccalo Oliveira 28 1 0 19 Apr 2024
CAM-Based Methods Can See through Walls Magamed Taimeskhanov R. Sicre Damien Garreau 21 1 0 02 Apr 2024
Grounding and Enhancing Grid-based Models for Neural Fields Zelin Zhao Fenglei Fan Wenlong Liao Junchi Yan 39 5 0 29 Mar 2024
From Activation to Initialization: Scaling Insights for Optimizing Neural Fields Hemanth Saratchandran Sameera Ramasinghe Simon Lucey AI4CE 38 1 0 28 Mar 2024
Approximation with Random Shallow ReLU Networks with Applications to Model Reference Adaptive Control Andrew G. Lamperski Tyler Lekang 35 3 0 25 Mar 2024
Data Acquisition via Experimental Design for Decentralized Data Markets Charles Lu Baihe Huang Sai Praneeth Karimireddy Praneeth Vepakomma Michael I. Jordan Ramesh Raskar FedML OOD 36 2 0 20 Mar 2024
Robust NAS under adversarial training: benchmark, theory, and beyond Yongtao Wu Fanghui Liu Carl-Johann Simon-Gabriel Grigorios G. Chrysos V. Cevher AAML OOD 35 3 0 19 Mar 2024
Generalization of Scaled Deep ResNets in the Mean-Field Regime Yihang Chen Fanghui Liu Yiping Lu Grigorios G. Chrysos V. Cevher 41 2 0 14 Mar 2024
On the Diminishing Returns of Width for Continual Learning E. Guha V. Lakshman CLL 31 4 0 11 Mar 2024
Improve Generalization Ability of Deep Wide Residual Network with A Suitable Scaling Factor Songtao Tian Zixiong Yu 21 1 0 07 Mar 2024
Restricted Bayesian Neural Network Sourav Ganguly Saprativa Bhattacharjee UQCV BDL 23 0 0 06 Mar 2024
Investigating Deep Watermark Security: An Adversarial Transferability Perspective Biqing Qi Junqi Gao Yiang Luo Jianxing Liu Ligang Wu Bowen Zhou AAML 31 3 0 26 Feb 2024
Convergence of Gradient Descent for Recurrent Neural Networks: A Nonasymptotic Analysis Semih Cayci A. Eryilmaz 26 3 0 19 Feb 2024
LoRA Training in the NTK Regime has No Spurious Local Minima Uijeong Jang Jason D. Lee Ernest K. Ryu 44 14 0 19 Feb 2024
Non-convergence to global minimizers for Adam and stochastic gradient descent optimization and constructions of local minimizers in the training of artificial neural networks Arnulf Jentzen Adrian Riekert 38 4 0 07 Feb 2024
Analyzing the Neural Tangent Kernel of Periodically Activated Coordinate Networks Hemanth Saratchandran Shin-Fang Chng Simon Lucey 21 2 0 07 Feb 2024
The Challenges of the Nonlinear Regime for Physics-Informed Neural Networks Andrea Bonfanti Giuseppe Bruno Cristina Cipriani 32 7 0 06 Feb 2024
Convex Relaxations of ReLU Neural Networks Approximate Global Optima in Polynomial Time Sungyoon Kim Mert Pilanci 46 4 0 06 Feb 2024
Non-asymptotic Analysis of Biased Adaptive Stochastic Approximation Sobihan Surendran Antoine Godichon-Baggioni Adeline Fermanian Sylvain Le Corff 37 1 0 05 Feb 2024
Understanding the Generalization Benefits of Late Learning Rate Decay Yinuo Ren Chao Ma Lexing Ying AI4CE 32 6 0 21 Jan 2024
A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Models Namjoon Suh Guang Cheng MedIm 30 12 0 14 Jan 2024
Understanding Representation Learnability of Nonlinear Self-Supervised Learning Ruofeng Yang Xiangyuan Li Bo Jiang Shuai Li SSL MLT 53 2 0 06 Jan 2024
A Mathematical Guide to Operator Learning Nicolas Boullé Alex Townsend 34 36 0 22 Dec 2023
A note on regularised NTK dynamics with an application to PAC-Bayesian training Eugenio Clerico Benjamin Guedj 33 0 0 20 Dec 2023
Factor-Assisted Federated Learning for Personalized Optimization with Heterogeneous Data Feifei Wang Huiyun Tang Yang Li FedML 27 0 0 07 Dec 2023
Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking Kaifeng Lyu Jikai Jin Zhiyuan Li Simon S. Du Jason D. Lee Wei Hu AI4CE 41 32 0 30 Nov 2023
Critical Influence of Overparameterization on Sharpness-aware Minimization Sungbin Shin Dongyeop Lee Maksym Andriushchenko Namhoon Lee AAML 44 1 0 29 Nov 2023
Weight fluctuations in (deep) linear neural networks and a derivation of the inverse-variance flatness relation Markus Gross A. Raulf Christoph Räth 43 0 0 23 Nov 2023
Distributed Constrained Combinatorial Optimization leveraging Hypergraph Neural Networks Nasimeh Heydaribeni Xinrui Zhan Ruisi Zhang Tina Eliassi-Rad F. Koushanfar AI4CE 32 8 0 15 Nov 2023
Stacked networks improve physics-informed training: applications to neural networks and deep operator networks Amanda A. Howard Sarah H. Murphy Shady E. Ahmed P. Stinis AI4CE 55 18 0 11 Nov 2023
On the Convergence of Encoder-only Shallow Transformers Yongtao Wu Fanghui Liu Grigorios G. Chrysos V. Cevher 44 5 0 02 Nov 2023
Initialization Matters: Privacy-Utility Analysis of Overparameterized Neural Networks Jiayuan Ye Zhenyu Zhu Fanghui Liu Reza Shokri V. Cevher 32 12 0 31 Oct 2023
Sample Complexity Bounds for Score-Matching: Causal Discovery and Generative Modeling Zhenyu Zhu Francesco Locatello V. Cevher 40 6 0 27 Oct 2023
A qualitative difference between gradient flows of convex functions in finite- and infinite-dimensional Hilbert spaces Jonathan W. Siegel Stephan Wojtowytsch 13 3 0 26 Oct 2023
On the Optimization and Generalization of Multi-head Attention Puneesh Deora Rouzbeh Ghaderi Hossein Taheri Christos Thrampoulidis MLT 47 33 0 19 Oct 2023
Differential Equation Scaling Limits of Shaped and Unshaped Neural Networks Mufan Bill Li Mihai Nica 22 2 0 18 Oct 2023
Wide Neural Networks as Gaussian Processes: Lessons from Deep Equilibrium Models Tianxiang Gao Xiaokai Huo Hailiang Liu Hongyang Gao BDL 17 8 0 16 Oct 2023
How Graph Neural Networks Learn: Lessons from Training Dynamics Chenxiao Yang Qitian Wu David Wipf Ruoyu Sun Junchi Yan AI4CE GNN 22 1 0 08 Oct 2023
Learning A Disentangling Representation For PU Learning Omar Zamzam H. Akrami Mahdi Soltanolkotabi Richard M. Leahy 17 0 0 05 Oct 2023
Sparse Deep Learning for Time Series Data: Theory and Applications Mingxuan Zhang Y. Sun Faming Liang AI4TS OOD BDL 39 2 0 05 Oct 2023
A Large-Scale 3D Face Mesh Video Dataset via Neural Re-parameterized Optimization Youwang Kim Lee Hyun Kim Sung-Bin Suekyeong Nam Janghoon Ju Tae-Hyun Oh CVBM 3DH 21 3 0 04 Oct 2023
JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention Yuandong Tian Yiping Wang Zhenyu (Allen) Zhang Beidi Chen Simon S. Du 29 35 0 01 Oct 2023
Global Convergence of SGD For Logistic Loss on Two Layer Neural Nets Pulkit Gopalani Samyak Jha Anirbit Mukherjee 19 2 0 17 Sep 2023