Gradient Descent Finds Global Minima of Deep Neural Networks

9 November 2018

Papers citing "Gradient Descent Finds Global Minima of Deep Neural Networks"

50 / 763 papers shown

Title
Do Residual Neural Networks discretize Neural Ordinary Differential Equations? Michael E. Sander Pierre Ablin Gabriel Peyré 35 25 0 29 May 2022
Global Convergence of Over-parameterized Deep Equilibrium Models Zenan Ling Xingyu Xie Qiuhao Wang Zongpeng Zhang Zhouchen Lin 32 12 0 27 May 2022
Generalization Bounds for Gradient Methods via Discrete and Continuous Prior Jun Yu Li Xu Luo Jian Li 24 4 0 27 May 2022
Quadratic models for understanding catapult dynamics of neural networks Libin Zhu Chaoyue Liu Adityanarayanan Radhakrishnan M. Belkin 35 13 0 24 May 2022
Transition to Linearity of General Neural Networks with Directed Acyclic Graph Architecture Libin Zhu Chaoyue Liu M. Belkin GNN AI4CE 20 4 0 24 May 2022
Gaussian Pre-Activations in Neural Networks: Myth or Reality? Pierre Wolinski Julyan Arbel AI4CE 70 8 0 24 May 2022
MetaSlicing: A Novel Resource Allocation Framework for Metaverse N. Chu D. Hoang Diep N. Nguyen Khoa T. Phan E. Dutkiewicz Dusist Niyato Tao Shu 36 46 0 23 May 2022
Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterization Simone Bombari Mohammad Hossein Amani Marco Mondelli 30 26 0 20 May 2022
Mean-Field Analysis of Two-Layer Neural Networks: Global Optimality with Linear Convergence Rates Jingwei Zhang Xunpeng Huang Jincheng Yu MLT 18 1 0 19 May 2022
Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks Blake Bordelon C. Pehlevan MLT 34 78 0 19 May 2022
On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias Itay Safran Gal Vardi Jason D. Lee MLT 59 23 0 18 May 2022
Gradient Descent Optimizes Infinite-Depth ReLU Implicit Networks with Linear Widths Tianxiang Gao Hongyang Gao MLT 35 5 0 16 May 2022
Policy Gradient Method For Robust Reinforcement Learning Yue Wang Shaofeng Zou 81 67 0 15 May 2022
Sharp Asymptotics of Kernel Ridge Regression Beyond the Linear Regime Hong Hu Yue M. Lu 53 15 0 13 May 2022
Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysis Wuyang Chen Wei Huang Xinyu Gong Boris Hanin Zhangyang Wang 30 7 0 11 May 2022
Analysis of convolutional neural network image classifiers in a rotationally symmetric model Michael Kohler Benjamin Kohler 17 5 0 11 May 2022
U-NO: U-shaped Neural Operators Md Ashiqur Rahman Zachary E. Ross Kamyar Azizzadenesheli AI4CE 35 132 0 23 Apr 2022
On Feature Learning in Neural Networks with Global Convergence Guarantees Zhengdao Chen Eric Vanden-Eijnden Joan Bruna MLT 36 13 0 22 Apr 2022
Theory of Graph Neural Networks: Representation and Learning Stefanie Jegelka GNN AI4CE 33 68 0 16 Apr 2022
Convergence and Implicit Regularization Properties of Gradient Descent for Deep Residual Networks R. Cont Alain Rossier Renyuan Xu MLT 26 6 0 14 Apr 2022
Overparameterized Linear Regression under Adversarial Attacks Antônio H. Ribeiro Thomas B. Schon AAML 14 18 0 13 Apr 2022
On Convergence Lemma and Convergence Stability for Piecewise Analytic Functions Xiaotie Deng Hanyu Li Ningyuan Li 15 0 0 04 Apr 2022
$Training Fully Connected Neural Networks is $\exists\mathbb{R}$-Complete$ Training Fully Connected Neural Networks is $\exists\mathbb{R}$ -Complete Daniel Bertschinger Christoph Hertrich Paul Jungeblut Tillmann Miltzow Simon Weber OffRL 59 30 0 04 Apr 2022
Convergence of gradient descent for deep neural networks S. Chatterjee ODL 21 20 0 30 Mar 2022
Random matrix analysis of deep neural network weight matrices M. Thamm Max Staats B. Rosenow 32 12 0 28 Mar 2022
Demystifying the Neural Tangent Kernel from a Practical Perspective: Can it be trusted for Neural Architecture Search without training? J. Mok Byunggook Na Ji-Hoon Kim Dongyoon Han Sungroh Yoon AAML 42 23 0 28 Mar 2022
On the Neural Tangent Kernel Analysis of Randomly Pruned Neural Networks Hongru Yang Zhangyang Wang MLT 27 8 0 27 Mar 2022
On the (Non-)Robustness of Two-Layer Neural Networks in Different Learning Regimes Elvis Dohmatob A. Bietti AAML 32 13 0 22 Mar 2022
On the Generalization Mystery in Deep Learning S. Chatterjee Piotr Zielinski OOD 20 33 0 18 Mar 2022
On the Spectral Bias of Convolutional Neural Tangent and Gaussian Process Kernels Amnon Geifman Meirav Galun David Jacobs Ronen Basri 30 13 0 17 Mar 2022
Global Convergence of MAML and Theory-Inspired Neural Architecture Search for Few-Shot Learning Haoxiang Wang Yite Wang Ruoyu Sun Bo-wen Li 29 27 0 17 Mar 2022
On the Convergence of Certified Robust Training with Interval Bound Propagation Yihan Wang Zhouxing Shi Quanquan Gu Cho-Jui Hsieh 25 9 0 16 Mar 2022
Towards understanding deep learning with the natural clustering prior Simon Carbonnelle 15 0 0 15 Mar 2022
Phenomenology of Double Descent in Finite-Width Neural Networks Sidak Pal Singh Aurelien Lucchi Thomas Hofmann Bernhard Schölkopf 29 10 0 14 Mar 2022
Deep Regression Ensembles Antoine Didisheim Bryan Kelly Semyon Malamud UQCV 9 4 0 10 Mar 2022
Transition to Linearity of Wide Neural Networks is an Emerging Property of Assembling Weak Models Chaoyue Liu Libin Zhu M. Belkin 12 4 0 10 Mar 2022
Benefit of Interpolation in Nearest Neighbor Algorithms Yue Xing Qifan Song Guang Cheng 9 28 0 23 Feb 2022
Reconstruction Task Finds Universal Winning Tickets Ruichen Li Binghui Li Qi Qian Liwei Wang 18 0 0 23 Feb 2022
Theoretical Analysis of Deep Neural Networks in Physical Layer Communication Xiaozhong Liu Haitao Zhao Dongtang Ma Kai Mei Jibo Wei 29 4 0 21 Feb 2022
An alternative approach to train neural networks using monotone variational inequality Chen Xu Xiuyuan Cheng Yao Xie 11 1 0 17 Feb 2022
Universality of empirical risk minimization Andrea Montanari Basil Saeed OOD 25 74 0 17 Feb 2022
Benign Overfitting in Two-layer Convolutional Neural Networks Yuan Cao Zixiang Chen M. Belkin Quanquan Gu MLT 19 83 0 14 Feb 2022
Learning from Randomly Initialized Neural Network Features Ehsan Amid Rohan Anil W. Kotłowski Manfred K. Warmuth MLT 6 14 0 13 Feb 2022
Predicting Out-of-Distribution Error with the Projection Norm Yaodong Yu Zitong Yang Alexander Wei Yi Ma Jacob Steinhardt OODD 12 43 0 11 Feb 2022
Deep Learning in Random Neural Fields: Numerical Experiments via Neural Tangent Kernel Kaito Watanabe Kotaro Sakamoto Ryo Karakida Sho Sonoda S. Amari OOD 19 1 0 10 Feb 2022
Finite-Sum Optimization: A New Perspective for Convergence to a Global Solution Lam M. Nguyen Trang H. Tran Marten van Dijk 38 3 0 07 Feb 2022
Demystify Optimization and Generalization of Over-parameterized PAC-Bayesian Learning Wei Huang Chunrui Liu Yilan Chen Tianyu Liu R. Xu BDL MLT 19 2 0 04 Feb 2022
Faster Convergence of Local SGD for Over-Parameterized Models Tiancheng Qin S. Rasoul Etesami César A. Uribe FedML 35 6 0 30 Jan 2022
Understanding Why Generalized Reweighting Does Not Improve Over ERM Runtian Zhai Chen Dan Zico Kolter Pradeep Ravikumar OOD 44 27 0 28 Jan 2022
Convergence of Invariant Graph Networks Chen Cai Yusu Wang 60 4 0 25 Jan 2022