On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport

24 May 2018

Papers citing "On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport"

50 / 483 papers shown

Title
Learning and generalization of one-hidden-layer neural networks, going beyond standard Gaussian data Hongkang Li Shuai Zhang Hao Wu MLT 21 8 0 07 Jul 2022
Neural Networks can Learn Representations with Gradient Descent Alexandru Damian Jason D. Lee Mahdi Soltanolkotabi SSL MLT 22 114 0 30 Jun 2022
Label noise (stochastic) gradient descent implicitly solves the Lasso for quadratic parametrisation Loucas Pillaud-Vivien J. Reygner Nicolas Flammarion NoLa 33 31 0 20 Jun 2022
Mirror Descent with Relative Smoothness in Measure Spaces, with application to Sinkhorn and EM Pierre-Cyril Aubin-Frankowski Anna Korba F. Léger 18 28 0 17 Jun 2022
The Manifold Hypothesis for Gradient-Based Explanations Sebastian Bordt Uddeshya Upadhyay Zeynep Akata U. V. Luxburg FAtt AAML 28 12 0 15 Jun 2022
Unbiased Estimation using Underdamped Langevin Dynamics Hamza Ruzayqat Neil K. Chada Ajay Jasra 33 4 0 14 Jun 2022
Parameter Convex Neural Networks Jingcheng Zhou Wei Wei Xing Li Bowen Pang Zhiming Zheng 6 0 0 11 Jun 2022
Explicit Regularization in Overparametrized Models via Noise Injection Antonio Orvieto Anant Raj Hans Kersting Francis R. Bach 10 26 0 09 Jun 2022
Adversarial Noises Are Linearly Separable for (Nearly) Random Neural Networks Huishuai Zhang Da Yu Yiping Lu Di He AAML 27 1 0 09 Jun 2022
Neural Collapse: A Review on Modelling Principles and Generalization Vignesh Kothapalli 23 71 0 08 Jun 2022
High-dimensional limit theorems for SGD: Effective dynamics and critical scaling Gerard Ben Arous Reza Gheissari Aukosh Jagannath 62 58 0 08 Jun 2022
The Neural Covariance SDE: Shaped Infinite Depth-and-Width Networks at Initialization Mufan Li Mihai Nica Daniel M. Roy 38 36 0 06 Jun 2022
Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs Etienne Boursier Loucas Pillaud-Vivien Nicolas Flammarion ODL 24 58 0 02 Jun 2022
Self-Consistency of the Fokker-Planck Equation Zebang Shen Zhenfu Wang Satyen Kale Alejandro Ribeiro Aim Karbasi Hamed Hassani 18 17 0 02 Jun 2022
Universality of Group Convolutional Neural Networks Based on Ridgelet Analysis on Groups Sho Sonoda Isao Ishikawa Masahiro Ikeda 30 9 0 30 May 2022
Excess Risk of Two-Layer ReLU Neural Networks in Teacher-Student Settings and its Superiority to Kernel Methods Shunta Akiyama Taiji Suzuki 30 6 0 30 May 2022
On Bridging the Gap between Mean Field and Finite Width in Deep Random Neural Networks with Batch Normalization Amir Joudaki Hadi Daneshmand Francis R. Bach AI4CE 19 2 0 25 May 2022
Distributional Hamilton-Jacobi-Bellman Equations for Continuous-Time Reinforcement Learning Harley Wiltzer D. Meger Marc G. Bellemare 19 12 0 24 May 2022
Empirical Phase Diagram for Three-layer Neural Networks with Infinite Width Hanxu Zhou Qixuan Zhou Zhenyuan Jin Tao Luo Yaoyu Zhang Zhi-Qin John Xu 25 20 0 24 May 2022
Mean-Field Analysis of Two-Layer Neural Networks: Global Optimality with Linear Convergence Rates Jingwei Zhang Xunpeng Huang Jincheng Yu MLT 18 1 0 19 May 2022
Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks Blake Bordelon Cengiz Pehlevan MLT 40 78 0 19 May 2022
Sharp asymptotics on the compression of two-layer neural networks Mohammad Hossein Amani Simone Bombari Marco Mondelli Rattana Pukdee Stefano Rini MLT 24 0 0 17 May 2022
Mean-Field Nonparametric Estimation of Interacting Particle Systems Rentian Yao Xiaohui Chen Yun Yang 43 9 0 16 May 2022
High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation Jimmy Ba Murat A. Erdogdu Taiji Suzuki Zhichao Wang Denny Wu Greg Yang MLT 40 121 0 03 May 2022
On Feature Learning in Neural Networks with Global Convergence Guarantees Zhengdao Chen Eric Vanden-Eijnden Joan Bruna MLT 36 13 0 22 Apr 2022
Polynomial-time Sparse Measure Recovery: From Mean Field Theory to Algorithm Design Hadi Daneshmand Francis R. Bach 10 1 0 16 Apr 2022
Overparameterized Linear Regression under Adversarial Attacks Antônio H. Ribeiro Thomas B. Schon AAML 14 18 0 13 Apr 2022
Convergence of gradient descent for deep neural networks S. Chatterjee ODL 21 20 0 30 Mar 2022
Estimation of high dimensional Gamma convolutions through random projections Oskar Laverny 11 1 0 25 Mar 2022
On the (Non-)Robustness of Two-Layer Neural Networks in Different Learning Regimes Elvis Dohmatob A. Bietti AAML 32 13 0 22 Mar 2022
Deep Regression Ensembles Antoine Didisheim Bryan Kelly Semyon Malamud UQCV 9 4 0 10 Mar 2022
Fully-Connected Network on Noncompact Symmetric Space and Ridgelet Transform based on Helgason-Fourier Analysis Sho Sonoda Isao Ishikawa Masahiro Ikeda 21 15 0 03 Mar 2022
Explicitising The Implicit Intrepretability of Deep Neural Networks Via Duality Chandrashekar Lakshminarayanan Ashutosh Kumar Singh A. Rajkumar AI4CE 26 1 0 01 Mar 2022
A blob method for inhomogeneous diffusion with applications to multi-agent control and sampling Katy Craig Karthik Elamvazhuthi M. Haberland O. Turanova 35 15 0 25 Feb 2022
A Distributed Algorithm for Measure-valued Optimization with Additive Objective Iman Nodozi A. Halder 15 1 0 17 Feb 2022
Provably convergent quasistatic dynamics for mean-field two-player zero-sum games Chao Ma Lexing Ying MLT 32 11 0 15 Feb 2022
Random Feature Amplification: Feature Learning and Generalization in Neural Networks Spencer Frei Niladri S. Chatterji Peter L. Bartlett MLT 30 29 0 15 Feb 2022
Simultaneous Transport Evolution for Minimax Equilibria on Measures Carles Domingo-Enrich Joan Bruna 18 3 0 14 Feb 2022
Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks R. Veiga Ludovic Stephan Bruno Loureiro Florent Krzakala Lenka Zdeborová MLT 10 31 0 01 Feb 2022
Improved Overparametrization Bounds for Global Convergence of Stochastic Gradient Descent for Shallow Neural Networks Bartlomiej Polaczyk J. Cyranka ODL 33 3 0 28 Jan 2022
Convex Analysis of the Mean Field Langevin Dynamics Atsushi Nitanda Denny Wu Taiji Suzuki MLT 68 64 0 25 Jan 2022
Overview frequency principle/spectral bias in deep learning Z. Xu Yaoyu Zhang Tao Luo FaML 33 66 0 19 Jan 2022
Convergence of Policy Gradient for Entropy Regularized MDPs with Neural Network Approximation in the Mean-Field Regime B. Kerimkulov J. Leahy David Siska Lukasz Szpruch 30 11 0 18 Jan 2022
Complexity from Adaptive-Symmetries Breaking: Global Minima in the Statistical Mechanics of Deep Neural Networks Shaun Li AI4CE 43 0 0 03 Jan 2022
How Infinitely Wide Neural Networks Can Benefit from Multi-task Learning -- an Exact Macroscopic Characterization Jakob Heiss Josef Teichmann Hanna Wutte MLT 10 2 0 31 Dec 2021
Wasserstein Flow Meets Replicator Dynamics: A Mean-Field Analysis of Representation Learning in Actor-Critic Yufeng Zhang Siyu Chen Zhuoran Yang Michael I. Jordan Zhaoran Wang 68 4 0 27 Dec 2021
Global convergence of ResNets: From finite to infinite width using linear parameterization Raphael Barboni Gabriel Peyré Franccois-Xavier Vialard 16 12 0 10 Dec 2021
Asymptotic properties of one-layer artificial neural networks with sparse connectivity Christian Hirsch Matthias Neumann Volker Schmidt 21 1 0 01 Dec 2021
Convergence of GANs Training: A Game and Stochastic Control Methodology Othmane Mounjid Xin Guo GAN 22 2 0 01 Dec 2021
Embedding Principle: a hierarchical structure of loss landscape of deep neural networks Yaoyu Zhang Yuqing Li Zhongwang Zhang Tao Luo Z. Xu 29 21 0 30 Nov 2021