On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport

24 May 2018

Papers citing "On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport"

50 / 483 papers shown

Title
Soft Mode in the Dynamics of Over-realizable On-line Learning for Soft Committee Machines Frederieke Richert Roman Worschech B. Rosenow 11 5 0 29 Apr 2021
A Class of Dimension-free Metrics for the Convergence of Empirical Measures Jiequn Han Ruimeng Hu Jihao Long 23 3 0 24 Apr 2021
On Energy-Based Models with Overparametrized Shallow Neural Networks Carles Domingo-Enrich A. Bietti Eric Vanden-Eijnden Joan Bruna BDL 33 9 0 15 Apr 2021
A Recipe for Global Convergence Guarantee in Deep Neural Networks Kenji Kawaguchi Qingyun Sun 16 11 0 12 Apr 2021
Noether: The More Things Change, the More Stay the Same Grzegorz Gluch R. Urbanke 16 17 0 12 Apr 2021
A proof of convergence for stochastic gradient descent in the training of artificial neural networks with ReLU activation for constant target functions Arnulf Jentzen Adrian Riekert MLT 32 13 0 01 Apr 2021
Why Do Local Methods Solve Nonconvex Problems? Tengyu Ma 18 13 0 24 Mar 2021
Weighted Neural Tangent Kernel: A Generalized and Improved Network-Induced Kernel Lei Tan Shutong Wu Xiaolin Huang 21 1 0 22 Mar 2021
Full Gradient DQN Reinforcement Learning: A Provably Convergent Scheme Konstantin Avrachenkov Vivek Borkar H. Dolhare K. Patil 27 9 0 10 Mar 2021
Unintended Effects on Adaptive Learning Rate for Training Neural Network with Output Scale Change Ryuichi Kanoh M. Sugiyama 8 0 0 05 Mar 2021
Sparsity in long-time control of neural ODEs C. Yagüe Borjan Geshkovski 8 8 0 26 Feb 2021
Do Input Gradients Highlight Discriminative Features? Harshay Shah Prateek Jain Praneeth Netrapalli AAML FAtt 21 57 0 25 Feb 2021
Convergence rates for gradient descent in the training of overparameterized artificial neural networks with biases Arnulf Jentzen T. Kröger ODL 28 7 0 23 Feb 2021
Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeed Maria Refinetti Sebastian Goldt Florent Krzakala Lenka Zdeborová 22 72 0 23 Feb 2021
A proof of convergence for gradient descent in the training of artificial neural networks for constant target functions Patrick Cheridito Arnulf Jentzen Adrian Riekert Florian Rossmannek 28 24 0 19 Feb 2021
WGAN with an Infinitely Wide Generator Has No Spurious Stationary Points Albert No Taeho Yoon Sehyun Kwon Ernest K. Ryu GAN 19 2 0 15 Feb 2021
A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Network Mo Zhou Rong Ge Chi Jin 74 44 0 04 Feb 2021
On the Proof of Global Convergence of Gradient Descent for Deep ReLU Networks with Linear Widths Quynh N. Nguyen 41 49 0 24 Jan 2021
Learning with Gradient Descent and Weakly Convex Losses Dominic Richards Michael G. Rabbat MLT 27 13 0 13 Jan 2021
Towards Understanding Learning in Neural Networks with Linear Teachers Roei Sarussi Alon Brutzkus Amir Globerson FedML MLT 55 20 0 07 Jan 2021
A Priori Generalization Analysis of the Deep Ritz Method for Solving High Dimensional Elliptic Equations Jianfeng Lu Yulong Lu Min Wang 36 37 0 05 Jan 2021
Particle Dual Averaging: Optimization of Mean Field Neural Networks with Global Convergence Rate Analysis Atsushi Nitanda Denny Wu Taiji Suzuki 8 29 0 31 Dec 2020
Perspective: A Phase Diagram for Deep Learning unifying Jamming, Feature Learning and Lazy Training Mario Geiger Leonardo Petrini M. Wyart DRL 25 11 0 30 Dec 2020
Mathematical Models of Overparameterized Neural Networks Cong Fang Hanze Dong Tong Zhang 27 22 0 27 Dec 2020
Recent advances in deep learning theory Fengxiang He Dacheng Tao AI4CE 24 50 0 20 Dec 2020
On the emergence of simplex symmetry in the final and penultimate layers of neural network classifiers E. Weinan Stephan Wojtowytsch 28 42 0 10 Dec 2020
Feature Learning in Infinite-Width Neural Networks Greg Yang J. E. Hu MLT 9 147 0 30 Nov 2020
Generalization and Memorization: The Bias Potential Model Hongkang Yang E. Weinan 25 11 0 29 Nov 2020
Implicit bias of deep linear networks in the large learning rate phase Wei Huang Weitao Du R. Xu Chunrui Liu 24 2 0 25 Nov 2020
Align, then memorise: the dynamics of learning with feedback alignment Maria Refinetti Stéphane dÁscoli Ruben Ohana Sebastian Goldt 26 36 0 24 Nov 2020
Neural collapse with unconstrained features D. Mixon Hans Parshall Jianzong Pi 19 114 0 23 Nov 2020
Normalization effects on shallow neural networks and related asymptotic expansions Jiahui Yu K. Spiliopoulos 13 6 0 20 Nov 2020
A contribution to Optimal Transport on incomparable spaces Titouan Vayer OT 25 19 0 09 Nov 2020
On the Convergence of Gradient Descent in GANs: MMD GAN As a Gradient Flow Youssef Mroueh Truyen V. Nguyen 29 25 0 04 Nov 2020
Dataset Dynamics via Gradient Flows in Probability Space David Alvarez-Melis Nicolò Fusi 26 18 0 24 Oct 2020
Global optimality of softmax policy gradient with single hidden layer neural networks in the mean-field regime Andrea Agazzi Jianfeng Lu 13 15 0 22 Oct 2020
Beyond Lazy Training for Over-parameterized Tensor Decomposition Xiang Wang Chenwei Wu J. Lee Tengyu Ma Rong Ge 11 14 0 22 Oct 2020
Deep Neural Networks Are Congestion Games: From Loss Landscape to Wardrop Equilibrium and Beyond Nina Vesseron I. Redko Charlotte Laclau 31 5 0 21 Oct 2020
A Continuous-Time Mirror Descent Approach to Sparse Phase Retrieval Fan Wu Patrick Rebeschini 37 14 0 20 Oct 2020
Deep Equals Shallow for ReLU Networks in Kernel Regimes A. Bietti Francis R. Bach 28 86 0 30 Sep 2020
Unbalanced Sobolev Descent Youssef Mroueh Mattia Rigotti 11 17 0 29 Sep 2020
Escaping Saddle-Points Faster under Interpolation-like Conditions Abhishek Roy Krishnakumar Balasubramanian Saeed Ghadimi P. Mohapatra 12 1 0 28 Sep 2020
How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks Keyulu Xu Mozhi Zhang Jingling Li S. Du Ken-ichi Kawarabayashi Stefanie Jegelka MLT 19 305 0 24 Sep 2020
Machine Learning and Computational Mathematics Weinan E PINN AI4CE 29 61 0 23 Sep 2020
Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don't E. Weinan Chao Ma Stephan Wojtowytsch Lei Wu AI4CE 22 133 0 22 Sep 2020
The Unbalanced Gromov Wasserstein Distance: Conic Formulation and Relaxation Thibault Séjourné François-Xavier Vialard Gabriel Peyré OT 27 67 0 09 Sep 2020
A Dynamical Central Limit Theorem for Shallow Neural Networks Zhengdao Chen Grant M. Rotskoff Joan Bruna Eric Vanden-Eijnden 18 29 0 21 Aug 2020
Active Importance Sampling for Variational Objectives Dominated by Rare Events: Consequences for Optimization and Generalization Grant M. Rotskoff Andrew R. Mitchell Eric Vanden-Eijnden 11 13 0 11 Aug 2020
Large-time asymptotics in deep learning Carlos Esteve Borjan Geshkovski Dario Pighin Enrique Zuazua 14 34 0 06 Aug 2020
Sketching Datasets for Large-Scale Learning (long version) Rémi Gribonval Antoine Chatalic Nicolas Keriven V. Schellekens Laurent Jacques P. Schniter 17 5 0 04 Aug 2020