A Mean-field Analysis of Deep ResNet and Beyond: Towards Provable Optimization Via Overparameterization From Depth

11 March 2020

Chao Ma

Papers citing "A Mean-field Analysis of Deep ResNet and Beyond: Towards Provable Optimization Via Overparameterization From Depth"

25 / 25 papers shown

Title
Convergence of Time-Averaged Mean Field Gradient Descent Dynamics for Continuous Multi-Player Zero-Sum Games Yulong Lu Pierre Monmarché MLT 34 0 0 12 May 2025
Understanding the training of infinitely deep and wide ResNets with Conditional Optimal Transport Raphael Barboni Gabriel Peyré Franccois-Xavier Vialard 37 3 0 19 Mar 2024
How Over-Parameterization Slows Down Gradient Descent in Matrix Sensing: The Curses of Symmetry and Initialization Nuoya Xiong Lijun Ding Simon S. Du 32 11 0 03 Oct 2023
Global Optimality of Elman-type RNN in the Mean-Field Regime Andrea Agazzi Jian-Xiong Lu Sayan Mukherjee MLT 31 1 0 12 Mar 2023
Over-Parameterization Exponentially Slows Down Gradient Descent for Learning a Single Neuron Weihang Xu S. Du 34 16 0 20 Feb 2023
ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradients Guihong Li Yuedong Yang Kartikeya Bhardwaj R. Marculescu 36 60 0 26 Jan 2023
Asymptotic Analysis of Deep Residual Networks R. Cont Alain Rossier Renyuan Xu 27 4 0 15 Dec 2022
A Functional-Space Mean-Field Theory of Partially-Trained Three-Layer Neural Networks Zhengdao Chen Eric Vanden-Eijnden Joan Bruna MLT 25 5 0 28 Oct 2022
On skip connections and normalisation layers in deep optimisation L. MacDonald Jack Valmadre Hemanth Saratchandran Simon Lucey ODL 19 1 0 10 Oct 2022
Deep Generalized Schrödinger Bridge Guan-Horng Liu T. Chen Oswin So Evangelos A. Theodorou OT AI4CE 13 35 0 20 Sep 2022
Do Residual Neural Networks discretize Neural Ordinary Differential Equations? Michael E. Sander Pierre Ablin Gabriel Peyré 32 25 0 29 May 2022
On Feature Learning in Neural Networks with Global Convergence Guarantees Zhengdao Chen Eric Vanden-Eijnden Joan Bruna MLT 33 13 0 22 Apr 2022
Provably convergent quasistatic dynamics for mean-field two-player zero-sum games Chao Ma Lexing Ying MLT 29 11 0 15 Feb 2022
On the Convergence of Shallow Neural Network Training with Randomly Masked Neurons Fangshuo Liao Anastasios Kyrillidis 38 16 0 05 Dec 2021
Subquadratic Overparameterization for Shallow Neural Networks Chaehwan Song Ali Ramezani-Kebrya Thomas Pethick Armin Eftekhari V. Cevher 27 31 0 02 Nov 2021
Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Pruned Neural Networks Shuai Zhang Meng Wang Sijia Liu Pin-Yu Chen Jinjun Xiong UQCV MLT 31 13 0 12 Oct 2021
On the Global Convergence of Gradient Descent for multi-layer ResNets in the mean-field regime Zhiyan Ding Shi Chen Qin Li S. Wright MLT AI4CE 38 11 0 06 Oct 2021
ResIST: Layer-Wise Decomposition of ResNets for Distributed Training Chen Dun Cameron R. Wolfe C. Jermaine Anastasios Kyrillidis 16 21 0 02 Jul 2021
Scaling Properties of Deep Residual Networks A. Cohen R. Cont Alain Rossier Renyuan Xu 22 18 0 25 May 2021
Global Convergence of Three-layer Neural Networks in the Mean Field Regime H. Pham Phan-Minh Nguyen MLT AI4CE 41 19 0 11 May 2021
A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Network Mo Zhou Rong Ge Chi Jin 74 44 0 04 Feb 2021
Neural Network Approximation: Three Hidden Layers Are Enough Zuowei Shen Haizhao Yang Shijun Zhang 30 115 0 25 Oct 2020
A Differential Game Theoretic Neural Optimizer for Training Residual Networks Guan-Horng Liu T. Chen Evangelos A. Theodorou 24 2 0 17 Jul 2020
Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theory Tao Luo Haizhao Yang 32 73 0 28 Jun 2020
Representing smooth functions as compositions of near-identity functions with implications for deep network optimization Peter L. Bartlett S. Evans Philip M. Long 73 31 0 13 Apr 2018