A Mean Field View of the Landscape of Two-Layers Neural Networks

18 April 2018

Papers citing "A Mean Field View of the Landscape of Two-Layers Neural Networks"

50 / 206 papers shown

Title
EPR-Net: Constructing non-equilibrium potential landscape via a variational force projection formulation Yue Zhao Wei Zhang Tiejun Li DiffM 16 8 0 05 Jan 2023
Learning Gaussian Mixtures Using the Wasserstein-Fisher-Rao Gradient Flow Yuling Yan Kaizheng Wang Philippe Rigollet 44 20 0 04 Jan 2023
An Analysis of Attention via the Lens of Exchangeability and Latent Variable Models Yufeng Zhang Boyi Liu Qi Cai Lingxiao Wang Zhaoran Wang 53 11 0 30 Dec 2022
The Underlying Correlated Dynamics in Neural Training Rotem Turjeman Tom Berkov I. Cohen Guy Gilboa 21 3 0 18 Dec 2022
Uniform-in-time propagation of chaos for mean field Langevin dynamics Fan Chen Zhenjie Ren Song-bo Wang 43 30 0 06 Dec 2022
Infinite-width limit of deep linear neural networks Lénaïc Chizat Maria Colombo Xavier Fernández-Real Alessio Figalli 31 14 0 29 Nov 2022
REPAIR: REnormalizing Permuted Activations for Interpolation Repair Keller Jordan Hanie Sedghi O. Saukh R. Entezari Behnam Neyshabur MoMe 46 94 0 15 Nov 2022
A Functional-Space Mean-Field Theory of Partially-Trained Three-Layer Neural Networks Zhengdao Chen Eric Vanden-Eijnden Joan Bruna MLT 25 5 0 28 Oct 2022
Stochastic Mirror Descent in Average Ensemble Models Taylan Kargin Fariborz Salehi B. Hassibi 24 1 0 27 Oct 2022
Proximal Mean Field Learning in Shallow Neural Networks Alexis M. H. Teter Iman Nodozi A. Halder FedML 43 1 0 25 Oct 2022
Global Convergence of SGD On Two Layer Neural Nets Pulkit Gopalani Anirbit Mukherjee 26 5 0 20 Oct 2022
Wasserstein Barycenter-based Model Fusion and Linear Mode Connectivity of Neural Networks A. K. Akash Sixu Li Nicolas García Trillos 34 12 0 13 Oct 2022
Annihilation of Spurious Minima in Two-Layer ReLU Networks Yossi Arjevani M. Field 16 8 0 12 Oct 2022
Meta-Principled Family of Hyperparameter Scaling Strategies Sho Yaida 55 16 0 10 Oct 2022
Analysis of the rate of convergence of an over-parametrized deep neural network estimate learned by gradient descent Michael Kohler A. Krzyżak 32 10 0 04 Oct 2022
Lazy vs hasty: linearization in deep networks impacts learning schedule based on example difficulty Thomas George Guillaume Lajoie A. Baratin 28 5 0 19 Sep 2022
Robustness in deep learning: The good (width), the bad (depth), and the ugly (initialization) Zhenyu Zhu Fanghui Liu Grigorios G. Chrysos V. Cevher 39 19 0 15 Sep 2022
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 255 314 0 11 Sep 2022
On the universal consistency of an over-parametrized deep neural network estimate learned by gradient descent Selina Drews Michael Kohler 30 13 0 30 Aug 2022
The Neural Race Reduction: Dynamics of Abstraction in Gated Networks Andrew M. Saxe Shagun Sodhani Sam Lewallen AI4CE 28 34 0 21 Jul 2022
Neural Networks can Learn Representations with Gradient Descent Alexandru Damian Jason D. Lee Mahdi Soltanolkotabi SSL MLT 19 114 0 30 Jun 2022
Learning sparse features can lead to overfitting in neural networks Leonardo Petrini Francesco Cagnetta Eric Vanden-Eijnden M. Wyart MLT 42 23 0 24 Jun 2022
Label noise (stochastic) gradient descent implicitly solves the Lasso for quadratic parametrisation Loucas Pillaud-Vivien J. Reygner Nicolas Flammarion NoLa 33 31 0 20 Jun 2022
Unbiased Estimation using Underdamped Langevin Dynamics Hamza Ruzayqat Neil K. Chada Ajay Jasra 33 4 0 14 Jun 2022
High-dimensional limit theorems for SGD: Effective dynamics and critical scaling Gerard Ben Arous Reza Gheissari Aukosh Jagannath 59 58 0 08 Jun 2022
Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs Etienne Boursier Loucas Pillaud-Vivien Nicolas Flammarion ODL 21 58 0 02 Jun 2022
PSO-Convolutional Neural Networks with Heterogeneous Learning Rate N. H. Phong A. Santos B. Ribeiro 21 8 0 20 May 2022
Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks Blake Bordelon C. Pehlevan MLT 34 78 0 19 May 2022
Sharp asymptotics on the compression of two-layer neural networks Mohammad Hossein Amani Simone Bombari Marco Mondelli Rattana Pukdee Stefano Rini MLT 24 0 0 17 May 2022
Mean-Field Nonparametric Estimation of Interacting Particle Systems Rentian Yao Xiaohui Chen Yun Yang 43 9 0 16 May 2022
Trajectory Inference via Mean-field Langevin in Path Space Lénaïc Chizat Stephen X. Zhang Matthieu Heitz Geoffrey Schiebinger 31 21 0 14 May 2022
High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation Jimmy Ba Murat A. Erdogdu Taiji Suzuki Zhichao Wang Denny Wu Greg Yang MLT 37 121 0 03 May 2022
On Feature Learning in Neural Networks with Global Convergence Guarantees Zhengdao Chen Eric Vanden-Eijnden Joan Bruna MLT 36 13 0 22 Apr 2022
On the (Non-)Robustness of Two-Layer Neural Networks in Different Learning Regimes Elvis Dohmatob A. Bietti AAML 32 13 0 22 Mar 2022
Fully-Connected Network on Noncompact Symmetric Space and Ridgelet Transform based on Helgason-Fourier Analysis Sho Sonoda Isao Ishikawa Masahiro Ikeda 21 15 0 03 Mar 2022
A blob method for inhomogeneous diffusion with applications to multi-agent control and sampling Katy Craig Karthik Elamvazhuthi M. Haberland O. Turanova 35 15 0 25 Feb 2022
Provably convergent quasistatic dynamics for mean-field two-player zero-sum games Chao Ma Lexing Ying MLT 32 11 0 15 Feb 2022
Random Feature Amplification: Feature Learning and Generalization in Neural Networks Spencer Frei Niladri S. Chatterji Peter L. Bartlett MLT 30 29 0 15 Feb 2022
Simultaneous Transport Evolution for Minimax Equilibria on Measures Carles Domingo-Enrich Joan Bruna 16 3 0 14 Feb 2022
Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks R. Veiga Ludovic Stephan Bruno Loureiro Florent Krzakala Lenka Zdeborová MLT 10 31 0 01 Feb 2022
Improved Overparametrization Bounds for Global Convergence of Stochastic Gradient Descent for Shallow Neural Networks Bartlomiej Polaczyk J. Cyranka ODL 33 3 0 28 Jan 2022
Convex Analysis of the Mean Field Langevin Dynamics Atsushi Nitanda Denny Wu Taiji Suzuki MLT 68 64 0 25 Jan 2022
Overview frequency principle/spectral bias in deep learning Z. Xu Yaoyu Zhang Tao Luo FaML 33 66 0 19 Jan 2022
Convergence of Policy Gradient for Entropy Regularized MDPs with Neural Network Approximation in the Mean-Field Regime B. Kerimkulov J. Leahy David Siska Lukasz Szpruch 27 11 0 18 Jan 2022
Neural Capacitance: A New Perspective of Neural Network Selection via Edge Dynamics Chunheng Jiang Tejaswini Pedapati Pin-Yu Chen Yizhou Sun Jianxi Gao 24 2 0 11 Jan 2022
Separation of Scales and a Thermodynamic Description of Feature Learning in Some CNNs Inbar Seroussi Gadi Naveh Z. Ringel 30 50 0 31 Dec 2021
Asymptotic properties of one-layer artificial neural networks with sparse connectivity Christian Hirsch Matthias Neumann Volker Schmidt 19 1 0 01 Dec 2021
DNN gradient lossless compression: Can GenNorm be the answer? Zhongzhu Chen Eduin E. Hernandez Yu-Chih Huang Stefano Rini 28 9 0 15 Nov 2021
Efficient Neural Network Training via Forward and Backward Propagation Sparsification Xiao Zhou Weizhong Zhang Zonghao Chen Shizhe Diao Tong Zhang 32 46 0 10 Nov 2021
Mean-field Analysis of Piecewise Linear Solutions for Wide ReLU Networks A. Shevchenko Vyacheslav Kungurtsev Marco Mondelli MLT 36 13 0 03 Nov 2021