From high-dimensional & mean-field dynamics to dimensionless ODEs: A unifying approach to SGD in two-layers networks

12 February 2023

Papers citing "From high-dimensional & mean-field dynamics to dimensionless ODEs: A unifying approach to SGD in two-layers networks"

24 / 24 papers shown

Title
A theoretical framework for overfitting in energy-based modeling Giovanni Catania A. Decelle Cyril Furtlehner Beatriz Seoane 57 2 0 31 Jan 2025
Exploring the Precise Dynamics of Single-Layer GAN Models: Leveraging Multi-Feature Discriminators for High-Dimensional Subspace Learning Andrew Bond Zafer Dogan 23 0 0 01 Nov 2024
A theoretical perspective on mode collapse in variational inference Roman Soletskyi Marylou Gabrié Bruno Loureiro DRL 29 2 0 17 Oct 2024
Exploring Loss Landscapes through the Lens of Spin Glass Theory Hao Liao Wei Zhang Zhanyi Huang Zexiao Long Mingyang Zhou Xiaoqun Wu Rui Mao Chi Ho Yeung 56 2 0 30 Jul 2024
A spring-block theory of feature learning in deep neural networks Chengzhi Shi Liming Pan Ivan Dokmanić AI4CE 40 1 0 28 Jul 2024
Online Learning and Information Exponents: On The Importance of Batch size, and Time/Complexity Tradeoffs Luca Arnaboldi Yatin Dandi Florent Krzakala Bruno Loureiro Luca Pesce Ludovic Stephan 47 1 0 04 Jun 2024
The High Line: Exact Risk and Learning Rate Curves of Stochastic Adaptive Learning Rate Algorithms Elizabeth Collins-Woodfin Inbar Seroussi Begona García Malaxechebarría Andrew W. Mackenzie Elliot Paquette Courtney Paquette 30 1 0 30 May 2024
Bias in Motion: Theoretical Insights into the Dynamics of Bias in SGD Training Anchit Jain Rozhin Nobahari A. Baratin Stefano Sarao Mannelli 36 4 0 28 May 2024
Repetita Iuvant: Data Repetition Allows SGD to Learn High-Dimensional Multi-Index Functions Luca Arnaboldi Yatin Dandi Florent Krzakala Luca Pesce Ludovic Stephan 70 12 0 24 May 2024
High dimensional analysis reveals conservative sharpening and a stochastic edge of stability Atish Agarwala Jeffrey Pennington 41 3 0 30 Apr 2024
Asymptotics of feature learning in two-layer networks after one gradient-step Hugo Cui Luca Pesce Yatin Dandi Florent Krzakala Yue M. Lu Lenka Zdeborová Bruno Loureiro MLT 58 16 0 07 Feb 2024
Weight fluctuations in (deep) linear neural networks and a derivation of the inverse-variance flatness relation Markus Gross A. Raulf Christoph Räth 43 0 0 23 Nov 2023
On the Impact of Overparameterization on the Training of a Shallow Neural Network in High Dimensions Simon Martin Francis Bach Giulio Biroli 23 9 0 07 Nov 2023
Grokking as the Transition from Lazy to Rich Training Dynamics Tanishq Kumar Blake Bordelon Samuel Gershman C. Pehlevan 35 31 0 09 Oct 2023
Grokking as a First Order Phase Transition in Two Layer Networks Noa Rubin Inbar Seroussi Z. Ringel 37 15 0 05 Oct 2023
Symmetric Single Index Learning Aaron Zweig Joan Bruna MLT 33 2 0 03 Oct 2023
On the different regimes of Stochastic Gradient Descent Antonio Sclocchi M. Wyart 28 17 0 19 Sep 2023
Gradient-Based Feature Learning under Structured Data Alireza Mousavi-Hosseini Denny Wu Taiji Suzuki Murat A. Erdogdu MLT 34 18 0 07 Sep 2023
On Single Index Models beyond Gaussian Data Joan Bruna Loucas Pillaud-Vivien Aaron Zweig 18 10 0 28 Jul 2023
The RL Perceptron: Generalisation Dynamics of Policy Learning in High Dimensions Nishil Patel Sebastian Lee Stefano Sarao Mannelli Sebastian Goldt Adrew Saxe OffRL 28 3 0 17 Jun 2023
How Two-Layer Neural Networks Learn, One (Giant) Step at a Time Yatin Dandi Florent Krzakala Bruno Loureiro Luca Pesce Ludovic Stephan MLT 34 26 0 29 May 2023
Escaping mediocrity: how two-layer networks learn hard generalized linear models with SGD Luca Arnaboldi Florent Krzakala Bruno Loureiro Ludovic Stephan MLT 33 3 0 29 May 2023
Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networks Blake Bordelon C. Pehlevan MLT 38 29 0 06 Apr 2023
Learning time-scales in two-layers neural networks Raphael Berthier Andrea Montanari Kangjie Zhou 38 33 0 28 Feb 2023