The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponents

5 February 2024

Lenka Zdeborová

Papers citing "The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponents"

22 / 22 papers shown

Title
Low-dimensional Functions are Efficiently Learnable under Randomly Biased Distributions Elisabetta Cornacchia Dan Mikulincer Elchanan Mossel 109 1 0 10 Feb 2025
Mean-Field Analysis for Learning Subspace-Sparse Polynomials with Gaussian Input Ziang Chen Rong Ge MLT 112 1 0 10 Jan 2025
Learning Gaussian Multi-Index Models with Gradient Flow: Time Complexity and Directional Convergence Berfin Simsek Amire Bendjeddou Daniel Hsu 138 2 0 13 Nov 2024
Robust Feature Learning for Multi-Index Models in High Dimensions Alireza Mousavi-Hosseini Adel Javanmard Murat A. Erdogdu OOD AAML 102 1 0 21 Oct 2024
Learning Multi-Index Models with Neural Networks via Mean-Field Langevin Dynamics Alireza Mousavi-Hosseini Denny Wu Murat A. Erdogdu MLT AI4CE 66 7 0 14 Aug 2024
Repetita Iuvant: Data Repetition Allows SGD to Learn High-Dimensional Multi-Index Functions Luca Arnaboldi Yatin Dandi Florent Krzakala Luca Pesce Ludovic Stephan 99 18 0 24 May 2024
A Theory of Non-Linear Feature Learning with One Gradient Step in Two-Layer Neural Networks Behrad Moniri Donghwan Lee Hamed Hassani Yan Sun MLT 84 22 0 11 Oct 2023
High-dimensional limit theorems for SGD: Effective dynamics and critical scaling Gerard Ben Arous Reza Gheissari Aukosh Jagannath 99 58 0 08 Jun 2022
High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation Jimmy Ba Murat A. Erdogdu Taiji Suzuki Zhichao Wang Denny Wu Greg Yang MLT 87 128 0 03 May 2022
Universality of empirical risk minimization Andrea Montanari Basil Saeed OOD 63 75 0 17 Feb 2022
The high-dimensional asymptotics of first order methods with random data Michael Celentano Chen Cheng Andrea Montanari AI4CE 37 39 0 14 Dec 2021
When Do Neural Networks Outperform Kernel Methods? Behrooz Ghorbani Song Mei Theodor Misiakiewicz Andrea Montanari 86 188 0 24 Jun 2020
Algorithms and SQ Lower Bounds for PAC Learning One-Hidden-Layer ReLU Networks Ilias Diakonikolas D. Kane Vasilis Kontonis Nikos Zarifis 59 65 0 22 Jun 2020
Phase retrieval in high dimensions: Statistical and computational phase transitions Antoine Maillard Bruno Loureiro Florent Krzakala Lenka Zdeborová 52 58 0 09 Jun 2020
Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networks Blake Bordelon Abdulkadir Canatar Cengiz Pehlevan 223 206 0 07 Feb 2020
Who is Afraid of Big Bad Minima? Analysis of Gradient-Flow in a Spiked Matrix-Tensor Model Stefano Sarao Mannelli Giulio Biroli C. Cammarota Florent Krzakala Lenka Zdeborová 52 42 0 18 Jul 2019
Limitations of Lazy Training of Two-layers Neural Networks Behrooz Ghorbani Song Mei Theodor Misiakiewicz Andrea Montanari MLT 55 143 0 21 Jun 2019
Mean Field Analysis of Neural Networks: A Central Limit Theorem Justin A. Sirignano K. Spiliopoulos MLT 67 194 0 28 Aug 2018
The committee machine: Computational to statistical gaps in learning a two-layers neural network Benjamin Aubin Antoine Maillard Jean Barbier Florent Krzakala N. Macris Lenka Zdeborová 72 106 0 14 Jun 2018
Trainability and Accuracy of Neural Networks: An Interacting Particle System Approach Grant M. Rotskoff Eric Vanden-Eijnden 106 122 0 02 May 2018
A Mean Field View of the Landscape of Two-Layers Neural Networks Song Mei Andrea Montanari Phan-Minh Nguyen MLT 91 858 0 18 Apr 2018
Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Models Jean Barbier Florent Krzakala N. Macris Léo Miolane Lenka Zdeborová 80 265 0 10 Aug 2017