Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks

Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks

11 May 2023

Alexandru Damian

Papers citing "Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks"

16 / 16 papers shown

Title
Robust Feature Learning for Multi-Index Models in High Dimensions Alireza Mousavi-Hosseini Adel Javanmard Murat A. Erdogdu OOD AAML 44 1 0 21 Oct 2024
Generalization for Least Squares Regression With Simple Spiked Covariances Jiping Li Rishi Sonthalia 28 0 0 17 Oct 2024
Learning Multi-Index Models with Neural Networks via Mean-Field Langevin Dynamics Alireza Mousavi-Hosseini Denny Wu Murat A. Erdogdu MLT AI4CE 32 6 0 14 Aug 2024
Crafting Heavy-Tails in Weight Matrix Spectrum without Gradient Noise Vignesh Kothapalli Tianyu Pang Shenyang Deng Zongmin Liu Yaoqing Yang 37 3 0 07 Jun 2024
Neural network learns low-dimensional polynomials with SGD near the information-theoretic limit Jason D. Lee Kazusato Oko Taiji Suzuki Denny Wu MLT 87 21 0 03 Jun 2024
Understanding Optimal Feature Transfer via a Fine-Grained Bias-Variance Analysis Yufan Li Subhabrata Sen Ben Adlam MLT 51 1 0 18 Apr 2024
Depth Separations in Neural Networks: Separating the Dimension from the Accuracy Itay Safran Daniel Reichman Paul Valiant 61 0 0 11 Feb 2024
Feature learning as alignment: a structural property of gradient descent in non-linear neural networks Daniel Beaglehole Ioannis Mitliagkas Atish Agarwala MLT 34 2 0 07 Feb 2024
A Theory of Non-Linear Feature Learning with One Gradient Step in Two-Layer Neural Networks Behrad Moniri Donghwan Lee Hamed Hassani Yan Sun MLT 40 19 0 11 Oct 2023
Nonparametric Classification on Low Dimensional Manifolds using Overparameterized Convolutional Residual Networks Kaiqi Zhang Zixuan Zhang Minshuo Chen Yuma Takeda Mengdi Wang Tuo Zhao Yu-Xiang Wang 32 0 0 04 Jul 2023
SGD learning on neural networks: leap complexity and saddle-to-saddle dynamics Emmanuel Abbe Enric Boix-Adserà Theodor Misiakiewicz FedML MLT 79 73 0 21 Feb 2023
Learning Single-Index Models with Shallow Neural Networks A. Bietti Joan Bruna Clayton Sanford M. Song 164 67 0 27 Oct 2022
Optimization-Based Separations for Neural Networks Itay Safran Jason D. Lee 137 14 0 04 Dec 2021
Increasing Depth Leads to U-Shaped Test Risk in Over-parameterized Convolutional Networks Eshaan Nichani Adityanarayanan Radhakrishnan Caroline Uhler 24 9 0 19 Oct 2020
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks Lechao Xiao Yasaman Bahri Jascha Narain Sohl-Dickstein S. Schoenholz Jeffrey Pennington 227 348 0 14 Jun 2018
Benefits of depth in neural networks Matus Telgarsky 148 602 0 14 Feb 2016