Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization

24 July 2023

Papers citing "Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization"

6 / 6 papers shown

Title
Directional Convergence, Benign Overfitting of Gradient Descent in leaky ReLU two-layer Neural Networks Ichiro Hashimoto MLT 22 0 0 22 May 2025
Early Directional Convergence in Deep Homogeneous Neural Networks for Small Initializations Akshay Kumar Jarvis Haupt ODL 49 3 0 12 Mar 2024
Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction Dominik Stöger Mahdi Soltanolkotabi ODL 56 75 0 28 Jun 2021
Gradient Descent Maximizes the Margin of Homogeneous Neural Networks Kaifeng Lyu Jian Li 61 332 0 13 Jun 2019
Implicit Regularization in Deep Matrix Factorization Sanjeev Arora Nadav Cohen Wei Hu Yuping Luo AI4CE 52 500 0 31 May 2019
Gradient Descent Quantizes ReLU Network Features Hartmut Maennel Olivier Bousquet Sylvain Gelly MLT 26 80 0 22 Mar 2018