Directional Convergence, Benign Overfitting of Gradient Descent in leaky ReLU two-layer Neural Networks

22 May 2025

Papers citing "Directional Convergence, Benign Overfitting of Gradient Descent in leaky ReLU two-layer Neural Networks"

28 / 28 papers shown

Title
Universality of Benign Overfitting in Binary Linear Classification Ichiro Hashimoto Stanislav Volgushev Piotr Zwiernik 78 1 0 17 Jan 2025
Implicit Bias of Gradient Descent for Two-layer ReLU and Leaky ReLU Networks on Nearly-orthogonal Data Yiwen Kou Zixiang Chen Quanquan Gu MLT 13 14 0 29 Oct 2023
Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization Hancheng Min Enrique Mallada René Vidal MLT 53 21 0 24 Jul 2023
Benign Overfitting for Two-layer ReLU Convolutional Neural Networks Yiwen Kou Zi-Yuan Chen Yuanzhou Chen Quanquan Gu MLT 60 16 0 07 Mar 2023
Benign Overfitting in Linear Classifiers and Leaky ReLU Networks from KKT Conditions for Margin Maximization Spencer Frei Gal Vardi Peter L. Bartlett Nathan Srebro 44 23 0 02 Mar 2023
Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Data Spencer Frei Gal Vardi Peter L. Bartlett Nathan Srebro Wei Hu MLT 40 41 0 13 Oct 2022
On the Implicit Bias in Deep-Learning Algorithms Gal Vardi FedML AI4CE 45 79 0 26 Aug 2022
Benign Overfitting in Two-layer Convolutional Neural Networks Yuan Cao Zixiang Chen M. Belkin Quanquan Gu MLT 41 88 0 14 Feb 2022
Benign Overfitting without Linearity: Neural Network Classifiers Trained by Gradient Descent for Noisy Linear Data Spencer Frei Niladri S. Chatterji Peter L. Bartlett MLT 53 72 0 11 Feb 2022
Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias Kaifeng Lyu Zhiyuan Li Runzhe Wang Sanjeev Arora MLT 51 74 0 26 Oct 2021
Risk Bounds for Over-parameterized Maximum Margin Classification on Sub-Gaussian Mixtures Yuan Cao Quanquan Gu M. Belkin 30 52 0 28 Apr 2021
Towards Understanding Learning in Neural Networks with Linear Teachers Roei Sarussi Alon Brutzkus Amir Globerson FedML MLT 69 21 0 07 Jan 2021
Binary Classification of Gaussian Mixtures: Abundance of Support Vectors, Benign Overfitting and Regularization Ke Wang Christos Thrampoulidis 46 28 0 18 Nov 2020
Benign overfitting in ridge regression Alexander Tsigler Peter L. Bartlett 47 164 0 29 Sep 2020
On the proliferation of support vectors in high dimensions Daniel J. Hsu Vidya Muthukumar Ji Xu 41 44 0 22 Sep 2020
Directional convergence and alignment in deep learning Ziwei Ji Matus Telgarsky 41 167 0 11 Jun 2020
Finite-sample Analysis of Interpolating Linear Classifiers in the Overparameterized Regime Niladri S. Chatterji Philip M. Long 24 109 0 25 Apr 2020
The generalization error of random features regression: Precise asymptotics and double descent curve Song Mei Andrea Montanari 68 631 0 14 Aug 2019
Benign Overfitting in Linear Regression Peter L. Bartlett Philip M. Long Gábor Lugosi Alexander Tsigler MLT 55 769 0 26 Jun 2019
Gradient Descent Maximizes the Margin of Homogeneous Neural Networks Kaifeng Lyu Jian Li 68 332 0 13 Jun 2019
Harmless interpolation of noisy data in regression Vidya Muthukumar Kailas Vodrahalli Vignesh Subramanian A. Sahai 47 204 0 21 Mar 2019
Surprises in High-Dimensional Ridgeless Least Squares Interpolation Trevor Hastie Andrea Montanari Saharon Rosset Robert Tibshirani 124 737 0 19 Mar 2019
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruosong Wang MLT 118 966 0 24 Jan 2019
Reconciling modern machine learning practice and the bias-variance trade-off M. Belkin Daniel J. Hsu Siyuan Ma Soumik Mandal 172 1,628 0 28 Dec 2018
Just Interpolate: Kernel "Ridgeless" Regression Can Generalize Tengyuan Liang Alexander Rakhlin 46 353 0 01 Aug 2018
Neural Tangent Kernel: Convergence and Generalization in Neural Networks Arthur Jacot Franck Gabriel Clément Hongler 170 3,160 0 20 Jun 2018
The Implicit Bias of Gradient Descent on Separable Data Daniel Soudry Elad Hoffer Mor Shpigel Nacson Suriya Gunasekar Nathan Srebro 66 908 0 27 Oct 2017
Understanding deep learning requires rethinking generalization Chiyuan Zhang Samy Bengio Moritz Hardt Benjamin Recht Oriol Vinyals HAI 264 4,620 0 10 Nov 2016