Gradient Descent Robustly Learns the Intrinsic Dimension of Data in Training Convolutional Neural Networks

11 April 2025

Papers citing "Gradient Descent Robustly Learns the Intrinsic Dimension of Data in Training Convolutional Neural Networks"

40 / 40 papers shown

Title
The Implicit Bias of Adam on Separable Data Chenyang Zhang Difan Zou Yuan Cao AI4CE 78 9 0 15 Jun 2024
$Implicit Bias of AdamW: $\ell_\infty$ Norm Constrained Optimization$ Implicit Bias of AdamW: $\ell_\infty$ Norm Constrained Optimization Shuo Xie Zhiyuan Li OffRL 78 23 0 05 Apr 2024
Implicit Bias of Gradient Descent for Two-layer ReLU and Leaky ReLU Networks on Nearly-orthogonal Data Yiwen Kou Zixiang Chen Quanquan Gu MLT 41 15 0 29 Oct 2023
Benign Overfitting in Two-Layer ReLU Convolutional Neural Networks for XOR Data Xuran Meng Difan Zou Yuan Cao MLT 90 9 0 03 Oct 2023
The Implicit Bias of Batch Normalization in Linear Models and Two-layer Linear Convolutional Neural Networks Yuan Cao Difan Zou Yuan-Fang Li Quanquan Gu MLT 100 5 0 20 Jun 2023
Benign Overfitting for Two-layer ReLU Convolutional Neural Networks Yiwen Kou Zi-Yuan Chen Yuanzhou Chen Quanquan Gu MLT 87 17 0 07 Mar 2023
Benign Overfitting in Linear Classifiers and Leaky ReLU Networks from KKT Conditions for Margin Maximization Spencer Frei Gal Vardi Peter L. Bartlett Nathan Srebro 75 23 0 02 Mar 2023
Vision Transformers provably learn spatial structure Samy Jelassi Michael E. Sander Yuan-Fang Li ViT MLT 89 82 0 13 Oct 2022
Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Data Spencer Frei Gal Vardi Peter L. Bartlett Nathan Srebro Wei Hu MLT 73 42 0 13 Oct 2022
Implicit Bias of Large Depth Networks: a Notion of Rank for Nonlinear Functions Arthur Jacot 97 27 0 29 Sep 2022
On the Implicit Bias in Deep-Learning Algorithms Gal Vardi FedML AI4CE 71 81 0 26 Aug 2022
Multiple Descent in the Multiple Random Feature Model Xuran Meng Jianfeng Yao Yuan Cao 72 7 0 21 Aug 2022
Benign Overfitting in Two-layer Convolutional Neural Networks Yuan Cao Zixiang Chen M. Belkin Quanquan Gu MLT 85 89 0 14 Feb 2022
Implicit Regularization Towards Rank Minimization in ReLU Networks Nadav Timor Gal Vardi Ohad Shamir 83 51 0 30 Jan 2022
Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias Kaifeng Lyu Zhiyuan Li Runzhe Wang Sanjeev Arora MLT 95 76 0 26 Oct 2021
Does Momentum Change the Implicit Regularization on Separable Data? Bohan Wang Qi Meng Huishuai Zhang Ruoyu Sun Wei Chen Zhirui Ma Tie-Yan Liu 99 18 0 08 Oct 2021
On the Impact of Stable Ranks in Deep Nets B. Georgiev L. Franken Mayukh Mukherjee Georgios Arvanitidis 62 3 0 05 Oct 2021
Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularization Difan Zou Yuan Cao Yuanzhi Li Quanquan Gu MLT AI4CE 110 44 0 25 Aug 2021
Towards Understanding the Condensation of Neural Networks at Initial Training Hanxu Zhou Qixuan Zhou Yaoyu Zhang Yaoyu Zhang Z. Xu MLT AI4CE 73 30 0 25 May 2021
Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning Zhiyuan Li Yuping Luo Kaifeng Lyu 95 130 0 17 Dec 2020
Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning Zeyuan Allen-Zhu Yuanzhi Li FedML 151 376 0 17 Dec 2020
The Implicit Bias for Adaptive Optimization Algorithms on Homogeneous Neural Networks Bohan Wang Qi Meng Wei Chen Tie-Yan Liu 73 36 0 11 Dec 2020
The Neural Tangent Kernel in High Dimensions: Triple Descent and a Multi-Scale Theory of Generalization Ben Adlam Jeffrey Pennington 58 125 0 15 Aug 2020
Directional convergence and alignment in deep learning Ziwei Ji Matus Telgarsky 68 171 0 11 Jun 2020
$On the Optimal Weighted $\ell_2$ Regularization in Overparameterized Linear Regression$ On the Optimal Weighted $\ell_2$ Regularization in Overparameterized Linear Regression Denny Wu Ji Xu 72 123 0 10 Jun 2020
Implicit Regularization in Deep Learning May Not Be Explainable by Norms Noam Razin Nadav Cohen 76 156 0 13 May 2020
Finite-sample Analysis of Interpolating Linear Classifiers in the Overparameterized Regime Niladri S. Chatterji Philip M. Long 83 109 0 25 Apr 2020
Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss Lénaïc Chizat Francis R. Bach MLT 154 341 0 11 Feb 2020
Benign Overfitting in Linear Regression Peter L. Bartlett Philip M. Long Gábor Lugosi Alexander Tsigler MLT 105 779 0 26 Jun 2019
Gradient Descent Maximizes the Margin of Homogeneous Neural Networks Kaifeng Lyu Jian Li 102 336 0 13 Jun 2019
Implicit Regularization in Deep Matrix Factorization Sanjeev Arora Nadav Cohen Wei Hu Yuping Luo AI4CE 99 509 0 31 May 2019
Surprises in High-Dimensional Ridgeless Least Squares Interpolation Trevor Hastie Andrea Montanari Saharon Rosset Robert Tibshirani 228 747 0 19 Mar 2019
Two models of double descent for weak features M. Belkin Daniel J. Hsu Ji Xu 117 375 0 18 Mar 2019
Reconciling modern machine learning practice and the bias-variance trade-off M. Belkin Daniel J. Hsu Siyuan Ma Soumik Mandal 249 1,660 0 28 Dec 2018
Gradient descent aligns the layers of deep linear networks Ziwei Ji Matus Telgarsky 123 257 0 04 Oct 2018
Stochastic Gradient Descent on Separable Data: Exact Convergence with a Fixed Learning Rate Mor Shpigel Nacson Nathan Srebro Daniel Soudry FedML MLT 90 102 0 05 Jun 2018
Characterizing Implicit Bias in Terms of Optimization Geometry Suriya Gunasekar Jason D. Lee Daniel Soudry Nathan Srebro AI4CE 75 413 0 22 Feb 2018
The Implicit Bias of Gradient Descent on Separable Data Daniel Soudry Elad Hoffer Mor Shpigel Nacson Suriya Gunasekar Nathan Srebro 178 924 0 27 Oct 2017
Implicit Regularization in Matrix Factorization Suriya Gunasekar Blake E. Woodworth Srinadh Bhojanapalli Behnam Neyshabur Nathan Srebro 81 493 0 25 May 2017
In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning Behnam Neyshabur Ryota Tomioka Nathan Srebro AI4CE 106 663 0 20 Dec 2014