Gradient Descent Maximizes the Margin of Homogeneous Neural Networks

13 June 2019

Papers citing "Gradient Descent Maximizes the Margin of Homogeneous Neural Networks"

50 / 245 papers shown

Title
SAD Neural Networks: Divergent Gradient Flows and Asymptotic Optimality via o-minimal Structures Julian Kranz Davide Gallon Steffen Dereich Arnulf Jentzen 19 0 0 14 May 2025
Gradient Descent Robustly Learns the Intrinsic Dimension of Data in Training Convolutional Neural Networks Chenyang Zhang Peifeng Gao Difan Zou Yuan Cao OOD MLT 59 0 0 11 Apr 2025
Free Random Projection for In-Context Reinforcement Learning Tomohiro Hayase B. Collins Nakamasa Inoue 26 0 0 09 Apr 2025
An Overview of Low-Rank Structures in the Training and Adaptation of Large Models Laura Balzano Tianjiao Ding B. Haeffele Soo Min Kwon Qing Qu Peng Wang Zhilin Wang Can Yaras OffRL AI4CE 62 0 0 25 Mar 2025
Low-rank bias, weight decay, and model merging in neural networks Ilja Kuzborskij Yasin Abbasi-Yadkori 49 0 0 24 Feb 2025
Convergence of Shallow ReLU Networks on Weakly Interacting Data Léo Dana Francis R. Bach Loucas Pillaud-Vivien MLT 52 1 0 24 Feb 2025
Implicit Geometry of Next-token Prediction: From Language Sparsity Patterns to Model Representations Yize Zhao Tina Behnia V. Vakilian Christos Thrampoulidis 55 8 0 20 Feb 2025
Scalable Model Merging with Progressive Layer-wise Distillation Jing Xu Jiazheng Li Junzhe Zhang MoMe FedML 90 0 0 18 Feb 2025
The late-stage training dynamics of (stochastic) subgradient descent on homogeneous neural networks Sholom Schechtman Nicolas Schreuder 158 0 0 08 Feb 2025
Grokking at the Edge of Numerical Stability Lucas Prieto Melih Barsbey Pedro A.M. Mediano Tolga Birdal 42 3 0 08 Jan 2025
On the Reconstruction of Training Data from Group Invariant Networks Ran Elbaz Gilad Yehudai Meirav Galun Haggai Maron 71 0 0 25 Nov 2024
Stealing Training Graphs from Graph Neural Networks Minhua Lin Enyan Dai Junjie Xu Jinyuan Jia Xiang Zhang Suhang Wang DiffM 33 1 0 17 Nov 2024
Slowing Down Forgetting in Continual Learning Pascal Janetzky Tobias Schlagenhauf Stefan Feuerriegel CLL 34 0 0 11 Nov 2024
Do Mice Grok? Glimpses of Hidden Progress During Overtraining in Sensory Cortex Tanishq Kumar Blake Bordelon C. Pehlevan Venkatesh N. Murthy Samuel Gershman OOD CLL SSL 50 0 0 05 Nov 2024
The Implicit Bias of Gradient Descent on Separable Multiclass Data Hrithik Ravi Clayton Scott Daniel Soudry Yutong Wang 37 2 0 02 Nov 2024
Guiding Neural Collapse: Optimising Towards the Nearest Simplex Equiangular Tight Frame Evan Markou Thalaiyasingam Ajanthan Stephen Gould 26 0 0 02 Nov 2024
Where Do Large Learning Rates Lead Us? Ildus Sadrtdinov M. Kodryan Eduard Pokonechny E. Lobacheva Dmitry Vetrov AI4CE 34 0 0 29 Oct 2024
Rethinking generalization of classifiers in separable classes scenarios and over-parameterized regimes Julius Martinetz C. Linse Thomas Martinetz 26 0 0 22 Oct 2024
Implicit Regularization of Sharpness-Aware Minimization for Scale-Invariant Problems Bingcong Li Liang Zhang Niao He 43 3 0 18 Oct 2024
A Mirror Descent Perspective of Smoothed Sign Descent Shuyang Wang Diego Klabjan 40 0 0 18 Oct 2024
Evaluating of Machine Unlearning: Robustness Verification Without Prior Modifications Heng Xu Tianqing Zhu Wanlei Zhou MU AAML 26 1 0 14 Oct 2024
Learning to Compress: Local Rank and Information Compression in Deep Neural Networks Niket Patel Ravid Shwartz-Ziv SSL 23 1 0 10 Oct 2024
Adversarial Vulnerability as a Consequence of On-Manifold Inseparibility Rajdeep Haldar Yue Xing Qifan Song Guang Lin 36 0 0 09 Oct 2024
Simplicity bias and optimization threshold in two-layer ReLU networks Etienne Boursier Nicolas Flammarion 31 2 0 03 Oct 2024
Towards Better Generalization: Weight Decay Induces Low-rank Bias for Neural Networks Ke Chen Chugang Yi Haizhao Yang MLT 33 0 0 03 Oct 2024
Trained Transformer Classifiers Generalize and Exhibit Benign Overfitting In-Context Spencer Frei Gal Vardi MLT 28 3 0 02 Oct 2024
Reconstructing Training Data From Real World Models Trained with Transfer Learning Yakir Oz Gilad Yehudai Gal Vardi Itai Antebi Michal Irani Niv Haim 38 2 0 22 Jul 2024
Why Do You Grok? A Theoretical Analysis of Grokking Modular Addition Mohamad Amin Mohamadi Zhiyuan Li Lei Wu Danica J. Sutherland 48 9 0 17 Jul 2024
Implicit Bias of Mirror Flow on Separable Data Scott Pesme Radu-Alexandru Dragomir Nicolas Flammarion 36 1 0 18 Jun 2024
The Implicit Bias of Adam on Separable Data Chenyang Zhang Difan Zou Yuan Cao AI4CE 45 7 0 15 Jun 2024
Large Stepsize Gradient Descent for Non-Homogeneous Two-Layer Networks: Margin Improvement and Fast Optimization Yuhang Cai Jingfeng Wu Song Mei Michael Lindsey Peter L. Bartlett 32 2 0 12 Jun 2024
Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning D. Kunin Allan Raventós Clémentine Dominé Feng Chen David Klindt Andrew M. Saxe Surya Ganguli MLT 45 15 0 10 Jun 2024
The Price of Implicit Bias in Adversarially Robust Generalization Nikolaos Tsilivis Natalie Frank Nathan Srebro Julia Kempe 45 3 0 07 Jun 2024
Improving Generalization and Convergence by Enhancing Implicit Regularization Mingze Wang Haotian He Jinbo Wang Zilin Wang Guanhua Huang Feiyu Xiong Zhiyu Li E. Weinan Lei Wu 45 6 0 31 May 2024
Classifying Overlapping Gaussian Mixtures in High Dimensions: From Optimal Classifiers to Neural Nets Khen Cohen Noam Levi Yaron Oz BDL 31 1 0 28 May 2024
Simplicity Bias of Two-Layer Networks beyond Linearly Separable Data Nikita Tsoy Nikola Konstantinov 37 4 0 27 May 2024
When does compositional structure yield compositional generalization? A kernel theory Samuel Lippl Kim Stachenfeld NAI CoGe 73 5 0 26 May 2024
Can Implicit Bias Imply Adversarial Robustness? Hancheng Min René Vidal 36 3 0 24 May 2024
Progressive Feedforward Collapse of ResNet Training Sicong Wang Kuo Gai Shihua Zhang AI4CE 33 4 0 02 May 2024
$Implicit Bias of AdamW: $\ell_\infty$ Norm Constrained Optimization$ Implicit Bias of AdamW: $\ell_\infty$ Norm Constrained Optimization Shuo Xie Zhiyuan Li OffRL 47 13 0 05 Apr 2024
Deep Support Vectors Junhoo Lee Hyunho Lee Kyomin Hwang Nojun Kwak 46 0 0 26 Mar 2024
Posterior Uncertainty Quantification in Neural Networks using Data Augmentation Luhuan Wu Sinead Williamson UQCV 37 6 0 18 Mar 2024
Transfer Learning Beyond Bounded Density Ratios Alkis Kalavasis Ilias Zadik Manolis Zampetakis 47 4 0 18 Mar 2024
Implicit Regularization of Gradient Flow on One-Layer Softmax Attention Heejune Sheen Siyu Chen Tianhao Wang Harrison H. Zhou MLT 35 10 0 13 Mar 2024
Early Directional Convergence in Deep Homogeneous Neural Networks for Small Initializations Akshay Kumar Jarvis Haupt ODL 44 3 0 12 Mar 2024
Benign overfitting in leaky ReLU networks with moderate input dimension Kedar Karhadkar Erin E. George Michael Murray Guido Montúfar Deanna Needell MLT 43 2 0 11 Mar 2024
Effect of Ambient-Intrinsic Dimension Gap on Adversarial Vulnerability Rajdeep Haldar Yue Xing Qifan Song 32 3 0 06 Mar 2024
Supervised Contrastive Representation Learning: Landscape Analysis with Unconstrained Features Tina Behnia Christos Thrampoulidis SSL 36 0 0 29 Feb 2024
Learning Associative Memories with Gradient Descent Vivien A. Cabannes Berfin Simsek A. Bietti 38 6 0 28 Feb 2024
DualView: Data Attribution from the Dual Perspective Galip Umit Yolcu Thomas Wiegand Wojciech Samek Sebastian Lapuschkin TDI FAtt 21 0 0 19 Feb 2024