Gradient Descent Maximizes the Margin of Homogeneous Neural Networks

13 June 2019

Papers citing "Gradient Descent Maximizes the Margin of Homogeneous Neural Networks"

50 / 246 papers shown

Title
Stochastic linear optimization never overfits with quadratically-bounded losses on general data Matus Telgarsky 11 11 0 14 Feb 2022
Benign Overfitting in Two-layer Convolutional Neural Networks Yuan Cao Zixiang Chen M. Belkin Quanquan Gu MLT 19 83 0 14 Feb 2022
The Sample Complexity of One-Hidden-Layer Neural Networks Gal Vardi Ohad Shamir Nathan Srebro 22 10 0 13 Feb 2022
Towards Data-Algorithm Dependent Generalization: a Case Study on Overparameterized Linear Regression Jing Xu Jiaye Teng Yang Yuan Andrew Chi-Chih Yao 23 1 0 12 Feb 2022
Gradient Methods Provably Converge to Non-Robust Networks Gal Vardi Gilad Yehudai Ohad Shamir 30 27 0 09 Feb 2022
The Implicit Bias of Gradient Descent on Generalized Gated Linear Networks Samuel Lippl L. F. Abbott SueYeon Chung MLT AI4CE 14 2 0 05 Feb 2022
Implicit Regularization Towards Rank Minimization in ReLU Networks Nadav Timor Gal Vardi Ohad Shamir 34 49 0 30 Jan 2022
Training invariances and the low-rank phenomenon: beyond linear networks Thien Le Stefanie Jegelka 20 29 0 28 Jan 2022
Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks Noam Razin Asaf Maman Nadav Cohen 46 29 0 27 Jan 2022
Global Convergence Analysis of Deep Linear Networks with A One-neuron Layer Kun Chen Dachao Lin Zhihua Zhang 14 1 0 08 Jan 2022
Benign Overfitting in Adversarially Robust Linear Classification Jinghui Chen Yuan Cao Quanquan Gu AAML SILM 34 10 0 31 Dec 2021
Is Importance Weighting Incompatible with Interpolating Classifiers? Ke Alexander Wang Niladri S. Chatterji Saminul Haque Tatsunori Hashimoto 24 20 0 24 Dec 2021
Understanding Square Loss in Training Overparametrized Neural Network Classifiers Tianyang Hu Jun Wang Wenjia Wang Zhenguo Li UQCV AAML 41 19 0 07 Dec 2021
On the Equivalence between Neural Network and Support Vector Machine Yilan Chen Wei Huang Lam M. Nguyen Tsui-Wei Weng AAML 22 18 0 11 Nov 2021
Scaffolding Sets M. Burhanpurkar Zhun Deng Cynthia Dwork Linjun Zhang 35 9 0 04 Nov 2021
Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias Kaifeng Lyu Zhiyuan Li Runzhe Wang Sanjeev Arora MLT 34 69 0 26 Oct 2021
Rethinking Neural vs. Matrix-Factorization Collaborative Filtering: the Theoretical Perspectives Zida Cheng Chuanwei Ruan Siheng Chen Sushant Kumar Ya-Qin Zhang 22 16 0 23 Oct 2021
Actor-critic is implicitly biased towards high entropy optimal policies Yuzheng Hu Ziwei Ji Matus Telgarsky 57 11 0 21 Oct 2021
What Happens after SGD Reaches Zero Loss? --A Mathematical Framework Zhiyuan Li Tianhao Wang Sanjeev Arora MLT 90 98 0 13 Oct 2021
The Convex Geometry of Backpropagation: Neural Network Gradient Flows Converge to Extreme Points of the Dual Convex Program Yifei Wang Mert Pilanci MLT MDE 55 11 0 13 Oct 2021
Implicit Bias of Linear Equivariant Networks Hannah Lawrence Kristian Georgiev A. Dienes B. Kiani AI4CE 40 14 0 12 Oct 2021
Does Momentum Change the Implicit Regularization on Separable Data? Bohan Wang Qi Meng Huishuai Zhang Ruoyu Sun Wei Chen Zhirui Ma Tie-Yan Liu 47 15 0 08 Oct 2021
Tighter Sparse Approximation Bounds for ReLU Neural Networks Carles Domingo-Enrich Youssef Mroueh 99 4 0 07 Oct 2021
An Unconstrained Layer-Peeled Perspective on Neural Collapse Wenlong Ji Yiping Lu Yiliang Zhang Zhun Deng Weijie J. Su 135 83 0 06 Oct 2021
On Margin Maximization in Linear and ReLU Networks Gal Vardi Ohad Shamir Nathan Srebro 50 28 0 06 Oct 2021
Logit Attenuating Weight Normalization Aman Gupta R. Ramanath Jun Shi Anika Ramachandran Sirou Zhou Mingzhou Zhou S. Keerthi 37 1 0 12 Aug 2021
Interpolation can hurt robust generalization even when there is no noise Konstantin Donhauser Alexandru cTifrea Michael Aerni Reinhard Heckel Fanny Yang 34 14 0 05 Aug 2021
Distribution of Classification Margins: Are All Data Equal? Andrzej Banburski Fernanda De La Torre Nishka Pant Ishana Shastri T. Poggio 28 4 0 21 Jul 2021
Continuous vs. Discrete Optimization of Deep Neural Networks Omer Elkabetz Nadav Cohen 68 44 0 14 Jul 2021
A Theoretical Analysis of Fine-tuning with Linear Teachers Gal Shachaf Alon Brutzkus Amir Globerson 34 17 0 04 Jul 2021
Fast Margin Maximization via Dual Acceleration Ziwei Ji Nathan Srebro Matus Telgarsky 15 35 0 01 Jul 2021
Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsity Arthur Jacot François Ged Berfin cSimcsek Clément Hongler Franck Gabriel 29 52 0 30 Jun 2021
Can contrastive learning avoid shortcut solutions? Joshua Robinson Li Sun Ke Yu Kayhan Batmanghelich Stefanie Jegelka S. Sra SSL 19 142 0 21 Jun 2021
Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity Scott Pesme Loucas Pillaud-Vivien Nicolas Flammarion 27 99 0 17 Jun 2021
Understanding Deflation Process in Over-parametrized Tensor Decomposition Rong Ge Y. Ren Xiang Wang Mo Zhou 8 17 0 11 Jun 2021
Early-stopped neural networks are consistent Ziwei Ji Justin D. Li Matus Telgarsky 14 36 0 10 Jun 2021
Improved OOD Generalization via Adversarial Training and Pre-training Mingyang Yi Lu Hou Jiacheng Sun Lifeng Shang Xin Jiang Qun Liu Zhi-Ming Ma VLM 28 83 0 24 May 2021
Properties of the After Kernel Philip M. Long 24 29 0 21 May 2021
Convergence and Implicit Bias of Gradient Flow on Overparametrized Linear Networks Hancheng Min Salma Tarmoun René Vidal Enrique Mallada MLT 11 4 0 13 May 2021
Directional Convergence Analysis under Spherically Symmetric Distribution Dachao Lin Zhihua Zhang MLT 12 0 0 09 May 2021
A Geometric Analysis of Neural Collapse with Unconstrained Features Zhihui Zhu Tianyu Ding Jinxin Zhou Xiao Li Chong You Jeremias Sulam Qing Qu 27 194 0 06 May 2021
Risk Bounds for Over-parameterized Maximum Margin Classification on Sub-Gaussian Mixtures Yuan Cao Quanquan Gu M. Belkin 11 51 0 28 Apr 2021
Achieving Small Test Error in Mildly Overparameterized Neural Networks Shiyu Liang Ruoyu Sun R. Srikant 20 3 0 24 Apr 2021
Understanding the role of importance weighting for deep learning Da Xu Yuting Ye Chuanwei Ruan FAtt 36 43 0 28 Mar 2021
How to decay your learning rate Aitor Lewkowycz 41 24 0 23 Mar 2021
Inductive Bias of Multi-Channel Linear Convolutional Networks with Bounded Weight Norm Meena Jagadeesan Ilya P. Razenshteyn Suriya Gunasekar 11 21 0 24 Feb 2021
On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descent Shahar Azulay E. Moroshko Mor Shpigel Nacson Blake E. Woodworth Nathan Srebro Amir Globerson Daniel Soudry AI4CE 33 73 0 19 Feb 2021
Bridging the Gap Between Adversarial Robustness and Optimization Bias Fartash Faghri Sven Gowal C. N. Vasconcelos David J. Fleet Fabian Pedregosa Nicolas Le Roux AAML 192 7 0 17 Feb 2021
Noisy Recurrent Neural Networks S. H. Lim N. Benjamin Erichson Liam Hodgkinson Michael W. Mahoney 14 52 0 09 Feb 2021
Towards Understanding Learning in Neural Networks with Linear Teachers Roei Sarussi Alon Brutzkus Amir Globerson FedML MLT 55 20 0 07 Jan 2021