The Implicit Bias of Gradient Descent on Separable Data

27 October 2017

Papers citing "The Implicit Bias of Gradient Descent on Separable Data"

50 / 240 papers shown

Title
Scaling Laws Under the Microscope: Predicting Transformer Performance from Small Scale Experiments Maor Ivgi Y. Carmon Jonathan Berant 19 17 0 13 Feb 2022
On Regularizing Coordinate-MLPs Sameera Ramasinghe L. MacDonald Simon Lucey 158 5 0 01 Feb 2022
Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks Noam Razin Asaf Maman Nadav Cohen 46 29 0 27 Jan 2022
Implicit Bias of MSE Gradient Optimization in Underparameterized Neural Networks Benjamin Bowman Guido Montúfar 28 11 0 12 Jan 2022
Benign Overfitting in Adversarially Robust Linear Classification Jinghui Chen Yuan Cao Quanquan Gu AAML SILM 34 10 0 31 Dec 2021
On the Impact of Hard Adversarial Instances on Overfitting in Adversarial Training Chen Liu Zhichao Huang Mathieu Salzmann Tong Zhang Sabine Süsstrunk AAML 23 13 0 14 Dec 2021
Variational autoencoders in the presence of low-dimensional data: landscape and implicit bias Frederic Koehler Viraj Mehta Chenghui Zhou Andrej Risteski DRL 41 12 0 13 Dec 2021
The perils of being unhinged: On the accuracy of classifiers minimizing a noise-robust convex loss Philip M. Long Rocco A. Servedio 19 2 0 08 Dec 2021
On the Convergence of Shallow Neural Network Training with Randomly Masked Neurons Fangshuo Liao Anastasios Kyrillidis 49 16 0 05 Dec 2021
Depth Without the Magic: Inductive Bias of Natural Gradient Descent A. Kerekes Anna Mészáros Ferenc Huszár ODL 37 4 0 22 Nov 2021
Deep Probability Estimation Sheng Liu Aakash Kaku Weicheng Zhu M. Leibovich S. Mohan ... Haoxiang Huang L. Zanna N. Razavian Jonathan Niles-Weed C. Fernandez‐Granda UQCV OOD 28 14 0 21 Nov 2021
On the Equivalence between Neural Network and Support Vector Machine Yilan Chen Wei Huang Lam M. Nguyen Tsui-Wei Weng AAML 25 18 0 11 Nov 2021
Mean-field Analysis of Piecewise Linear Solutions for Wide ReLU Networks A. Shevchenko Vyacheslav Kungurtsev Marco Mondelli MLT 44 13 0 03 Nov 2021
Regularization by Misclassification in ReLU Neural Networks Elisabetta Cornacchia Jan Hązła Ido Nachum Amir Yehudayoff NoLa 25 2 0 03 Nov 2021
Simple data balancing achieves competitive worst-group-accuracy Badr Youbi Idrissi Martín Arjovsky Mohammad Pezeshki David Lopez-Paz 51 173 0 27 Oct 2021
Rethinking Neural vs. Matrix-Factorization Collaborative Filtering: the Theoretical Perspectives Zida Cheng Chuanwei Ruan Siheng Chen Sushant Kumar Ya Zhang 27 16 0 23 Oct 2021
Understanding Dimensional Collapse in Contrastive Self-supervised Learning Li Jing Pascal Vincent Yann LeCun Yuandong Tian SSL 25 338 0 18 Oct 2021
Towards Understanding the Data Dependency of Mixup-style Training Muthuraman Chidambaram Xiang Wang Yuzheng Hu Chenwei Wu Rong Ge UQCV 49 24 0 14 Oct 2021
The Convex Geometry of Backpropagation: Neural Network Gradient Flows Converge to Extreme Points of the Dual Convex Program Yifei Wang Mert Pilanci MLT MDE 55 11 0 13 Oct 2021
Implicit Bias of Linear Equivariant Networks Hannah Lawrence Kristian Georgiev A. Dienes B. Kiani AI4CE 40 14 0 12 Oct 2021
Imitating Deep Learning Dynamics via Locally Elastic Stochastic Differential Equations Jiayao Zhang Hua Wang Weijie J. Su 35 7 0 11 Oct 2021
Large Learning Rate Tames Homogeneity: Convergence and Balancing Effect Yuqing Wang Minshuo Chen T. Zhao Molei Tao AI4CE 57 40 0 07 Oct 2021
On Margin Maximization in Linear and ReLU Networks Gal Vardi Ohad Shamir Nathan Srebro 52 28 0 06 Oct 2021
Spectral Bias in Practice: The Role of Function Frequency in Generalization Sara Fridovich-Keil Raphael Gontijo-Lopes Rebecca Roelofs 41 28 0 06 Oct 2021
A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning Yehuda Dar Vidya Muthukumar Richard G. Baraniuk 34 71 0 06 Sep 2021
Logit Attenuating Weight Normalization Aman Gupta R. Ramanath Jun Shi Anika Ramachandran Sirou Zhou Mingzhou Zhou S. Keerthi 40 1 0 12 Aug 2021
On the Power of Differentiable Learning versus PAC and SQ Learning Emmanuel Abbe Pritish Kamath Eran Malach Colin Sandon Nathan Srebro MLT 77 23 0 09 Aug 2021
Unified Regularity Measures for Sample-wise Learning and Generalization Chi Zhang Xiaoning Ma Yu Liu Le Wang Yuanqi Su Yuehu Liu 39 1 0 09 Aug 2021
Interpolation can hurt robust generalization even when there is no noise Konstantin Donhauser Alexandru cTifrea Michael Aerni Reinhard Heckel Fanny Yang 34 14 0 05 Aug 2021
Can contrastive learning avoid shortcut solutions? Joshua Robinson Li Sun Ke Yu Kayhan Batmanghelich Stefanie Jegelka S. Sra SSL 19 142 0 21 Jun 2021
Deep Learning Through the Lens of Example Difficulty R. Baldock Hartmut Maennel Behnam Neyshabur 47 156 0 17 Jun 2021
What can linearized neural networks actually say about generalization? Guillermo Ortiz-Jiménez Seyed-Mohsen Moosavi-Dezfooli P. Frossard 29 43 0 12 Jun 2021
Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization Kartik Ahuja Ethan Caballero Dinghuai Zhang Jean-Christophe Gagnon-Audet Yoshua Bengio Ioannis Mitliagkas Irina Rish OOD 15 251 0 11 Jun 2021
From inexact optimization to learning via gradient concentration Bernhard Stankewitz Nicole Mücke Lorenzo Rosasco 23 5 0 09 Jun 2021
Can Subnetwork Structure be the Key to Out-of-Distribution Generalization? Dinghuai Zhang Kartik Ahuja Yilun Xu Yisen Wang Aaron Courville OOD 22 95 0 05 Jun 2021
Improved OOD Generalization via Adversarial Training and Pre-training Mingyang Yi Lu Hou Jiacheng Sun Lifeng Shang Xin Jiang Qun Liu Zhi-Ming Ma VLM 31 83 0 24 May 2021
A Geometric Analysis of Neural Collapse with Unconstrained Features Zhihui Zhu Tianyu Ding Jinxin Zhou Xiao Li Chong You Jeremias Sulam Qing Qu 40 196 0 06 May 2021
Two-layer neural networks with values in a Banach space Yury Korolev 29 23 0 05 May 2021
AdaBoost and robust one-bit compressed sensing Geoffrey Chinot Felix Kuchelmeister Matthias Löffler Sara van de Geer 35 5 0 05 May 2021
RATT: Leveraging Unlabeled Data to Guarantee Generalization Saurabh Garg Sivaraman Balakrishnan J. Zico Kolter Zachary Chase Lipton 30 30 0 01 May 2021
Risk Bounds for Over-parameterized Maximum Margin Classification on Sub-Gaussian Mixtures Yuan Cao Quanquan Gu M. Belkin 11 51 0 28 Apr 2021
Understanding the role of importance weighting for deep learning Da Xu Yuting Ye Chuanwei Ruan FAtt 39 43 0 28 Mar 2021
The Low-Rank Simplicity Bias in Deep Networks Minyoung Huh H. Mobahi Richard Y. Zhang Brian Cheung Pulkit Agrawal Phillip Isola 27 109 0 18 Mar 2021
On complementing end-to-end human behavior predictors with planning Liting Sun Xiaogang Jia Anca Dragan 35 17 0 09 Mar 2021
Label-Imbalanced and Group-Sensitive Classification under Overparameterization Ganesh Ramachandra Kini Orestis Paraskevas Samet Oymak Christos Thrampoulidis 29 93 0 02 Mar 2021
Do Input Gradients Highlight Discriminative Features? Harshay Shah Prateek Jain Praneeth Netrapalli AAML FAtt 26 57 0 25 Feb 2021
Dissecting Supervised Contrastive Learning Florian Graf Christoph Hofer Marc Niethammer Roland Kwitt SSL 117 70 0 17 Feb 2021
Convergence of stochastic gradient descent schemes for Lojasiewicz-landscapes Steffen Dereich Sebastian Kassing 34 27 0 16 Feb 2021
Connecting Interpretability and Robustness in Decision Trees through Separation Michal Moshkovitz Yao-Yuan Yang Kamalika Chaudhuri 33 22 0 14 Feb 2021
Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Training Cong Fang Hangfeng He Qi Long Weijie J. Su FAtt 130 167 0 29 Jan 2021