The Surprising Harmfulness of Benign Overfitting for Adversarial Robustness

19 January 2024

Tong Zhang

Papers citing "The Surprising Harmfulness of Benign Overfitting for Adversarial Robustness"

47 / 47 papers shown

Title
Double Descent Meets Out-of-Distribution Detection: Theoretical Insights and Empirical Analysis on the role of model complexity Mouin Ben Ammar David Brellmann Arturo Mendoza Antoine Manzanera Gianni Franchi OODD 84 0 0 04 Nov 2024
More is Better in Modern Machine Learning: when Infinite Overparameterization is Optimal and Overfitting is Obligatory James B. Simon Dhruva Karkada Nikhil Ghosh Mikhail Belkin AI4CE BDL 81 14 0 24 Nov 2023
Benign Overfitting in Deep Neural Networks under Lazy Training Zhenyu Zhu Fanghui Liu Grigorios G. Chrysos Francesco Locatello Volkan Cevher AI4CE 41 10 0 30 May 2023
The curse of overparametrization in adversarial training: Precise analysis of robust generalization for random features regression Hamed Hassani Adel Javanmard AAML 26 36 0 13 Jan 2022
Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks Hanxun Huang Yisen Wang S. Erfani Quanquan Gu James Bailey Xingjun Ma AAML TPM 86 102 0 07 Oct 2021
Interpolation can hurt robust generalization even when there is no noise Konstantin Donhauser Alexandru cTifrea Michael Aerni Reinhard Heckel Fanny Yang 59 15 0 05 Aug 2021
Benign Overfitting in Multiclass Classification: All Roads Lead to Interpolation Ke Wang Vidya Muthukumar Christos Thrampoulidis 51 49 0 21 Jun 2021
Towards an Understanding of Benign Overfitting in Neural Networks Zhu Li Zhi Zhou Arthur Gretton MLT 79 35 0 06 Jun 2021
A Universal Law of Robustness via Isoperimetry Sébastien Bubeck Mark Sellke 38 218 0 26 May 2021
Provable Robustness of Adversarial Training for Learning Halfspaces with Noise Difan Zou Spencer Frei Quanquan Gu 33 13 0 19 Apr 2021
Do Wider Neural Networks Really Help Adversarial Robustness? Boxi Wu Jinghui Chen Deng Cai Xiaofei He Quanquan Gu AAML 53 95 0 03 Oct 2020
A law of robustness for two-layers neural networks Sébastien Bubeck Yuanzhi Li Dheeraj M. Nagaraj 62 57 0 30 Sep 2020
The Neural Tangent Kernel in High Dimensions: Triple Descent and a Multi-Scale Theory of Generalization Ben Adlam Jeffrey Pennington 49 125 0 15 Aug 2020
Classification vs regression in overparameterized regimes: Does the loss function matter? Vidya Muthukumar Adhyyan Narang Vignesh Subramanian M. Belkin Daniel J. Hsu A. Sahai 81 151 0 16 May 2020
Finite-sample Analysis of Interpolating Linear Classifiers in the Overparameterized Regime Niladri S. Chatterji Philip M. Long 41 108 0 25 Apr 2020
Overfitting in adversarially robust deep learning Leslie Rice Eric Wong Zico Kolter 94 801 0 26 Feb 2020
Precise Tradeoffs in Adversarial Training for Linear Regression Adel Javanmard Mahdi Soltanolkotabi Hamed Hassani AAML 50 107 0 24 Feb 2020
Deep Double Descent: Where Bigger Models and More Data Hurt Preetum Nakkiran Gal Kaplun Yamini Bansal Tristan Yang Boaz Barak Ilya Sutskever 119 940 0 04 Dec 2019
Convergence of Adversarial Training in Overparametrized Neural Networks Ruiqi Gao Tianle Cai Haochuan Li Liwei Wang Cho-Jui Hsieh Jason D. Lee AAML 94 109 0 19 Jun 2019
Adversarial Training Can Hurt Generalization Aditi Raghunathan Sang Michael Xie Fanny Yang John C. Duchi Percy Liang 82 242 0 14 Jun 2019
Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks Yuan Cao Quanquan Gu MLT AI4CE 80 389 0 30 May 2019
Adversarial Examples Are Not Bugs, They Are Features Andrew Ilyas Shibani Santurkar Dimitris Tsipras Logan Engstrom Brandon Tran Aleksander Madry SILM 89 1,839 0 06 May 2019
Harmless interpolation of noisy data in regression Vidya Muthukumar Kailas Vodrahalli Vignesh Subramanian A. Sahai 74 202 0 21 Mar 2019
Surprises in High-Dimensional Ridgeless Least Squares Interpolation Trevor Hastie Andrea Montanari Saharon Rosset Robert Tibshirani 184 743 0 19 Mar 2019
Two models of double descent for weak features M. Belkin Daniel J. Hsu Ji Xu 90 374 0 18 Mar 2019
Theoretically Principled Trade-off between Robustness and Accuracy Hongyang R. Zhang Yaodong Yu Jiantao Jiao Eric Xing L. Ghaoui Michael I. Jordan 129 2,549 0 24 Jan 2019
Reconciling modern machine learning practice and the bias-variance trade-off M. Belkin Daniel J. Hsu Siyuan Ma Soumik Mandal 227 1,647 0 28 Dec 2018
Are adversarial examples inevitable? Ali Shafahi Wenjie Huang Christoph Studer Soheil Feizi Tom Goldstein SILM 62 282 0 06 Sep 2018
Just Interpolate: Kernel "Ridgeless" Regression Can Generalize Tengyuan Liang Alexander Rakhlin 60 353 0 01 Aug 2018
Neural Tangent Kernel: Convergence and Generalization in Neural Networks Arthur Jacot Franck Gabriel Clément Hongler 267 3,195 0 20 Jun 2018
On the adversarial robustness of robust estimators Lifeng Lai Erhan Bayraktar 33 10 0 11 Jun 2018
Robustness May Be at Odds with Accuracy Dimitris Tsipras Shibani Santurkar Logan Engstrom Alexander Turner Aleksander Madry AAML 99 1,778 0 30 May 2018
Adversarially Robust Generalization Requires More Data Ludwig Schmidt Shibani Santurkar Dimitris Tsipras Kunal Talwar Aleksander Madry OOD AAML 131 790 0 30 Apr 2018
To understand deep learning we need to understand kernel learning M. Belkin Siyuan Ma Soumik Mandal 60 419 0 05 Feb 2018
Evasion Attacks against Machine Learning at Test Time Battista Biggio Igino Corona Davide Maiorca B. Nelson Nedim Srndic Pavel Laskov Giorgio Giacinto Fabio Roli AAML 155 2,151 0 21 Aug 2017
Exploring Generalization in Deep Learning Behnam Neyshabur Srinadh Bhojanapalli David A. McAllester Nathan Srebro FAtt 146 1,255 0 27 Jun 2017
Towards Deep Learning Models Resistant to Adversarial Attacks Aleksander Madry Aleksandar Makelov Ludwig Schmidt Dimitris Tsipras Adrian Vladu SILM OOD 304 12,063 0 19 Jun 2017
The Marginal Value of Adaptive Gradient Methods in Machine Learning Ashia Wilson Rebecca Roelofs Mitchell Stern Nathan Srebro Benjamin Recht ODL 58 1,030 0 23 May 2017
Understanding deep learning requires rethinking generalization Chiyuan Zhang Samy Bengio Moritz Hardt Benjamin Recht Oriol Vinyals HAI 336 4,626 0 10 Nov 2016
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 421 2,937 0 15 Sep 2016
Path-SGD: Path-Normalized Optimization in Deep Neural Networks Behnam Neyshabur Ruslan Salakhutdinov Nathan Srebro ODL 79 307 0 08 Jun 2015
Explaining the Success of AdaBoost and Random Forests as Interpolating Classifiers A. Wyner Matthew A. Olson J. Bleich David Mease 87 269 0 28 Apr 2015
In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning Behnam Neyshabur Ryota Tomioka Nathan Srebro AI4CE 90 657 0 20 Dec 2014
Explaining and Harnessing Adversarial Examples Ian Goodfellow Jonathon Shlens Christian Szegedy AAML GAN 274 19,049 0 20 Dec 2014
Intriguing properties of neural networks Christian Szegedy Wojciech Zaremba Ilya Sutskever Joan Bruna D. Erhan Ian Goodfellow Rob Fergus AAML 268 14,918 1 21 Dec 2013
Margins, Shrinkage, and Boosting Matus Telgarsky 80 73 0 18 Mar 2013
The spectrum of kernel random matrices N. Karoui 151 224 0 04 Jan 2010