v1v2v3v4 (latest)

Gradient Starvation: A Learning Proclivity in Neural Networks

18 November 2020

Aaron Courville

Papers citing "Gradient Starvation: A Learning Proclivity in Neural Networks"

41 / 91 papers shown

Title
Excessive Invariance Causes Adversarial Vulnerability J. Jacobsen Jens Behrmann R. Zemel Matthias Bethge AAML 68 166 0 01 Nov 2018
A mathematical theory of semantic development in deep neural networks Andrew M. Saxe James L. McClelland Surya Ganguli 73 271 0 23 Oct 2018
Gradient Descent Provably Optimizes Over-parameterized Neural Networks S. Du Xiyu Zhai Barnabás Póczós Aarti Singh MLT ODL 219 1,272 0 04 Oct 2018
An analytic theory of generalization dynamics and transfer learning in deep linear networks Andrew Kyle Lampinen Surya Ganguli OOD 82 131 0 27 Sep 2018
On the Learning Dynamics of Deep Neural Networks Rémi Tachet des Combes Mohammad Pezeshki Samira Shabanian Aaron Courville Yoshua Bengio 54 38 0 18 Sep 2018
Recognition in Terra Incognita Sara Beery Grant Van Horn Pietro Perona 92 849 0 13 Jul 2018
A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks Kimin Lee Kibok Lee Honglak Lee Jinwoo Shin OODD 187 2,051 0 10 Jul 2018
Training behavior of deep neural network in frequency domain Zhi-Qin John Xu Yaoyu Zhang Yan Xiao AI4CE 72 320 0 03 Jul 2018
Confounding variables can degrade generalization performance of radiological deep learning models J. Zech Marcus A. Badgeley Manway Liu A. Costa J. Titano Eric K. Oermann OOD 85 1,176 0 02 Jul 2018
On the Spectral Bias of Neural Networks Nasim Rahaman A. Baratin Devansh Arpit Felix Dräxler Min Lin Fred Hamprecht Yoshua Bengio Aaron Courville 152 1,439 0 22 Jun 2018
Neural Tangent Kernel: Convergence and Generalization in Neural Networks Arthur Jacot Franck Gabriel Clément Hongler 267 3,203 0 20 Jun 2018
Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balanced S. Du Wei Hu Jason D. Lee MLT 131 242 0 04 Jun 2018
Implicit Bias of Gradient Descent on Linear Convolutional Networks Suriya Gunasekar Jason D. Lee Daniel Soudry Nathan Srebro MDE 124 412 0 01 Jun 2018
Deep learning generalizes because the parameter-function map is biased towards simple functions Guillermo Valle Pérez Chico Q. Camargo A. Louis MLT AI4CE 85 231 0 22 May 2018
Gradient Descent for One-Hidden-Layer Neural Networks: Polynomial Convergence and SQ Lower Bounds Santosh Vempala John Wilmes MLT 84 51 0 07 May 2018
Black-box Adversarial Attacks with Limited Queries and Information Andrew Ilyas Logan Engstrom Anish Athalye Jessy Lin MLAU AAML 163 1,200 0 23 Apr 2018
Annotation Artifacts in Natural Language Inference Data Suchin Gururangan Swabha Swayamdipta Omer Levy Roy Schwartz Samuel R. Bowman Noah A. Smith 150 1,176 0 06 Mar 2018
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey Naveed Akhtar Ajmal Mian AAML 97 1,867 0 02 Jan 2018
Theory of Deep Learning III: explaining the non-overfitting puzzle T. Poggio Kenji Kawaguchi Q. Liao Brando Miranda Lorenzo Rosasco Xavier Boix Jack Hidary H. Mhaskar ODL 58 128 0 30 Dec 2017
Measuring the tendency of CNNs to Learn Surface Statistical Regularities Jason Jo Yoshua Bengio AAML 71 250 0 30 Nov 2017
Synthetic and Natural Noise Both Break Neural Machine Translation Yonatan Belinkov Yonatan Bisk 111 742 0 06 Nov 2017
The Implicit Bias of Gradient Descent on Separable Data Daniel Soudry Elad Hoffer Mor Shpigel Nacson Suriya Gunasekar Nathan Srebro 158 917 0 27 Oct 2017
SGD Learns Over-parameterized Networks that Provably Generalize on Linearly Separable Data Alon Brutzkus Amir Globerson Eran Malach Shai Shalev-Shwartz MLT 151 279 0 27 Oct 2017
High-dimensional dynamics of generalization error in neural networks Madhu S. Advani Andrew M. Saxe AI4CE 139 469 0 10 Oct 2017
Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints Jieyu Zhao Tianlu Wang Mark Yatskar Vicente Ordonez Kai-Wei Chang FaML 97 971 0 29 Jul 2017
Towards Deep Learning Models Resistant to Adversarial Attacks Aleksander Madry Aleksandar Makelov Ludwig Schmidt Dimitris Tsipras Adrian Vladu SILM OOD 307 12,069 0 19 Jun 2017
A Closer Look at Memorization in Deep Networks Devansh Arpit Stanislaw Jastrzebski Nicolas Ballas David M. Krueger Emmanuel Bengio ... Tegan Maharaj Asja Fischer Aaron Courville Yoshua Bengio Simon Lacoste-Julien TDI 125 1,818 0 16 Jun 2017
Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks Shiyu Liang Yixuan Li R. Srikant UQCV OODD 168 2,072 0 08 Jun 2017
Implicit Regularization in Matrix Factorization Suriya Gunasekar Blake E. Woodworth Srinadh Bhojanapalli Behnam Neyshabur Nathan Srebro 79 491 0 25 May 2017
Geometry of Optimization and Implicit Regularization in Deep Learning Behnam Neyshabur Ryota Tomioka Ruslan Salakhutdinov Nathan Srebro AI4CE 65 133 0 08 May 2017
Understanding deep learning requires rethinking generalization Chiyuan Zhang Samy Bengio Moritz Hardt Benjamin Recht Oriol Vinyals HAI 339 4,629 0 10 Nov 2016
A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks Dan Hendrycks Kevin Gimpel UQCV 158 3,454 0 07 Oct 2016
"Why Should I Trust You?": Explaining the Predictions of Any Classifier Marco Tulio Ribeiro Sameer Singh Carlos Guestrin FAtt FaML 1.2K 16,990 0 16 Feb 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 194,020 0 10 Dec 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 463 43,305 0 11 Feb 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 1.8K 150,115 0 22 Dec 2014
In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning Behnam Neyshabur Ryota Tomioka Nathan Srebro AI4CE 94 658 0 20 Dec 2014
Explaining and Harnessing Adversarial Examples Ian Goodfellow Jonathon Shlens Christian Szegedy AAML GAN 277 19,066 0 20 Dec 2014
Deep Learning Face Attributes in the Wild Ziwei Liu Ping Luo Xiaogang Wang Xiaoou Tang CVBM 244 8,408 0 28 Nov 2014
Intriguing properties of neural networks Christian Szegedy Wojciech Zaremba Ilya Sutskever Joan Bruna D. Erhan Ian Goodfellow Rob Fergus AAML 275 14,927 1 21 Dec 2013
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks Andrew M. Saxe James L. McClelland Surya Ganguli ODL 173 1,845 0 20 Dec 2013