Implicit Regularization in Deep Learning

6 September 2017

Papers citing "Implicit Regularization in Deep Learning"

41 / 41 papers shown

Title
High-entropy Advantage in Neural Networks' Generalizability Entao Yang Wei Wei Yue Shang Ge Zhang AI4CE 92 0 0 17 Mar 2025
Critical Influence of Overparameterization on Sharpness-aware Minimization Sungbin Shin Dongyeop Lee Maksym Andriushchenko Namhoon Lee AAML 95 2 0 29 Nov 2023
Asynchronous Graph Generator Christopher P. Ley Felipe Tobar AI4TS 86 0 0 29 Sep 2023
Extrapolation for Large-batch Training in Deep Learning Tao R. Lin Lingjing Kong Sebastian U. Stich Martin Jaggi 74 36 0 10 Jun 2020
Can Implicit Bias Explain Generalization? Stochastic Convex Optimization as a Case Study Assaf Dauber M. Feder Tomer Koren Roi Livni 58 24 0 13 Mar 2020
Spectrally-normalized margin bounds for neural networks Peter L. Bartlett Dylan J. Foster Matus Telgarsky ODL 199 1,217 0 26 Jun 2017
Implicit Regularization in Matrix Factorization Suriya Gunasekar Blake E. Woodworth Srinadh Bhojanapalli Behnam Neyshabur Nathan Srebro 77 491 0 25 May 2017
Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data Gintare Karolina Dziugaite Daniel M. Roy 106 813 0 31 Mar 2017
Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks Peter L. Bartlett Nick Harvey Christopher Liaw Abbas Mehrabian 203 431 0 08 Mar 2017
Understanding deep learning requires rethinking generalization Chiyuan Zhang Samy Bengio Moritz Hardt Benjamin Recht Oriol Vinyals HAI 336 4,625 0 10 Nov 2016
Entropy-SGD: Biasing Gradient Descent Into Wide Valleys Pratik Chaudhari A. Choromańska Stefano Soatto Yann LeCun Carlo Baldassi C. Borgs J. Chayes Levent Sagun R. Zecchina ODL 96 773 0 06 Nov 2016
Generalization Error of Invariant Classifiers Jure Sokolić Raja Giryes Guillermo Sapiro M. Rodrigues 59 78 0 14 Oct 2016
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 421 2,936 0 15 Sep 2016
Path-Normalized Optimization of Recurrent Neural Networks with ReLU Activations Behnam Neyshabur Yuhuai Wu Ruslan Salakhutdinov Nathan Srebro AI4CE ODL 56 30 0 23 May 2016
Regularizing RNNs by Stabilizing Activations David M. Krueger Roland Memisevic 71 80 0 26 Nov 2015
Data-Dependent Path Normalization in Neural Networks Behnam Neyshabur Ryota Tomioka Ruslan Salakhutdinov Nathan Srebro 59 22 0 20 Nov 2015
Unitary Evolution Recurrent Neural Networks Martín Arjovsky Amar Shah Yoshua Bengio ODL 75 769 0 20 Nov 2015
Improving performance of recurrent neural network with relu nonlinearity S. Talathi Aniket A. Vartak ODL 56 88 0 12 Nov 2015
Natural Neural Networks Guillaume Desjardins Karen Simonyan Razvan Pascanu Koray Kavukcuoglu 105 176 0 01 Jul 2015
Path-SGD: Path-Normalized Optimization in Deep Neural Networks Behnam Neyshabur Ruslan Salakhutdinov Nathan Srebro ODL 79 307 0 08 Jun 2015
A Simple Way to Initialize Recurrent Networks of Rectified Linear Units Quoc V. Le Navdeep Jaitly Geoffrey E. Hinton ODL 86 719 0 03 Apr 2015
Optimizing Neural Networks with Kronecker-factored Approximate Curvature James Martens Roger C. Grosse ODL 101 1,013 0 19 Mar 2015
Norm-Based Capacity Control in Neural Networks Behnam Neyshabur Ryota Tomioka Nathan Srebro 290 587 0 27 Feb 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 463 43,289 0 11 Feb 2015
Breaking the Curse of Dimensionality with Convex Neural Networks Francis R. Bach 180 706 0 30 Dec 2014
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 1.8K 150,039 0 22 Dec 2014
In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning Behnam Neyshabur Ryota Tomioka Nathan Srebro AI4CE 90 657 0 20 Dec 2014
On the Computational Efficiency of Training Neural Networks Roi Livni Shai Shalev-Shwartz Ohad Shamir 143 480 0 05 Oct 2014
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 1.6K 100,348 0 04 Sep 2014
Deep Learning in Neural Networks: An Overview Jürgen Schmidhuber HAI 243 16,354 0 30 Apr 2014
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks Andrew M. Saxe James L. McClelland Surya Ganguli ODL 165 1,844 0 20 Dec 2013
From average case complexity to improper learning complexity Amit Daniely N. Linial Shai Shalev-Shwartz 115 120 0 10 Nov 2013
Riemannian metrics for neural networks II: recurrent networks and learning symbolic data sequences Yann Ollivier 55 17 0 03 Jun 2013
Maxout Networks Ian Goodfellow David Warde-Farley M. Berk Mirza Aaron Courville Yoshua Bengio OOD 238 2,178 0 18 Feb 2013
Regularization and nonlinearities for neural language models: when are they needed? Marius Pachitariu M. Sahani 78 46 0 23 Jan 2013
Revisiting Natural Gradient for Deep Networks Razvan Pascanu Yoshua Bengio ODL 134 389 0 16 Jan 2013
No More Pesky Learning Rates Tom Schaul Sixin Zhang Yann LeCun 137 478 0 06 Jun 2012
Krylov Subspace Descent for Deep Learning Oriol Vinyals Daniel Povey ODL 71 148 0 18 Nov 2011
On the Universality of Online Mirror Descent Nathan Srebro Karthik Sridharan Ambuj Tewari 175 145 0 20 Jul 2011
Robustness and Generalization Huan Xu Shie Mannor OOD 188 461 0 13 May 2010
Collaborative Filtering in a Non-Uniform World: Learning with the Weighted Trace Norm Ruslan Salakhutdinov Nathan Srebro 152 235 0 14 Feb 2010