Coherent Gradients: An Approach to Understanding Generalization in Gradient Descent-based Optimization

25 February 2020

Papers citing "Coherent Gradients: An Approach to Understanding Generalization in Gradient Descent-based Optimization"

24 / 24 papers shown

Title
Enhanced Sample Selection with Confidence Tracking: Identifying Correctly Labeled yet Hard-to-Learn Samples in Noisy Data Weiran Pan Wei Wei Feida Zhu Yong Deng NoLa 400 0 0 24 Apr 2025
Leveraging Gradients for Unsupervised Accuracy Estimation under Distribution Shift Renchunzi Xie Ambroise Odonnat Vasilii Feofanov I. Redko Jianfeng Zhang Bo An UQCV 106 1 0 17 Jan 2024
Circuit-Based Intrinsic Methods to Detect Overfitting S. Chatterjee A. Mishchenko 23 6 0 03 Jul 2019
Bad Global Minima Exist and SGD Can Reach Them Shengchao Liu Dimitris Papailiopoulos D. Achlioptas 42 80 0 06 Jun 2019
SGD on Neural Networks Learns Functions of Increasing Complexity Preetum Nakkiran Gal Kaplun Dimitris Kalimeris Tristan Yang Benjamin L. Edelman Fred Zhang Boaz Barak MLT 122 242 0 28 May 2019
The Impact of Neural Network Overparameterization on Gradient Confusion and Stochastic Gradient Descent Karthik A. Sankararaman Soham De Zheng Xu Wenjie Huang Tom Goldstein ODL 47 103 0 15 Apr 2019
Uniform convergence may be unable to explain generalization in deep learning Vaishnavh Nagarajan J. Zico Kolter MoMe AI4CE 39 314 0 13 Feb 2019
Stiffness: A New Perspective on Generalization in Neural Networks Stanislav Fort Pawel Krzysztof Nowak Stanislaw Jastrzebski S. Narayanan 114 94 0 28 Jan 2019
A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks Umut Simsekli Levent Sagun Mert Gurbuzbalaban 75 241 0 18 Jan 2019
Reconciling modern machine learning practice and the bias-variance trade-off M. Belkin Daniel J. Hsu Siyuan Ma Soumik Mandal 164 1,628 0 28 Dec 2018
On the Spectral Bias of Neural Networks Nasim Rahaman A. Baratin Devansh Arpit Felix Dräxler Min Lin Fred Hamprecht Yoshua Bengio Aaron Courville 81 1,408 0 22 Jun 2018
Deep learning generalizes because the parameter-function map is biased towards simple functions Guillermo Valle Pérez Chico Q. Camargo A. Louis MLT AI4CE 40 231 0 22 May 2018
The Global Optimization Geometry of Shallow Linear Neural Networks Zhihui Zhu Daniel Soudry Yonina C. Eldar M. Wakin ODL 38 36 0 13 May 2018
Stronger generalization bounds for deep nets via a compression approach Sanjeev Arora Rong Ge Behnam Neyshabur Yi Zhang MLT AI4CE 64 637 0 14 Feb 2018
Generalization in Deep Learning Kenji Kawaguchi L. Kaelbling Yoshua Bengio ODL 61 459 0 16 Oct 2017
Spectrally-normalized margin bounds for neural networks Peter L. Bartlett Dylan J. Foster Matus Telgarsky ODL 116 1,208 0 26 Jun 2017
A Closer Look at Memorization in Deep Networks Devansh Arpit Stanislaw Jastrzebski Nicolas Ballas David M. Krueger Emmanuel Bengio ... Tegan Maharaj Asja Fischer Aaron Courville Yoshua Bengio Simon Lacoste-Julien TDI 90 1,801 0 16 Jun 2017
Deep Learning is Robust to Massive Label Noise David Rolnick Andreas Veit Serge J. Belongie Nir Shavit NoLa 57 553 0 30 May 2017
Sharp Minima Can Generalize For Deep Nets Laurent Dinh Razvan Pascanu Samy Bengio Yoshua Bengio ODL 90 763 0 15 Mar 2017
Understanding deep learning requires rethinking generalization Chiyuan Zhang Samy Bengio Moritz Hardt Benjamin Recht Oriol Vinyals HAI 241 4,612 0 10 Nov 2016
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 345 2,913 0 15 Sep 2016
Deep Learning with Differential Privacy Martín Abadi Andy Chu Ian Goodfellow H. B. McMahan Ilya Mironov Kunal Talwar Li Zhang FedML SyDa 143 6,049 0 01 Jul 2016
Train faster, generalize better: Stability of stochastic gradient descent Moritz Hardt Benjamin Recht Y. Singer 80 1,234 0 03 Sep 2015
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 223 1,189 0 30 Nov 2014