Train faster, generalize better: Stability of stochastic gradient descent

3 September 2015

Benjamin Recht

Papers citing "Train faster, generalize better: Stability of stochastic gradient descent"

50 / 275 papers shown

Title
Stability and Generalization of Graph Convolutional Neural Networks Saurabh Verma Zhi-Li Zhang GNN MLT 30 154 0 03 May 2019
Deep Multi-View Learning using Neuron-Wise Correlation-Maximizing Regularizers Kui Jia Jiehong Lin Mingkui Tan Dacheng Tao 3DV 32 32 0 25 Apr 2019
Implicit regularization for deep neural networks driven by an Ornstein-Uhlenbeck like process Guy Blanc Neha Gupta Gregory Valiant Paul Valiant 27 144 0 19 Apr 2019
A Selective Overview of Deep Learning Jianqing Fan Cong Ma Yiqiao Zhong BDL VLM 38 136 0 10 Apr 2019
Positively Scale-Invariant Flatness of ReLU Neural Networks Mingyang Yi Qi Meng Wei-neng Chen Zhi-Ming Ma Tie-Yan Liu 26 17 0 06 Mar 2019
Implicit Regularization in Over-parameterized Neural Networks M. Kubo Ryotaro Banno Hidetaka Manabe Masataka Minoji 25 23 0 05 Mar 2019
High probability generalization bounds for uniformly stable algorithms with nearly optimal rate Vitaly Feldman J. Vondrák 35 154 0 27 Feb 2019
Beating SGD Saturation with Tail-Averaging and Minibatching Nicole Mücke Gergely Neu Lorenzo Rosasco 20 35 0 22 Feb 2019
An Empirical Study of Large-Batch Stochastic Gradient Descent with Structured Covariance Noise Yeming Wen Kevin Luk Maxime Gazeau Guodong Zhang Harris Chan Jimmy Ba ODL 25 22 0 21 Feb 2019
Stable and Fair Classification Lingxiao Huang Nisheeth K. Vishnoi FaML 24 71 0 21 Feb 2019
Uniform convergence may be unable to explain generalization in deep learning Vaishnavh Nagarajan J. Zico Kolter MoMe AI4CE 17 310 0 13 Feb 2019
Quasi-Newton Methods for Machine Learning: Forget the Past, Just Sample A. Berahas Majid Jahani Peter Richtárik Martin Takávc 24 40 0 28 Jan 2019
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruosong Wang MLT 55 961 0 24 Jan 2019
Veridical Data Science Bin Yu Karl Kumbier 23 162 0 23 Jan 2019
Overfitting Mechanism and Avoidance in Deep Neural Networks Shaeke Salman Xiuwen Liu 17 139 0 19 Jan 2019
Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks Zhi-Qin John Xu Tao Luo Yaoyu Zhang Yan Xiao Zheng Ma 40 504 0 19 Jan 2019
Generalization in Deep Networks: The Role of Distance from Initialization Vaishnavh Nagarajan J. Zico Kolter ODL 15 95 0 07 Jan 2019
Stagewise Training Accelerates Convergence of Testing Error Over SGD Zhuoning Yuan Yan Yan Rong Jin Tianbao Yang 60 11 0 10 Dec 2018
Deep Frank-Wolfe For Neural Network Optimization Leonard Berrada Andrew Zisserman M. P. Kumar ODL 21 40 0 19 Nov 2018
Gradient Descent Finds Global Minima of Deep Neural Networks S. Du Jason D. Lee Haochuan Li Liwei Wang Masayoshi Tomizuka ODL 44 1,126 0 09 Nov 2018
Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel Colin Wei Jason D. Lee Qiang Liu Tengyu Ma 28 245 0 12 Oct 2018
Graph-Dependent Implicit Regularisation for Distributed Stochastic Subgradient Descent Dominic Richards Patrick Rebeschini 16 18 0 18 Sep 2018
On the Generalization of Stochastic Gradient Descent with Momentum Ali Ramezani-Kebrya Kimon Antonakopoulos V. Cevher Ashish Khisti Ben Liang MLT 19 24 0 12 Sep 2018
Understanding training and generalization in deep learning by Fourier analysis Zhi-Qin John Xu AI4CE 24 92 0 13 Aug 2018
Generalization Error in Deep Learning Daniel Jakubovitz Raja Giryes M. Rodrigues AI4CE 32 109 0 03 Aug 2018
Laplacian Smoothing Gradient Descent Stanley Osher Bao Wang Penghang Yin Xiyang Luo Farzin Barekat Minh Pham A. Lin ODL 22 43 0 17 Jun 2018
PAC-Bayes Control: Learning Policies that Provably Generalize to Novel Environments Anirudha Majumdar M. Goldstein Anoopkumar Sonar 23 18 0 11 Jun 2018
Training Faster by Separating Modes of Variation in Batch-normalized Models Mahdi M. Kalayeh M. Shah 27 42 0 07 Jun 2018
Stable Recurrent Models John Miller Moritz Hardt 19 116 0 25 May 2018
Deep learning generalizes because the parameter-function map is biased towards simple functions Guillermo Valle Pérez Chico Q. Camargo A. Louis MLT AI4CE 18 226 0 22 May 2018
Measuring and regularizing networks in function space Ari S. Benjamin David Rolnick Konrad Paul Kording 21 138 0 21 May 2018
Stochastic modified equations for the asynchronous stochastic gradient descent Jing An Jian-wei Lu Lexing Ying 21 79 0 21 May 2018
Constrained-CNN losses for weakly supervised segmentation H. Kervadec Jose Dolz Meng Tang Eric Granger Yuri Boykov Ismail Ben Ayed 27 239 0 12 May 2018
Privacy-preserving Prediction Cynthia Dwork Vitaly Feldman 25 90 0 27 Mar 2018
Constrained Deep Learning using Conditional Gradient and Applications in Computer Vision Sathya Ravi Tuan Dinh Vishnu Suresh Lokhande Vikas Singh AI4CE 33 22 0 17 Mar 2018
On the Power of Over-parametrization in Neural Networks with Quadratic Activation S. Du Jason D. Lee 27 267 0 03 Mar 2018
A Walk with SGD Chen Xing Devansh Arpit Christos Tsirigotis Yoshua Bengio 27 118 0 24 Feb 2018
Stronger generalization bounds for deep nets via a compression approach Sanjeev Arora Rong Ge Behnam Neyshabur Yi Zhang MLT AI4CE 41 631 0 14 Feb 2018
Towards Understanding the Generalization Bias of Two Layer Convolutional Linear Classifiers with Gradient Descent Yifan Wu Barnabás Póczós Aarti Singh MLT 30 8 0 13 Feb 2018
Learning Compact Neural Networks with Regularization Samet Oymak MLT 41 39 0 05 Feb 2018
Generalization Error Bounds for Noisy, Iterative Algorithms Ankit Pensia Varun Jog Po-Ling Loh 29 109 0 12 Jan 2018
Improving Generalization Performance by Switching from Adam to SGD N. Keskar R. Socher ODL 41 521 0 20 Dec 2017
Convergence of Unregularized Online Learning Algorithms Yunwen Lei Lei Shi Zheng-Chu Guo 22 14 0 09 Aug 2017
Regularizing and Optimizing LSTM Language Models Stephen Merity N. Keskar R. Socher 95 1,091 0 07 Aug 2017
Exploring Generalization in Deep Learning Behnam Neyshabur Srinadh Bhojanapalli David A. McAllester Nathan Srebro FAtt 110 1,239 0 27 Jun 2017
Stochastic Training of Neural Networks via Successive Convex Approximations Simone Scardapane P. Di Lorenzo 24 9 0 15 Jun 2017
Recovery Guarantees for One-hidden-layer Neural Networks Kai Zhong Zhao Song Prateek Jain Peter L. Bartlett Inderjit S. Dhillon MLT 34 336 0 10 Jun 2017
Are Saddles Good Enough for Deep Learning? Adepu Ravi Sankar V. Balasubramanian 43 5 0 07 Jun 2017
Classification regions of deep neural networks Alhussein Fawzi Seyed-Mohsen Moosavi-Dezfooli P. Frossard Stefano Soatto 31 51 0 26 May 2017
Train longer, generalize better: closing the generalization gap in large batch training of neural networks Elad Hoffer Itay Hubara Daniel Soudry ODL 44 795 0 24 May 2017