v1v2 (latest)

Train faster, generalize better: Stability of stochastic gradient descent

3 September 2015

Moritz Hardt

Benjamin Recht

Y. Singer

ArXiv (abs)PDF HTML

Papers citing "Train faster, generalize better: Stability of stochastic gradient descent"

50 / 679 papers shown

Title
Implicit Regularization of Accelerated Methods in Hilbert Spaces Nicolò Pagliana Lorenzo Rosasco 103 18 0 30 May 2019
Where is the Information in a Deep Neural Network? Alessandro Achille Giovanni Paolini Stefano Soatto 101 82 0 29 May 2019
Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for Regression Problems Tianle Cai Ruiqi Gao Jikai Hou Siyu Chen Dong Wang Di He Zhihua Zhang Liwei Wang ODL 86 57 0 28 May 2019
Implicit Rugosity Regularization via Data Augmentation Daniel LeJeune Randall Balestriero Hamid Javadi Richard G. Baraniuk 83 4 0 28 May 2019
SGD on Neural Networks Learns Functions of Increasing Complexity Preetum Nakkiran Gal Kaplun Dimitris Kalimeris Tristan Yang Benjamin L. Edelman Fred Zhang Boaz Barak MLT 164 248 0 28 May 2019
Quantifying the generalization error in deep learning in terms of data distribution and neural network smoothness Pengzhan Jin Lu Lu Yifa Tang George Karniadakis 89 61 0 27 May 2019
Physics-informed Autoencoders for Lyapunov-stable Fluid Flow Prediction N. Benjamin Erichson Michael Muehlebach Michael W. Mahoney AI4CE PINN 75 141 0 26 May 2019
Explicitizing an Implicit Bias of the Frequency Principle in Two-layer Neural Networks Yaoyu Zhang Zhi-Qin John Xu Yaoyu Zhang Zheng Ma MLT AI4CE 130 38 0 24 May 2019
Blockwise Adaptivity: Faster Training and Better Generalization in Deep Learning Shuai Zheng James T. Kwok ODL 63 5 0 23 May 2019
Orthogonal Deep Neural Networks Kui Jia Shuai Li Yuxin Wen Tongliang Liu Dacheng Tao 102 134 0 15 May 2019
Budgeted Training: Rethinking Deep Neural Network Training Under Resource Constraints Mengtian Li Ersin Yumer Deva Ramanan 72 49 0 12 May 2019
Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentation Colin Wei Tengyu Ma 102 110 0 09 May 2019
Stochastic Iterative Hard Thresholding for Graph-structured Sparsity Optimization Baojian Zhou F. Chen Yiming Ying 80 7 0 09 May 2019
Stability and Generalization of Graph Convolutional Neural Networks Saurabh Verma Zhi-Li Zhang GNN MLT 131 159 0 03 May 2019
Differentiable Visual Computing Tzu-Mao Li 54 15 0 27 Apr 2019
Stability and Optimization Error of Stochastic Gradient Descent for Pairwise Learning Wei Shen Zhenhuan Yang Yiming Ying Xiaoming Yuan 54 17 0 25 Apr 2019
Deep Multi-View Learning using Neuron-Wise Correlation-Maximizing Regularizers Kui Jia Jiehong Lin Mingkui Tan Dacheng Tao 3DV 60 32 0 25 Apr 2019
Implicit regularization for deep neural networks driven by an Ornstein-Uhlenbeck like process Guy Blanc Neha Gupta Gregory Valiant Paul Valiant 179 147 0 19 Apr 2019
Connections Between Adaptive Control and Optimization in Machine Learning Joseph E. Gaudio T. Gibson Anuradha M. Annaswamy M. Bolender E. Lavretsky AI4CE 47 44 0 11 Apr 2019
A Selective Overview of Deep Learning Jianqing Fan Cong Ma Yiqiao Zhong BDL VLM 210 135 0 10 Apr 2019
Constrained Deep Networks: Lagrangian Optimization via Log-Barrier Extensions H. Kervadec Jose Dolz Jing Yuan Christian Desrosiers Eric Granger Ismail Ben Ayed 120 55 0 08 Apr 2019
Information Bottleneck and its Applications in Deep Learning Hassan Hafez-Kolahi S. Kasaei 71 19 0 07 Apr 2019
On the Stability and Generalization of Learning with Kernel Activation Functions M. Cirillo Simone Scardapane S. Van Vaerenbergh A. Uncini 25 0 0 28 Mar 2019
TATi-Thermodynamic Analytics ToolkIt: TensorFlow-based software for posterior sampling in machine learning applications Frederik Heber Zofia Trstanova Benedict Leimkuhler 40 1 0 20 Mar 2019
Theory III: Dynamics and Generalization in Deep Networks Andrzej Banburski Q. Liao Alycia Lee Lorenzo Rosasco Fernanda De La Torre Jack Hidary T. Poggio AI4CE 78 3 0 12 Mar 2019
An Exponential Efron-Stein Inequality for Lq Stable Learning Rules Karim Abou-Moustafa Csaba Szepesvári 80 11 0 12 Mar 2019
Positively Scale-Invariant Flatness of ReLU Neural Networks Mingyang Yi Qi Meng Wei-neng Chen Zhi-Ming Ma Tie-Yan Liu 76 18 0 06 Mar 2019
Implicit Regularization in Over-parameterized Neural Networks M. Kubo Ryotaro Banno Hidetaka Manabe Masataka Minoji 90 23 0 05 Mar 2019
SGD without Replacement: Sharper Rates for General Smooth Convex Functions Prateek Jain Dheeraj M. Nagaraj Praneeth Netrapalli 90 87 0 04 Mar 2019
Time-Delay Momentum: A Regularization Perspective on the Convergence and Generalization of Stochastic Momentum for Deep Learning Ziming Zhang Wenju Xu Alan Sullivan 110 1 0 02 Mar 2019
High probability generalization bounds for uniformly stable algorithms with nearly optimal rate Vitaly Feldman J. Vondrák 82 155 0 27 Feb 2019
Beating SGD Saturation with Tail-Averaging and Minibatching Nicole Mücke Gergely Neu Lorenzo Rosasco 106 37 0 22 Feb 2019
An Empirical Study of Large-Batch Stochastic Gradient Descent with Structured Covariance Noise Yeming Wen Kevin Luk Maxime Gazeau Guodong Zhang Harris Chan Jimmy Ba ODL 73 22 0 21 Feb 2019
Stable and Fair Classification Lingxiao Huang Nisheeth K. Vishnoi FaML 156 72 0 21 Feb 2019
Information Losses in Neural Classifiers from Sampling Brandon Foggo N. Yu Jie Shi Yuanqi Gao 63 7 0 15 Feb 2019
Uniform convergence may be unable to explain generalization in deep learning Vaishnavh Nagarajan J. Zico Kolter MoMe AI4CE 100 317 0 13 Feb 2019
Generalization Error Bounds of Gradient Descent for Learning Over-parameterized Deep ReLU Networks Yuan Cao Quanquan Gu ODL MLT AI4CE 171 159 0 04 Feb 2019
On Generalization Error Bounds of Noisy Gradient Methods for Non-Convex Learning Jian Li Xuanyuan Luo Mingda Qiao 82 89 0 02 Feb 2019
Natural Analysts in Adaptive Data Analysis Tijana Zrnic Moritz Hardt 80 17 0 30 Jan 2019
Quasi-Newton Methods for Machine Learning: Forget the Past, Just Sample A. Berahas Majid Jahani Peter Richtárik Martin Takávc 109 41 0 28 Jan 2019
SGD: General Analysis and Improved Rates Robert Mansel Gower Nicolas Loizou Xun Qian Alibek Sailanbayev Egor Shulgin Peter Richtárik 111 383 0 27 Jan 2019
On the cross-validation bias due to unsupervised pre-processing Amit Moscovich Saharon Rosset SSL 64 18 0 25 Jan 2019
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruosong Wang MLT 245 974 0 24 Jan 2019
Veridical Data Science Bin Yu Karl Kumbier 104 170 0 23 Jan 2019
Overfitting Mechanism and Avoidance in Deep Neural Networks Shaeke Salman Xiuwen Liu 63 145 0 19 Jan 2019
Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks Zhi-Qin John Xu Yaoyu Zhang Yaoyu Zhang Yan Xiao Zheng Ma 156 523 0 19 Jan 2019
Generalization in Deep Networks: The Role of Distance from Initialization Vaishnavh Nagarajan J. Zico Kolter ODL 98 96 0 07 Jan 2019
Improving Generalization of Deep Neural Networks by Leveraging Margin Distribution Shen-Huan Lyu Lu Wang Zhi Zhou 49 11 0 27 Dec 2018
Generalization Bounds for Uniformly Stable Algorithms Vitaly Feldman J. Vondrák 80 89 0 24 Dec 2018
Provable limitations of deep learning Emmanuel Abbe Colin Sandon AAML 85 45 0 16 Dec 2018