v1v2 (latest)

Lower Generalization Bounds for GD and SGD in Smooth Stochastic Convex Optimization

19 March 2023

Papers citing "Lower Generalization Bounds for GD and SGD in Smooth Stochastic Convex Optimization"

35 / 35 papers shown

Title
Generalization and Stability of Interpolating Neural Networks with Minimal Width Hossein Taheri Christos Thrampoulidis 95 16 0 18 Feb 2023
Max-Margin Works while Large Margin Fails: Generalization without Uniform Convergence Margalit Glasgow Colin Wei Mary Wootters Tengyu Ma 92 5 0 16 Jun 2022
Dimension Independent Generalization of DP-SGD for Overparameterized Smooth Convex Optimization Yi-An Ma T. V. Marinov Tong Zhang 66 8 0 03 Jun 2022
Beyond Lipschitz: Sharp Generalization and Excess Risk Bounds for Full-Batch GD Konstantinos E. Nikolakakis Farzin Haddadpour Amin Karbasi Dionysios S. Kalogerias 123 19 0 26 Apr 2022
Stability vs Implicit Bias of Gradient Methods on Separable Data and Beyond Matan Schliserman Tomer Koren 62 24 0 27 Feb 2022
Optimistic Rates: A Unifying Theory for Interpolation Learning and Regularization in Linear Regression Lijia Zhou Frederic Koehler Danica J. Sutherland Nathan Srebro 131 25 0 08 Dec 2021
Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias Kaifeng Lyu Zhiyuan Li Runzhe Wang Sanjeev Arora MLT 95 76 0 26 Oct 2021
SGD: The Role of Implicit Regularization, Batch-size and Multiple-epochs Satyen Kale Ayush Sekhari Karthik Sridharan 240 29 0 11 Jul 2021
PAC-Bayes, MAC-Bayes and Conditional Mutual Information: Fast rate bounds that handle general VC classes Peter Grünwald Thomas Steinke Lydia Zakynthinou 75 30 0 17 Jun 2021
Towards Understanding Generalization via Decomposing Excess Risk Dynamics Jiaye Teng Jianhao Ma Yang Yuan 51 4 0 11 Jun 2021
Stability of SGD: Tightness Analysis and Improved Bounds Yikai Zhang Wenjia Zhang Sammy Bald Vamsi Pingali Chao Chen Mayank Goswami MLT 51 38 0 10 Feb 2021
SGD Generalizes Better Than GD (And Regularization Doesn't Help) I Zaghloul Amir Tomer Koren Roi Livni 54 46 0 01 Feb 2021
Fine-Grained Analysis of Stability and Generalization for Stochastic Gradient Descent Yunwen Lei Yiming Ying MLT 93 129 0 15 Jun 2020
Stability of Stochastic Gradient Descent on Nonsmooth Convex Losses Raef Bassily Vitaly Feldman Cristóbal Guzmán Kunal Talwar MLT 61 198 0 12 Jun 2020
Generalization error in high-dimensional perceptrons: Approaching Bayes error with convex optimization Benjamin Aubin Florent Krzakala Yue M. Lu Lenka Zdeborová 68 56 0 11 Jun 2020
Improving Generalization by Controlling Label-Noise Information in Neural Network Weights Hrayr Harutyunyan Kyle Reing Greg Ver Steeg Aram Galstyan NoLa 57 56 0 19 Feb 2020
Fantastic Generalization Measures and Where to Find Them Yiding Jiang Behnam Neyshabur H. Mobahi Dilip Krishnan Samy Bengio AI4CE 145 611 0 04 Dec 2019
How does Early Stopping Help Generalization against Label Noise? Hwanjun Song Minseok Kim Dongmin Park Jae-Gil Lee NoLa 87 76 0 19 Nov 2019
Private Stochastic Convex Optimization with Optimal Rates Raef Bassily Vitaly Feldman Kunal Talwar Abhradeep Thakurta 87 246 0 27 Aug 2019
Fast-rate PAC-Bayes Generalization Bounds via Shifted Rademacher Processes Jun Yang Shengyang Sun Daniel M. Roy 65 28 0 20 Aug 2019
Benign Overfitting in Linear Regression Peter L. Bartlett Philip M. Long Gábor Lugosi Alexander Tsigler MLT 105 779 0 26 Jun 2019
High probability generalization bounds for uniformly stable algorithms with nearly optimal rate Vitaly Feldman J. Vondrák 72 155 0 27 Feb 2019
Uniform convergence may be unable to explain generalization in deep learning Vaishnavh Nagarajan J. Zico Kolter MoMe AI4CE 90 317 0 13 Feb 2019
Fast and Faster Convergence of SGD for Over-Parameterized Models and an Accelerated Perceptron Sharan Vaswani Francis R. Bach Mark Schmidt 102 301 0 16 Oct 2018
Statistical Optimality of Stochastic Gradient Descent on Hard Learning Problems through Multiple Passes Loucas Pillaud-Vivien Alessandro Rudi Francis R. Bach 152 103 0 25 May 2018
Exponential convergence of testing error for stochastic gradient methods Loucas Pillaud-Vivien Alessandro Rudi Francis R. Bach 72 32 0 13 Dec 2017
The Implicit Bias of Gradient Descent on Separable Data Daniel Soudry Elad Hoffer Mor Shpigel Nacson Suriya Gunasekar Nathan Srebro 178 924 0 27 Oct 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 808 132,725 0 12 Jun 2017
Train longer, generalize better: closing the generalization gap in large batch training of neural networks Elad Hoffer Itay Hubara Daniel Soudry ODL 185 800 0 24 May 2017
On the Exponentially Weighted Aggregate with the Laplace Prior A. Dalalyan Edwin Grappin Q. Paris 47 19 0 25 Nov 2016
Densely Connected Convolutional Networks Gao Huang Zhuang Liu Laurens van der Maaten Kilian Q. Weinberger PINN 3DV 852 36,910 0 25 Aug 2016
Generalization of ERM in Stochastic Convex Optimization: The Dimension Strikes Back Vitaly Feldman 58 69 0 15 Aug 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.3K 194,641 0 10 Dec 2015
Train faster, generalize better: Stability of stochastic gradient descent Moritz Hardt Benjamin Recht Y. Singer 118 1,243 0 03 Sep 2015
Fast rates for support vector machines using Gaussian kernels Ingo Steinwart C. Scovel 295 283 0 14 Aug 2007