SGD Generalizes Better Than GD (And Regularization Doesn't Help)

1 February 2021

Papers citing "SGD Generalizes Better Than GD (And Regularization Doesn't Help)"

35 / 35 papers shown

Title
Rapid Overfitting of Multi-Pass Stochastic Gradient Descent in Stochastic Convex Optimization Shira Vansover-Hager Tomer Koren Roi Livni 39 0 0 13 May 2025
The Double Descent Behavior in Two Layer Neural Network for Binary Classification Chathurika S Abeykoon A. Beknazaryan Hailin Sang 53 1 0 27 Apr 2025
Beyond Propagation of Chaos: A Stochastic Algorithm for Mean Field Optimization Chandan Tankala Dheeraj M. Nagaraj Anant Raj 44 0 0 17 Mar 2025
The Implicit Bias of Gradient Descent on Separable Multiclass Data Hrithik Ravi Clayton Scott Daniel Soudry Yutong Wang 45 2 0 02 Nov 2024
LoRA-GA: Low-Rank Adaptation with Gradient Approximation Shaowen Wang Linxi Yu Jian Li ALM AI4CE 31 30 0 06 Jul 2024
The Sample Complexity of Gradient Descent in Stochastic Convex Optimization Roi Livni MLT 39 1 0 07 Apr 2024
Information Complexity of Stochastic Convex Optimization: Applications to Generalization and Memorization Idan Attias Gintare Karolina Dziugaite Mahdi Haghifam Roi Livni Daniel M. Roy 30 6 0 14 Feb 2024
Convex SGD: Generalization Without Early Stopping Julien Hendrickx A. Olshevsky MLT LRM 25 1 0 08 Jan 2024
The Sample Complexity Of ERMs In Stochastic Convex Optimization Dan Carmon Roi Livni Amir Yehudayoff 19 3 0 09 Nov 2023
Generalization Bounds for Label Noise Stochastic Gradient Descent Jung Eun Huh Patrick Rebeschini 13 1 0 01 Nov 2023
Semantic Segmentation of Porosity in 4D Spatio-Temporal X-ray μCT of Titanium Coated Ni wires using Deep Learning Pradyumna Elavarthi Arun J. Bhattacharjee A. P. Y. Puente Anca L. Ralescu 27 0 0 24 Jun 2023
Improved Stability and Generalization Guarantees of the Decentralized SGD Algorithm B. L. Bars A. Bellet Marc Tommasi Kevin Scaman Giovanni Neglia 24 1 0 05 Jun 2023
Select without Fear: Almost All Mini-Batch Schedules Generalize Optimally Konstantinos E. Nikolakakis Amin Karbasi Dionysis Kalogerias 33 5 0 03 May 2023
The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learning Micah Goldblum Marc Finzi K. Rowan A. Wilson UQCV FedML 26 38 0 11 Apr 2023
Domain Generalization with Adversarial Intensity Attack for Medical Image Segmentation Zheyu Zhang Bin Wang Lanhong Yao Ugur Demir Debesh Jha I. Turkbey Boqing Gong Ulas Bagci AAML MedIm OOD 32 11 0 05 Apr 2023
Comparison between layer-to-layer network training and conventional network training using Deep Convolutional Neural Networks Kiran Kumar Ashish Bhyravabhottla WonSook Lee 16 0 0 27 Mar 2023
MagicEye: An Intelligent Wearable Towards Independent Living of Visually Impaired S. C. Sethuraman G. R. Tadkapally Saraju Mohanty Gautam Galada Anitha Subramanian 17 2 0 24 Mar 2023
Lower Generalization Bounds for GD and SGD in Smooth Stochastic Convex Optimization Peiyuan Zhang Jiaye Teng J.N. Zhang 39 4 0 19 Mar 2023
Information Theoretic Lower Bounds for Information Theoretic Upper Bounds Roi Livni 21 14 0 09 Feb 2023
Limitations of Information-Theoretic Generalization Bounds for Gradient Descent Methods in Stochastic Convex Optimization Mahdi Haghifam Borja Rodríguez Gálvez Ragnar Thobaben Mikael Skoglund Daniel M. Roy Gintare Karolina Dziugaite 31 17 0 27 Dec 2022
On the Overlooked Structure of Stochastic Gradients Zeke Xie Qian-Yuan Tang Mingming Sun P. Li 31 6 0 05 Dec 2022
Stability and Generalization Analysis of Gradient Methods for Shallow Neural Networks Yunwen Lei Rong Jin Yiming Ying MLT 40 18 0 19 Sep 2022
$Generalization Bounds for Stochastic Gradient Descent via Localized $\varepsilon$-Covers$ Generalization Bounds for Stochastic Gradient Descent via Localized $\varepsilon$ -Covers Sejun Park Umut Simsekli Murat A. Erdogdu 43 9 0 19 Sep 2022
Trajectory-dependent Generalization Bounds for Deep Neural Networks via Fractional Brownian Motion Chengli Tan Jiang Zhang Junmin Liu 43 1 0 09 Jun 2022
Differentially Private Generalized Linear Models Revisited R. Arora Raef Bassily Cristóbal Guzmán Michael Menart Enayat Ullah FedML 28 16 0 06 May 2022
Making Progress Based on False Discoveries Roi Livni 38 0 0 19 Apr 2022
Benign Underfitting of Stochastic Gradient Descent Tomer Koren Roi Livni Yishay Mansour Uri Sherman MLT 20 13 0 27 Feb 2022
Thinking Outside the Ball: Optimal Learning with Gradient Descent for Generalized Linear Stochastic Convex Optimization I Zaghloul Amir Roi Livni Nathan Srebro 30 6 0 27 Feb 2022
Black-Box Generalization: Stability of Zeroth-Order Learning Konstantinos E. Nikolakakis Farzin Haddadpour Dionysios S. Kalogerias Amin Karbasi MLT 24 2 0 14 Feb 2022
Stochastic Training is Not Necessary for Generalization Jonas Geiping Micah Goldblum Phillip E. Pope Michael Moeller Tom Goldstein 89 72 0 29 Sep 2021
Improved Learning Rates for Stochastic Optimization: Two Theoretical Viewpoints Shaojie Li Yong Liu 26 13 0 19 Jul 2021
SGD: The Role of Implicit Regularization, Batch-size and Multiple-epochs Satyen Kale Ayush Sekhari Karthik Sridharan 193 29 0 11 Jul 2021
Never Go Full Batch (in Stochastic Convex Optimization) I Zaghloul Amir Y. Carmon Tomer Koren Roi Livni 37 14 0 29 Jun 2021
Stability and Deviation Optimal Risk Bounds with Convergence Rate $O(1/n)$ Yegor Klochkov Nikita Zhivotovskiy 31 61 0 22 Mar 2021
A simpler approach to obtaining an O(1/t) convergence rate for the projected stochastic subgradient method Simon Lacoste-Julien Mark W. Schmidt Francis R. Bach 128 259 0 10 Dec 2012