Train faster, generalize better: Stability of stochastic gradient descent

3 September 2015

Benjamin Recht

Papers citing "Train faster, generalize better: Stability of stochastic gradient descent"

50 / 264 papers shown

Title
Online Learning and Unlearning Yaxi Hu Bernhard Schölkopf Amartya Sanyal MU OnRL 58 0 0 13 May 2025
Rapid Overfitting of Multi-Pass Stochastic Gradient Descent in Stochastic Convex Optimization Shira Vansover-Hager Tomer Koren Roi Livni 39 0 0 13 May 2025
Stability Regularized Cross-Validation Ryan Cory-Wright A. Gómez 31 0 0 11 May 2025
Gradient Descent as a Shrinkage Operator for Spectral Bias Simon Lucey 38 0 0 25 Apr 2025
NeuralGrok: Accelerate Grokking by Neural Gradient Transformation Xinyu Zhou Simin Fan Martin Jaggi Jie Fu 41 0 0 24 Apr 2025
Leave-One-Out Stable Conformal Prediction Kiljae Lee Yuan Zhang 52 0 0 16 Apr 2025
Better Rates for Random Task Orderings in Continual Linear Models Itay Evron Ran Levinstein Matan Schliserman Uri Sherman Tomer Koren Daniel Soudry Nathan Srebro CLL 40 0 0 06 Apr 2025
Randomized Pairwise Learning with Adaptive Sampling: A PAC-Bayes Analysis Sijia Zhou Yunwen Lei Ata Kabán 34 0 0 03 Apr 2025
Learning Variational Inequalities from Data: Fast Generalization Rates under Strong Monotonicity Eric Zhao Tatjana Chavdarova Michael I. Jordan 50 0 0 20 Feb 2025
Stability-based Generalization Bounds for Variational Inference Yadi Wei Roni Khardon BDL 49 0 0 17 Feb 2025
Understanding the Generalization Error of Markov algorithms through Poissonization Benjamin Dupuis Maxime Haddouche George Deligiannidis Umut Simsekli 49 0 0 11 Feb 2025
Understanding Generalization of Federated Learning: the Trade-off between Model Stability and Optimization Dun Zeng Zheshun Wu Shiyu Liu Yu Pan Xiaoying Tang Zenglin Xu MLT FedML 89 1 0 25 Nov 2024
Understanding Generalization in Quantum Machine Learning with Margins Tak Hur Daniel K. Park AI4CE 39 1 0 11 Nov 2024
Nonlinear Stochastic Gradient Descent and Heavy-tailed Noise: A Unified Framework and High-probability Guarantees Aleksandar Armacki Shuhua Yu Pranay Sharma Gauri Joshi Dragana Bajović D. Jakovetić S. Kar 59 2 0 17 Oct 2024
Sharper Guarantees for Learning Neural Network Classifiers with Gradient Methods Hossein Taheri Christos Thrampoulidis Arya Mazumdar MLT 36 0 0 13 Oct 2024
OledFL: Unleashing the Potential of Decentralized Federated Learning via Opposite Lookahead Enhancement Qinglun Li Miao Zhang Mengzhu Wang Quanjun Yin Li Shen OODD FedML 26 0 0 09 Oct 2024
How Much Can We Forget about Data Contamination? Sebastian Bordt Suraj Srinivas Valentyn Boreiko U. V. Luxburg 54 1 0 04 Oct 2024
A-FedPD: Aligning Dual-Drift is All Federated Primal-Dual Learning Needs Yan Sun Li Shen Dacheng Tao FedML 32 0 0 27 Sep 2024
Loss Gradient Gaussian Width based Generalization and Optimization Guarantees A. Banerjee Qiaobo Li Yingxue Zhou 52 0 0 11 Jun 2024
Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance Dimitris Oikonomou Nicolas Loizou 55 4 0 06 Jun 2024
Uniformly Stable Algorithms for Adversarial Training and Beyond Jiancong Xiao Jiawei Zhang Zhimin Luo Asuman Ozdaglar AAML 48 0 0 03 May 2024
The Sample Complexity of Gradient Descent in Stochastic Convex Optimization Roi Livni MLT 39 1 0 07 Apr 2024
Statistical Mechanics and Artificial Neural Networks: Principles, Models, and Applications Lucas Böttcher Gregory R. Wheeler 32 0 0 05 Apr 2024
AdaBatchGrad: Combining Adaptive Batch Size and Adaptive Step Size P. Ostroukhov Aigerim Zhumabayeva Chulu Xiang Alexander Gasnikov Martin Takáč Dmitry Kamzolov ODL 46 2 0 07 Feb 2024
Leveraging Gradients for Unsupervised Accuracy Estimation under Distribution Shift Renchunzi Xie Ambroise Odonnat Vasilii Feofanov I. Redko Jianfeng Zhang Bo An UQCV 80 1 0 17 Jan 2024
Convex SGD: Generalization Without Early Stopping Julien Hendrickx A. Olshevsky MLT LRM 25 1 0 08 Jan 2024
SoK: Unintended Interactions among Machine Learning Defenses and Risks Vasisht Duddu S. Szyller Nadarajah Asokan AAML 49 2 0 07 Dec 2023
Generalization Bounds for Label Noise Stochastic Gradient Descent Jung Eun Huh Patrick Rebeschini 13 1 0 01 Nov 2023
Initialization Matters: Privacy-Utility Analysis of Overparameterized Neural Networks Jiayuan Ye Zhenyu Zhu Fanghui Liu Reza Shokri V. Cevher 34 12 0 31 Oct 2023
Stability and Generalization of the Decentralized Stochastic Gradient Descent Ascent Algorithm Miaoxi Zhu Li Shen Bo Du Dacheng Tao 26 6 0 31 Oct 2023
Optimal Guarantees for Algorithmic Reproducibility and Gradient Complexity in Convex Optimization Liang Zhang Junchi Yang Amin Karbasi Niao He 34 2 0 26 Oct 2023
Demystifying the Myths and Legends of Nonconvex Convergence of SGD Aritra Dutta El Houcine Bergou Soumia Boucherouite Nicklas Werge M. Kandemir Xin Li 28 0 0 19 Oct 2023
Online Estimation with Rolling Validation: Adaptive Nonparametric Estimation with Streaming Data Tianyu Zhang Jing Lei 13 1 0 18 Oct 2023
Differentially Private Non-convex Learning for Multi-layer Neural Networks Hanpu Shen Cheng-Long Wang Zihang Xiang Yiming Ying Di Wang 51 7 0 12 Oct 2023
Adversarial Style Transfer for Robust Policy Optimization in Deep Reinforcement Learning Md Masudur Rahman Yexiang Xue 29 4 0 29 Aug 2023
Stability and Generalization of Stochastic Compositional Gradient Descent Algorithms Minghao Yang Xiyuan Wei Tianbao Yang Yiming Ying 50 1 0 07 Jul 2023
Nonconvex Stochastic Bregman Proximal Gradient Method with Application to Deep Learning Kuan-Fu Ding Jingyang Li Kim-Chuan Toh 33 8 0 26 Jun 2023
Repeated Random Sampling for Minimizing the Time-to-Accuracy of Learning Patrik Okanovic R. Waleffe Vasilis Mageirakos Konstantinos E. Nikolakakis Amin Karbasi Dionysis Kalogerias Nezihe Merve Gürel Theodoros Rekatsinas DD 55 12 0 28 May 2023
Generalization Guarantees of Gradient Descent for Multi-Layer Neural Networks Puyu Wang Yunwen Lei Di Wang Yiming Ying Ding-Xuan Zhou MLT 31 4 0 26 May 2023
Fast Convergence in Learning Two-Layer Neural Networks with Separable Data Hossein Taheri Christos Thrampoulidis MLT 16 3 0 22 May 2023
Stability and Generalization of lp-Regularized Stochastic Learning for GCN Shiyu Liu Linsen Wei Shaogao Lv Ming Li MLT 25 0 0 20 May 2023
Uniform-in-Time Wasserstein Stability Bounds for (Noisy) Stochastic Gradient Descent Lingjiong Zhu Mert Gurbuzbalaban Anant Raj Umut Simsekli 34 6 0 20 May 2023
Is Aggregation the Only Choice? Federated Learning via Layer-wise Model Recombination Ming Hu Zhihao Yue Zhiwei Ling Cheng Chen Yihao Huang Xian Wei Xiang Lian Yang Liu Mingsong Chen FedML 23 8 0 18 May 2023
Understanding the Generalization Ability of Deep Learning Algorithms: A Kernelized Renyi's Entropy Perspective Yuxin Dong Tieliang Gong Hao Chen Chen Li 23 4 0 02 May 2023
Learning Trajectories are Generalization Indicators Jingwen Fu Zhizheng Zhang Dacheng Yin Yan Lu Nanning Zheng AI4CE 36 3 0 25 Apr 2023
Differentially Private Stochastic Convex Optimization in (Non)-Euclidean Space Revisited Jinyan Su Changhong Zhao Di Wang 33 4 0 31 Mar 2023
Cyclic and Randomized Stepsizes Invoke Heavier Tails in SGD than Constant Stepsize Mert Gurbuzbalaban Yuanhan Hu Umut Simsekli Lingjiong Zhu LRM 23 1 0 10 Feb 2023
Generalization in Graph Neural Networks: Improved PAC-Bayesian Bounds on Graph Diffusion Haotian Ju Dongyue Li Aneesh Sharma Hongyang R. Zhang 31 40 0 09 Feb 2023
U-Clip: On-Average Unbiased Stochastic Gradient Clipping Bryn Elesedy Marcus Hutter 19 1 0 06 Feb 2023
Efficient Gradient Approximation Method for Constrained Bilevel Optimization Siyuan Xu Minghui Zhu 29 20 0 03 Feb 2023