Stability of SGD: Tightness Analysis and Improved Bounds

10 February 2021

Yikai Zhang

Papers citing "Stability of SGD: Tightness Analysis and Improved Bounds"

26 / 26 papers shown

Title
A Theoretical Perspective: How to Prevent Model Collapse in Self-consuming Training Loops Shi Fu Yingjie Wang Yuzhu Chen Xinmei Tian Dacheng Tao 53 1 0 26 Feb 2025
Towards Auto-Regressive Next-Token Prediction: In-Context Learning Emerges from Generalization Zixuan Gong Xiaolin Hu Huayi Tang Yong Liu 38 0 0 24 Feb 2025
Boosting the Performance of Decentralized Federated Learning via Catalyst Acceleration Qinglun Li Miao Zhang Yingqi Liu Quanjun Yin Li Shen Xiaochun Cao FedML 44 0 0 09 Oct 2024
OledFL: Unleashing the Potential of Decentralized Federated Learning via Opposite Lookahead Enhancement Qinglun Li Miao Zhang Mengzhu Wang Quanjun Yin Li Shen OODD FedML 26 0 0 09 Oct 2024
Generalization Error Matters in Decentralized Learning Under Byzantine Attacks Haoxiang Ye Qing Ling 39 0 0 11 Jul 2024
Convex SGD: Generalization Without Early Stopping Julien Hendrickx A. Olshevsky MLT LRM 25 1 0 08 Jan 2024
Stability and Generalization for Minibatch SGD and Local SGD Yunwen Lei Tao Sun Mingrui Liu 32 3 0 02 Oct 2023
A Unified Framework for Generative Data Augmentation: A Comprehensive Survey Yunhao Chen Zihui Yan Yunjie Zhu 29 3 0 30 Sep 2023
Towards Understanding the Generalizability of Delayed Stochastic Gradient Descent Xiaoge Deng Li Shen Shengwei Li Tao Sun Dongsheng Li Dacheng Tao 28 3 0 18 Aug 2023
Minimax Excess Risk of First-Order Methods for Statistical Learning with Data-Dependent Oracles Kevin Scaman Mathieu Even B. L. Bars Laurent Massoulié 14 1 0 10 Jul 2023
Understanding How Consistency Works in Federated Learning via Stage-wise Relaxed Initialization Yan Sun Li Shen Dacheng Tao FedML 20 14 0 09 Jun 2023
Improved Stability and Generalization Guarantees of the Decentralized SGD Algorithm B. L. Bars A. Bellet Marc Tommasi Kevin Scaman Giovanni Neglia 24 1 0 05 Jun 2023
AI Liability Insurance With an Example in AI-Powered E-diagnosis System Yunfei Ge Quanyan Zhu 16 0 0 01 Jun 2023
Toward Understanding Generative Data Augmentation Chenyu Zheng Guoqiang Wu Chongxuan Li 29 25 0 27 May 2023
Uniform-in-Time Wasserstein Stability Bounds for (Noisy) Stochastic Gradient Descent Lingjiong Zhu Mert Gurbuzbalaban Anant Raj Umut Simsekli 34 6 0 20 May 2023
Select without Fear: Almost All Mini-Batch Schedules Generalize Optimally Konstantinos E. Nikolakakis Amin Karbasi Dionysis Kalogerias 33 5 0 03 May 2023
Learning Trajectories are Generalization Indicators Jingwen Fu Zhizheng Zhang Dacheng Yin Yan Lu Nanning Zheng AI4CE 33 3 0 25 Apr 2023
Lower Generalization Bounds for GD and SGD in Smooth Stochastic Convex Optimization Peiyuan Zhang Jiaye Teng J.N. Zhang 39 4 0 19 Mar 2023
On the generalization of learning algorithms that do not converge N. Chandramoorthy Andreas Loukas Khashayar Gatmiry Stefanie Jegelka MLT 19 11 0 16 Aug 2022
Beyond Lipschitz: Sharp Generalization and Excess Risk Bounds for Full-Batch GD Konstantinos E. Nikolakakis Farzin Haddadpour Amin Karbasi Dionysios S. Kalogerias 43 17 0 26 Apr 2022
Risk Bounds of Multi-Pass SGD for Least Squares in the Interpolation Regime Difan Zou Jingfeng Wu Vladimir Braverman Quanquan Gu Sham Kakade 16 5 0 07 Mar 2022
On the bias of K-fold cross validation with stable learners Anass Aghbalou François Portier Anne Sabourin 19 6 0 21 Feb 2022
Black-Box Generalization: Stability of Zeroth-Order Learning Konstantinos E. Nikolakakis Farzin Haddadpour Dionysios S. Kalogerias Amin Karbasi MLT 24 2 0 14 Feb 2022
Improved Learning Rates for Stochastic Optimization: Two Theoretical Viewpoints Shaojie Li Yong Liu 23 13 0 19 Jul 2021
Towards Understanding Generalization via Decomposing Excess Risk Dynamics Jiaye Teng Jianhao Ma Yang Yuan 29 4 0 11 Jun 2021
Stochastic Gradient Descent for Non-smooth Optimization: Convergence Results and Optimal Averaging Schemes Ohad Shamir Tong Zhang 101 570 0 08 Dec 2012