v1v2 (latest)

Train faster, generalize better: Stability of stochastic gradient descent

3 September 2015

Moritz Hardt

Benjamin Recht

Y. Singer

ArXiv (abs)PDF HTML

Papers citing "Train faster, generalize better: Stability of stochastic gradient descent"

50 / 679 papers shown

Title
Stability and Generalization of the Decentralized Stochastic Gradient Descent Ascent Algorithm Miaoxi Zhu Li Shen Bo Du Dacheng Tao 76 7 0 31 Oct 2023
Sample-Conditioned Hypothesis Stability Sharpens Information-Theoretic Generalization Bounds Ziqiao Wang Yongyi Mao 94 7 0 31 Oct 2023
High-probability Convergence Bounds for Nonlinear Stochastic Gradient Descent Under Heavy-tailed Noise Aleksandar Armacki Pranay Sharma Gauri Joshi Dragana Bajović D. Jakovetić S. Kar 110 7 0 28 Oct 2023
Optimal Guarantees for Algorithmic Reproducibility and Gradient Complexity in Convex Optimization Liang Zhang Junchi Yang Amin Karbasi Niao He 90 2 0 26 Oct 2023
Detecting Pretraining Data from Large Language Models Weijia Shi Anirudh Ajith Mengzhou Xia Yangsibo Huang Daogao Liu Terra Blevins Danqi Chen Luke Zettlemoyer MIALM 122 201 0 25 Oct 2023
Graph Neural Networks with a Distribution of Parametrized Graphs See Hian Lee Feng Ji Kelin Xia Wee Peng Tay 81 0 0 25 Oct 2023
Demystifying the Myths and Legends of Nonconvex Convergence of SGD Aritra Dutta El Houcine Bergou Soumia Boucherouite Nicklas Werge M. Kandemir Xin Li 75 0 0 19 Oct 2023
Online Estimation with Rolling Validation: Adaptive Nonparametric Estimation with Streaming Data Tianyu Zhang Jing Lei 136 1 0 18 Oct 2023
Butterfly Effects of SGD Noise: Error Amplification in Behavior Cloning and Autoregression Adam Block Dylan J. Foster Akshay Krishnamurthy Max Simchowitz Cyril Zhang 85 7 0 17 Oct 2023
Differentially Private Non-convex Learning for Multi-layer Neural Networks Hanpu Shen Cheng-Long Wang Zihang Xiang Yiming Ying Di Wang 81 8 0 12 Oct 2023
Post-hoc Bias Scoring Is Optimal For Fair Classification Wenlong Chen Yegor Klochkov Yang Liu FaML 91 8 0 09 Oct 2023
Stability and Generalization for Minibatch SGD and Local SGD Yunwen Lei Tao Sun Mingrui Liu 95 4 0 02 Oct 2023
A Unified Framework for Generative Data Augmentation: A Comprehensive Survey Yunhao Chen Zihui Yan Yunjie Zhu 80 3 0 30 Sep 2023
Source Inference Attacks: Beyond Membership Inference Attacks in Federated Learning Hongsheng Hu Xuyun Zhang Z. Salcic Lichao Sun K. Choo Gillian Dobbie 69 17 0 30 Sep 2023
Fantastic Generalization Measures are Nowhere to be Found Michael C. Gastpar Ido Nachum Jonathan Shafer T. Weinberger 91 15 0 24 Sep 2023
Generalization error bounds for iterative learning algorithms with bounded updates Jingwen Fu Nanning Zheng 85 1 0 10 Sep 2023
Rethinking the Power of Graph Canonization in Graph Representation Learning with Stability Zehao Dong Muhan Zhang Philip R. O. Payne Michael Province C. Cruchaga Tianyu Zhao Fuhai Li Yixin Chen 94 1 0 01 Sep 2023
Adversarial Style Transfer for Robust Policy Optimization in Deep Reinforcement Learning Md Masudur Rahman Yexiang Xue 61 4 0 29 Aug 2023
Towards Understanding the Generalizability of Delayed Stochastic Gradient Descent Xiaoge Deng Li Shen Shengwei Li Tao Sun Dongsheng Li Dacheng Tao 85 3 0 18 Aug 2023
Stability and Generalization of Hypergraph Collaborative Networks Michael K. Ng Hanrui Wu A. Yip GNN 62 4 0 04 Aug 2023
High Probability Analysis for Non-Convex Stochastic Optimization with Clipping Shaojie Li Yong Liu 58 3 0 25 Jul 2023
Flatness-Aware Minimization for Domain Generalization Xingxuan Zhang Renzhe Xu Han Yu Yancheng Dong Pengfei Tian Peng Cu 94 22 0 20 Jul 2023
Towards Optimal Neural Networks: the Role of Sample Splitting in Hyperparameter Selection Shijin Gong Xinyu Zhang 32 0 0 15 Jul 2023
Minimax Excess Risk of First-Order Methods for Statistical Learning with Data-Dependent Oracles Kevin Scaman Mathieu Even B. L. Bars Laurent Massoulié 59 1 0 10 Jul 2023
Stability and Generalization of Stochastic Compositional Gradient Descent Algorithms Minghao Yang Xiyuan Wei Tianbao Yang Yiming Ying 101 1 0 07 Jul 2023
GraSS: Contrastive Learning with Gradient Guided Sampling Strategy for Remote Sensing Image Semantic Segmentation Zhaoyang Zhang Zhen Ren Chao Tao Yunsheng Zhang Cheng-Shuang Peng Haifeng Li SSL 89 12 0 28 Jun 2023
Nonconvex Stochastic Bregman Proximal Gradient Method with Application to Deep Learning Kuan-Fu Ding Jingyang Li Kim-Chuan Toh 140 8 0 26 Jun 2023
Enhancing Adversarial Training via Reweighting Optimization Trajectory Tianjin Huang Shiwei Liu Tianlong Chen Meng Fang Lijuan Shen Vlaod Menkovski Lu Yin Yulong Pei Mykola Pechenizkiy AAML 84 5 0 25 Jun 2023
On Minimizing the Impact of Dataset Shifts on Actionable Explanations Anna P. Meyer Dan Ley Suraj Srinivas Himabindu Lakkaraju FAtt 69 6 0 11 Jun 2023
Understanding How Consistency Works in Federated Learning via Stage-wise Relaxed Initialization Yan Sun Li Shen Dacheng Tao FedML 91 16 0 09 Jun 2023
Understanding Generalization of Federated Learning via Stability: Heterogeneity Matters Zhenyu Sun Xiaochun Niu Ermin Wei FedML MLT 75 22 0 06 Jun 2023
End-to-end Differentiable Clustering with Associative Memories Bishwajit Saha Dmitry Krotov Mohammed J Zaki Parikshit Ram 69 7 0 05 Jun 2023
Nonparametric Iterative Machine Teaching Chen Zhang Xiaofeng Cao Weiyang Liu Ivor Tsang James T. Kwok 101 8 0 05 Jun 2023
Improved Stability and Generalization Guarantees of the Decentralized SGD Algorithm B. L. Bars A. Bellet Marc Tommasi Kevin Scaman Giovanni Neglia 85 2 0 05 Jun 2023
Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation and Regression Runtian Zhai Bing Liu Andrej Risteski Zico Kolter Pradeep Ravikumar SSL 123 10 0 01 Jun 2023
Three-Way Trade-Off in Multi-Objective Learning: Optimization, Generalization and Conflict-Avoidance Lisha Chen H. Fernando Yiming Ying Tianyi Chen 94 25 0 31 May 2023
Hypothesis Transfer Learning with Surrogate Classification Losses: Generalization Bounds through Algorithmic Stability Anass Aghbalou Guillaume Staerman 77 6 0 31 May 2023
Online-to-PAC Conversions: Generalization Bounds via Regret Analysis Gábor Lugosi Gergely Neu 87 12 0 31 May 2023
Repeated Random Sampling for Minimizing the Time-to-Accuracy of Learning Patrik Okanovic R. Waleffe Vasilis Mageirakos Konstantinos E. Nikolakakis Amin Karbasi Dionysis Kalogerias Nezihe Merve Gürel Theodoros Rekatsinas DD 104 14 0 28 May 2023
Toward Understanding Generative Data Augmentation Chenyu Zheng Guoqiang Wu Chongxuan Li 96 31 0 27 May 2023
Generalization Guarantees of Gradient Descent for Multi-Layer Neural Networks Puyu Wang Yunwen Lei Di Wang Yiming Ying Ding-Xuan Zhou MLT 69 4 0 26 May 2023
On progressive sharpening, flat minima and generalisation L. MacDonald Jack Valmadre Simon Lucey 82 4 0 24 May 2023
Fast Convergence in Learning Two-Layer Neural Networks with Separable Data Hossein Taheri Christos Thrampoulidis MLT 58 3 0 22 May 2023
Stability and Generalization of lp-Regularized Stochastic Learning for GCN Shiyu Liu Linsen Wei Shaogao Lv Ming Li MLT 76 0 0 20 May 2023
Uniform-in-Time Wasserstein Stability Bounds for (Noisy) Stochastic Gradient Descent Lingjiong Zhu Mert Gurbuzbalaban Anant Raj Umut Simsekli 70 6 0 20 May 2023
Is Aggregation the Only Choice? Federated Learning via Layer-wise Model Recombination Ming Hu Zhihao Yue Zhiwei Ling Cheng Chen Yihao Huang Xian Wei Xiang Lian Yang Liu Mingsong Chen FedML 70 10 0 18 May 2023
Physical Layer Authentication and Security Design in the Machine Learning Era T. M. Hoang Alireza Vahid H. Tuan L. Hanzo 84 23 0 16 May 2023
Towards Understanding the Generalization of Graph Neural Networks Huayi Tang Y. Liu GNN AI4CE 95 32 0 14 May 2023
Random Smoothing Regularization in Kernel Gradient Descent Learning Liang Ding Tianyang Hu Jiahan Jiang Donghao Li Wei Cao Yuan Yao 76 6 0 05 May 2023
Select without Fear: Almost All Mini-Batch Schedules Generalize Optimally Konstantinos E. Nikolakakis Amin Karbasi Dionysis Kalogerias 96 5 0 03 May 2023