Train faster, generalize better: Stability of stochastic gradient descent

3 September 2015

Benjamin Recht

Papers citing "Train faster, generalize better: Stability of stochastic gradient descent"

50 / 275 papers shown

Title
Efficient Gradient Approximation Method for Constrained Bilevel Optimization Siyuan Xu Minghui Zhu 36 20 0 03 Feb 2023
Bagging Provides Assumption-free Stability Jake A. Soloff Rina Foygel Barber Rebecca Willett 24 9 0 30 Jan 2023
On the Lipschitz Constant of Deep Networks and Double Descent Matteo Gamba Hossein Azizpour Mårten Björkman 33 7 0 28 Jan 2023
Algorithmic Stability of Heavy-Tailed SGD with General Loss Functions Anant Raj Lingjiong Zhu Mert Gurbuzbalaban Umut Simsekli 36 15 0 27 Jan 2023
Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing Jikai Jin Zhiyuan Li Kaifeng Lyu S. Du Jason D. Lee MLT 56 34 0 27 Jan 2023
A Stability Analysis of Fine-Tuning a Pre-Trained Model Z. Fu Anthony Man-Cho So Nigel Collier 28 3 0 24 Jan 2023
Stretched and measured neural predictions of complex network dynamics V. Vasiliauskaite Nino Antulov-Fantulin 38 1 0 12 Jan 2023
Sharper Analysis for Minibatch Stochastic Proximal Point Methods: Stability, Smoothness, and Deviation Xiao-Tong Yuan P. Li 41 2 0 09 Jan 2023
Resampling Sensitivity of High-Dimensional PCA Haoyu Wang 29 0 0 30 Dec 2022
Limitations of Information-Theoretic Generalization Bounds for Gradient Descent Methods in Stochastic Convex Optimization Mahdi Haghifam Borja Rodríguez Gálvez Ragnar Thobaben Mikael Skoglund Daniel M. Roy Gintare Karolina Dziugaite 31 17 0 27 Dec 2022
Iterative regularization in classification via hinge loss diagonal descent Vassilis Apidopoulos T. Poggio Lorenzo Rosasco S. Villa 32 2 0 24 Dec 2022
On the Overlooked Structure of Stochastic Gradients Zeke Xie Qian-Yuan Tang Mingming Sun P. Li 33 6 0 05 Dec 2022
Two Facets of SDE Under an Information-Theoretic Lens: Generalization of SGD via Training Trajectories and via Terminal States Ziqiao Wang Yongyi Mao 35 10 0 19 Nov 2022
On the Algorithmic Stability and Generalization of Adaptive Optimization Methods Han Nguyen Hai Pham Sashank J. Reddi Barnabas Poczos ODL AI4CE 24 2 0 08 Nov 2022
Do highly over-parameterized neural networks generalize since bad solutions are rare? Julius Martinetz T. Martinetz 32 1 0 07 Nov 2022
Distributed DP-Helmet: Scalable Differentially Private Non-interactive Averaging of Single Layers Moritz Kirschte Sebastian Meiser Saman Ardalan Esfandiar Mohammadi FedML 34 0 0 03 Nov 2022
Optimal Algorithms for Stochastic Complementary Composite Minimization Alexandre d’Aspremont Cristóbal Guzmán Clément Lezane 33 3 0 03 Nov 2022
FedCross: Towards Accurate Federated Learning via Multi-Model Cross-Aggregation Ming Hu Peiheng Zhou Zhihao Yue Zhiwei Ling Yihao Huang Anran Li Yang Liu Xiang Lian Mingsong Chen FedML 24 14 0 15 Oct 2022
On Stability and Generalization of Bilevel Optimization Problem Meng Ding Ming Lei Yunwen Lei Di Wang Jinhui Xu 32 1 0 03 Oct 2022
Stability Analysis and Generalization Bounds of Adversarial Training Jiancong Xiao Yanbo Fan Ruoyu Sun Jue Wang Zhimin Luo AAML 38 30 0 03 Oct 2022
Adaptive Smoothness-weighted Adversarial Training for Multiple Perturbations with Its Stability Analysis Jiancong Xiao Zeyu Qin Yanbo Fan Baoyuan Wu Jue Wang Zhimin Luo AAML 39 7 0 02 Oct 2022
Neural Networks Efficiently Learn Low-Dimensional Representations with SGD Alireza Mousavi-Hosseini Sejun Park M. Girotti Ioannis Mitliagkas Murat A. Erdogdu MLT 324 48 0 29 Sep 2022
Exploring the Algorithm-Dependent Generalization of AUPRC Optimization with List Stability Peisong Wen Qianqian Xu Zhiyong Yang Yuan He Qingming Huang 55 10 0 27 Sep 2022
On the Stability Analysis of Open Federated Learning Systems Youbang Sun H. Fernando Tianyi Chen Shahin Shahrampour FedML 31 1 0 25 Sep 2022
Stability and Generalization for Markov Chain Stochastic Gradient Methods Puyu Wang Yunwen Lei Yiming Ying Ding-Xuan Zhou 24 18 0 16 Sep 2022
On Generalization of Decentralized Learning with Separable Data Hossein Taheri Christos Thrampoulidis FedML 44 11 0 15 Sep 2022
On the Reuse Bias in Off-Policy Reinforcement Learning Chengyang Ying Zhongkai Hao Xinning Zhou Hang Su Dong Yan Jun Zhu OffRL 45 3 0 15 Sep 2022
Differentially Private Stochastic Gradient Descent with Low-Noise Puyu Wang Yunwen Lei Yiming Ying Ding-Xuan Zhou FedML 51 5 0 09 Sep 2022
Generalisation under gradient descent via deterministic PAC-Bayes Eugenio Clerico Tyler Farghly George Deligiannidis Benjamin Guedj Arnaud Doucet 33 4 0 06 Sep 2022
SYNTHESIS: A Semi-Asynchronous Path-Integrated Stochastic Gradient Method for Distributed Learning in Computing Clusters Zhuqing Liu Xin Zhang Jia-Wei Liu 38 1 0 17 Aug 2022
On the generalization of learning algorithms that do not converge N. Chandramoorthy Andreas Loukas Khashayar Gatmiry Stefanie Jegelka MLT 23 11 0 16 Aug 2022
Uniform Stability for First-Order Empirical Risk Minimization Amit Attia Tomer Koren 25 5 0 17 Jul 2022
Bootstrap State Representation using Style Transfer for Better Generalization in Deep Reinforcement Learning Md Masudur Rahman Yexiang Xue OffRL 34 4 0 15 Jul 2022
On Leave-One-Out Conditional Mutual Information For Generalization Mohamad Rida Rammal Alessandro Achille Aditya Golatkar Suhas Diggavi Stefano Soatto VLM 41 6 0 01 Jul 2022
Sparse Double Descent: Where Network Pruning Aggravates Overfitting Zhengqi He Zeke Xie Quanzhi Zhu Zengchang Qin 86 27 0 17 Jun 2022
Trajectory-dependent Generalization Bounds for Deep Neural Networks via Fractional Brownian Motion Chengli Tan Jiang Zhang Junmin Liu 45 1 0 09 Jun 2022
Multi-class Classification with Fuzzy-feature Observations: Theory and Algorithms Guangzhi Ma Jie Lu Feng Liu Zhen Fang Guangquan Zhang 23 6 0 09 Jun 2022
Subject Membership Inference Attacks in Federated Learning Anshuman Suri Pallika H. Kanani Virendra J. Marathe Daniel W. Peterson 30 25 0 07 Jun 2022
Dimension Independent Generalization of DP-SGD for Overparameterized Smooth Convex Optimization Yi Ma T. V. Marinov Tong Zhang 27 8 0 03 Jun 2022
Algorithmic Stability of Heavy-Tailed Stochastic Gradient Descent on Least Squares Anant Raj Melih Barsbey Mert Gurbuzbalaban Lingjiong Zhu Umut Simsekli 24 9 0 02 Jun 2022
Differentially Private Shapley Values for Data Evaluation Lauren Watson R. Andreeva Hao Yang Rik Sarkar TDI FAtt FedML 21 6 0 01 Jun 2022
AANG: Automating Auxiliary Learning Lucio Dery Paul Michel M. Khodak Graham Neubig Ameet Talwalkar 43 9 0 27 May 2022
Selective Classification Via Neural Network Training Dynamics Stephan Rabanser Anvith Thudi Kimia Hamidieh Adam Dziedzic Nicolas Papernot 29 21 0 26 May 2022
Learning from time-dependent streaming data with online stochastic algorithms Antoine Godichon-Baggioni Nicklas Werge Olivier Wintenberger 40 3 0 25 May 2022
Uniform Generalization Bound on Time and Inverse Temperature for Gradient Descent Algorithm and its Application to Analysis of Simulated Annealing Keisuke Suzuki AI4CE 33 0 0 25 May 2022
Weak Convergence of Approximate reflection coupling and its Application to Non-convex Optimization Keisuke Suzuki 36 5 0 24 May 2022
Beyond Lipschitz: Sharp Generalization and Excess Risk Bounds for Full-Batch GD Konstantinos E. Nikolakakis Farzin Haddadpour Amin Karbasi Dionysios S. Kalogerias 43 17 0 26 Apr 2022
Sharper Utility Bounds for Differentially Private Models Yilin Kang Yong Liu Jian Li Weiping Wang FedML 35 3 0 22 Apr 2022
Stability and Risk Bounds of Iterative Hard Thresholding Xiao-Tong Yuan P. Li 39 12 0 17 Mar 2022
Stability vs Implicit Bias of Gradient Methods on Separable Data and Beyond Matan Schliserman Tomer Koren 24 23 0 27 Feb 2022