v1v2 (latest)

Train faster, generalize better: Stability of stochastic gradient descent

3 September 2015

Moritz Hardt

Benjamin Recht

Y. Singer

ArXiv (abs)PDF HTML

Papers citing "Train faster, generalize better: Stability of stochastic gradient descent"

50 / 679 papers shown

Title
Stability and Generalization Analysis of Gradient Methods for Shallow Neural Networks Yunwen Lei Rong Jin Yiming Ying MLT 102 19 0 19 Sep 2022
$Generalization Bounds for Stochastic Gradient Descent via Localized $\varepsilon$-Covers$ Generalization Bounds for Stochastic Gradient Descent via Localized $\varepsilon$ -Covers Sejun Park Umut Simsekli Murat A. Erdogdu 107 9 0 19 Sep 2022
Stability and Generalization for Markov Chain Stochastic Gradient Methods Puyu Wang Yunwen Lei Yiming Ying Ding-Xuan Zhou 78 18 0 16 Sep 2022
On Generalization of Decentralized Learning with Separable Data Hossein Taheri Christos Thrampoulidis FedML 100 11 0 15 Sep 2022
On the Reuse Bias in Off-Policy Reinforcement Learning Chengyang Ying Zhongkai Hao Xinning Zhou Hang Su Dong Yan Jun Zhu OffRL 85 3 0 15 Sep 2022
Differentially Private Stochastic Gradient Descent with Low-Noise Puyu Wang Yunwen Lei Yiming Ying Ding-Xuan Zhou FedML 87 5 0 09 Sep 2022
Generalisation under gradient descent via deterministic PAC-Bayes Eugenio Clerico Tyler Farghly George Deligiannidis Benjamin Guedj Arnaud Doucet 160 4 0 06 Sep 2022
Super-model ecosystem: A domain-adaptation perspective Fengxiang He Dacheng Tao DiffM 86 1 0 30 Aug 2022
Generalization In Multi-Objective Machine Learning Peter Súkeník Christoph H. Lampert AI4CE 89 6 0 29 Aug 2022
Visualizing high-dimensional loss landscapes with Hessian directions Lucas Böttcher Gregory R. Wheeler 79 14 0 28 Aug 2022
SYNTHESIS: A Semi-Asynchronous Path-Integrated Stochastic Gradient Method for Distributed Learning in Computing Clusters Zhuqing Liu Xin Zhang Jia-Wei Liu 107 1 0 17 Aug 2022
On the generalization of learning algorithms that do not converge N. Chandramoorthy Andreas Loukas Khashayar Gatmiry Stefanie Jegelka MLT 97 11 0 16 Aug 2022
Training Overparametrized Neural Networks in Sublinear Time Yichuan Deng Han Hu Zhao Song Omri Weinstein Danyang Zhuo BDL 103 28 0 09 Aug 2022
Learning from few examples: Classifying sex from retinal images via deep learning Aaron Berk Gulcenur Ozturan Parsa Delavari D. Maberley Özgür Yilmaz Ipek Oruc 31 3 0 20 Jul 2022
Uniform Stability for First-Order Empirical Risk Minimization Amit Attia Tomer Koren 68 5 0 17 Jul 2022
Bootstrap State Representation using Style Transfer for Better Generalization in Deep Reinforcement Learning Md Masudur Rahman Yexiang Xue OffRL 73 4 0 15 Jul 2022
High-Dimensional Private Empirical Risk Minimization by Greedy Coordinate Descent Paul Mangold A. Bellet Joseph Salmon Marc Tommasi 89 5 0 04 Jul 2022
On Leave-One-Out Conditional Mutual Information For Generalization Mohamad Rida Rammal Alessandro Achille Aditya Golatkar Suhas Diggavi Stefano Soatto VLM 95 6 0 01 Jul 2022
Studying Generalization Through Data Averaging C. Gomez-Uribe FedML 138 0 0 28 Jun 2022
Analyzing Explainer Robustness via Probabilistic Lipschitzness of Prediction Functions Zulqarnain Khan Davin Hill A. Masoomi Joshua Bone Jennifer Dy AAML 138 4 0 24 Jun 2022
Dissecting U-net for Seismic Application: An In-Depth Study on Deep Learning Multiple Removal Ricard Durall A. Ghanim N. Ettrich J. Keuper 29 2 0 24 Jun 2022
Near-optimal control of dynamical systems with neural ordinary differential equations Lucas Böttcher Thomas Asikis AI4CE 67 19 0 22 Jun 2022
f-divergences and their applications in lossy compression and bounding generalization error Saeed Masiha A. Gohari Mohammad Hossein Yassaee 96 14 0 21 Jun 2022
Sparse Double Descent: Where Network Pruning Aggravates Overfitting Zhengqi He Zeke Xie Quanzhi Zhu Zengchang Qin 140 28 0 17 Jun 2022
Implicit Regularization or Implicit Conditioning? Exact Risk Trajectories of SGD in High Dimensions Courtney Paquette Elliot Paquette Ben Adlam Jeffrey Pennington 63 14 0 15 Jun 2022
Stability and Generalization of Stochastic Optimization with Nonconvex and Nonsmooth Problems Yunwen Lei 55 19 0 14 Jun 2022
On Convergence of FedProx: Local Dissimilarity Invariant Bounds, Non-smoothness and Beyond Xiao-Tong Yuan P. Li FedML 78 62 0 10 Jun 2022
What is a Good Metric to Study Generalization of Minimax Learners? Asuman Ozdaglar S. Pattathil Jiawei Zhang Jianchao Tan 79 14 0 09 Jun 2022
Trajectory-dependent Generalization Bounds for Deep Neural Networks via Fractional Brownian Motion Chengli Tan Jiang Zhang Junmin Liu 83 1 0 09 Jun 2022
Multi-class Classification with Fuzzy-feature Observations: Theory and Algorithms Guangzhi Ma Jie Lu Feng Liu Zhen Fang Guangquan Zhang 67 6 0 09 Jun 2022
Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learning Momin Abbas Quan-Wu Xiao Lisha Chen Pin-Yu Chen Tianyi Chen 113 84 0 08 Jun 2022
Boosting the Confidence of Generalization for $L_2$ -Stable Randomized Learning Algorithms Xiao-Tong Yuan Ping Li 103 4 0 08 Jun 2022
Subject Membership Inference Attacks in Federated Learning Anshuman Suri Pallika H. Kanani Virendra J. Marathe Daniel W. Peterson 59 27 0 07 Jun 2022
Generalization Error Bounds for Deep Neural Networks Trained by SGD Mingze Wang Chao Ma 41 14 0 07 Jun 2022
Concentration of the missing mass in metric spaces Andreas Maurer 54 1 0 04 Jun 2022
Dimension Independent Generalization of DP-SGD for Overparameterized Smooth Convex Optimization Yi-An Ma T. V. Marinov Tong Zhang 66 9 0 03 Jun 2022
Debiased Machine Learning without Sample-Splitting for Stable Estimators Qizhao Chen Vasilis Syrgkanis Morgane Austern CML 90 18 0 03 Jun 2022
Understanding Deep Learning via Decision Boundary Shiye Lei Fengxiang He Yancheng Yuan Dacheng Tao 76 14 0 03 Jun 2022
Adversarial Unlearning: Reducing Confidence Along Adversarial Directions Amrith Rajagopal Setlur Benjamin Eysenbach Virginia Smith Sergey Levine 79 19 0 03 Jun 2022
Algorithmic Stability of Heavy-Tailed Stochastic Gradient Descent on Least Squares Anant Raj Melih Barsbey Mert Gurbuzbalaban Lingjiong Zhu Umut Simsekli 78 9 0 02 Jun 2022
Differentially Private Shapley Values for Data Evaluation Lauren Watson R. Andreeva Hao Yang Rik Sarkar TDI FAtt FedML 78 6 0 01 Jun 2022
Benign Overfitting in Classification: Provably Counter Label Noise with Larger Models Kaiyue Wen Jiaye Teng J.N. Zhang NoLa 68 5 0 01 Jun 2022
Bring Your Own Algorithm for Optimal Differentially Private Stochastic Minimax Optimization Liang Zhang K. K. Thekumparampil Sewoong Oh Niao He 96 20 0 01 Jun 2022
Generalization Bounds of Nonconvex-(Strongly)-Concave Stochastic Minimax Optimization Siqi Zhang Yifan Hu Liang Zhang Niao He 91 4 0 28 May 2022
AANG: Automating Auxiliary Learning Lucio Dery Paul Michel M. Khodak Graham Neubig Ameet Talwalkar 122 9 0 27 May 2022
Generalization Bounds for Gradient Methods via Discrete and Continuous Prior Jun Yu Li Xu Luo Jian Li 80 4 0 27 May 2022
Privacy of Noisy Stochastic Gradient Descent: More Iterations without More Privacy Loss Jason M. Altschuler Kunal Talwar FedML 146 61 0 27 May 2022
Selective Prediction via Training Dynamics Stephan Rabanser Anvith Thudi Kimia Hamidieh Adam Dziedzic Nicolas Papernot Akram Bin Sediq Hamza Sokun Nicolas Papernot 120 22 0 26 May 2022
Learning from time-dependent streaming data with online stochastic algorithms Antoine Godichon-Baggioni Nicklas Werge Olivier Wintenberger 122 3 0 25 May 2022
Uniform Generalization Bound on Time and Inverse Temperature for Gradient Descent Algorithm and its Application to Analysis of Simulated Annealing Keisuke Suzuki AI4CE 83 0 0 25 May 2022