Train faster, generalize better: Stability of stochastic gradient descent

3 September 2015

Benjamin Recht

Papers citing "Train faster, generalize better: Stability of stochastic gradient descent"

50 / 275 papers shown

Title
Benign Underfitting of Stochastic Gradient Descent Tomer Koren Roi Livni Yishay Mansour Uri Sherman MLT 22 13 0 27 Feb 2022
Thinking Outside the Ball: Optimal Learning with Gradient Descent for Generalized Linear Stochastic Convex Optimization I Zaghloul Amir Roi Livni Nathan Srebro 32 6 0 27 Feb 2022
Benefit of Interpolation in Nearest Neighbor Algorithms Yue Xing Qifan Song Guang Cheng 17 28 0 23 Feb 2022
Tackling benign nonconvexity with smoothing and stochastic gradients Harsh Vardhan Sebastian U. Stich 31 8 0 18 Feb 2022
Improving Computational Complexity in Statistical Models with Second-Order Information Tongzheng Ren Jiacheng Zhuo Sujay Sanghavi Nhat Ho 11 1 0 09 Feb 2022
Differentially Private SGDA for Minimax Problems Zhenhuan Yang Shu Hu Yunwen Lei Kush R. Varshney Siwei Lyu Yiming Ying 36 19 0 22 Jan 2022
Low-Pass Filtering SGD for Recovering Flat Optima in the Deep Learning Optimization Landscape Devansh Bisla Jing Wang A. Choromańska 27 34 0 20 Jan 2022
On generalization bounds for deep networks based on loss surface implicit regularization Masaaki Imaizumi Johannes Schmidt-Hieber ODL 34 3 0 12 Jan 2022
Stability Based Generalization Bounds for Exponential Family Langevin Dynamics A. Banerjee Tiancong Chen Xinyan Li Yingxue Zhou 34 8 0 09 Jan 2022
Stable Conformal Prediction Sets Eugène Ndiaye 35 20 0 19 Dec 2021
Communication-Efficient Distributed Learning via Sparse and Adaptive Stochastic Gradient Xiaoge Deng Dongsheng Li Tao Sun Xicheng Lu FedML 26 0 0 08 Dec 2021
Black-box tests for algorithmic stability Byol Kim Rina Foygel Barber AAML 25 13 0 30 Nov 2021
Multi-fidelity Stability for Graph Representation Learning Yihan He Joan Bruna 22 0 0 25 Nov 2021
Subspace Adversarial Training Tao Li Yingwen Wu Sizhe Chen Kun Fang Xiaolin Huang AAML OOD 44 57 0 24 Nov 2021
Gaussian Process Inference Using Mini-batch Stochastic Gradient Descent: Convergence Guarantees and Empirical Benefits Hao Chen Lili Zheng Raed Al Kontar Garvesh Raskutti 24 3 0 19 Nov 2021
Training Neural Networks with Fixed Sparse Masks Yi-Lin Sung Varun Nair Colin Raffel FedML 32 197 0 18 Nov 2021
Convergence and Stability of the Stochastic Proximal Point Algorithm with Momentum J. Kim Panos Toulis Anastasios Kyrillidis 28 8 0 11 Nov 2021
Fast and Accurate Graph Learning for Huge Data via Minipatch Ensembles Tianyi Yao Minjie Wang Genevera I. Allen 16 1 0 22 Oct 2021
Differentially Private Coordinate Descent for Composite Empirical Risk Minimization Paul Mangold A. Bellet Joseph Salmon Marc Tommasi 32 14 0 22 Oct 2021
Deep Active Learning by Leveraging Training Dynamics Haonan Wang Wei Huang Ziwei Wu A. Margenot Hanghang Tong Jingrui He AI4CE 29 33 0 16 Oct 2021
Towards Open-World Feature Extrapolation: An Inductive Graph Learning Approach Qitian Wu Chenxiao Yang Junchi Yan 27 32 0 09 Oct 2021
On the Generalization of Models Trained with SGD: Information-Theoretic Bounds and Implications Ziqiao Wang Yongyi Mao FedML MLT 39 22 0 07 Oct 2021
Spectral Bias in Practice: The Role of Function Frequency in Generalization Sara Fridovich-Keil Raphael Gontijo-Lopes Rebecca Roelofs 41 28 0 06 Oct 2021
Adversarial Representation Learning With Closed-Form Solvers Bashir Sadeghi Lan Wang Vishnu Boddeti 37 5 0 12 Sep 2021
NASI: Label- and Data-agnostic Neural Architecture Search at Initialization Yao Shu Shaofeng Cai Zhongxiang Dai Beng Chin Ooi K. H. Low 27 43 0 02 Sep 2021
Stability and Generalization for Randomized Coordinate Descent Puyu Wang Liang Wu Yunwen Lei 27 7 0 17 Aug 2021
Pointer Value Retrieval: A new benchmark for understanding the limits of neural network generalization Chiyuan Zhang M. Raghu Jon M. Kleinberg Samy Bengio OOD 32 30 0 27 Jul 2021
Improved Learning Rates for Stochastic Optimization: Two Theoretical Viewpoints Shaojie Li Yong Liu 26 13 0 19 Jul 2021
Differentially Private Stochastic Optimization: New Results in Convex and Non-Convex Settings Raef Bassily Cristóbal Guzmán Michael Menart 52 55 0 12 Jul 2021
Optimal Rates for Random Order Online Optimization Uri Sherman Tomer Koren Yishay Mansour 21 8 0 29 Jun 2021
Shuffle Private Stochastic Convex Optimization Albert Cheu Matthew Joseph Jieming Mao Binghui Peng FedML 36 25 0 17 Jun 2021
Learning subtree pattern importance for Weisfeiler-Lehmanbased graph kernels Dai Hai Nguyen Canh Hao Nguyen Hiroshi Mamitsuka 16 6 0 08 Jun 2021
RATT: Leveraging Unlabeled Data to Guarantee Generalization Saurabh Garg Sivaraman Balakrishnan J. Zico Kolter Zachary Chase Lipton 32 30 0 01 May 2021
Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization Zeke Xie Li-xin Yuan Zhanxing Zhu Masashi Sugiyama 32 29 0 31 Mar 2021
Topology-Aware Segmentation Using Discrete Morse Theory Xiaoling Hu Yusu Wang Fuxin Li Dimitris Samaras Chao Chen 29 89 0 18 Mar 2021
Distributed Deep Learning Using Volunteer Computing-Like Paradigm Medha Atre B. Jha Ashwini Rao 20 11 0 16 Mar 2021
Membership Inference Attacks on Machine Learning: A Survey Hongsheng Hu Z. Salcic Lichao Sun Gillian Dobbie Philip S. Yu Xuyun Zhang MIACV 35 412 0 14 Mar 2021
Reframing Neural Networks: Deep Structure in Overcomplete Representations Calvin Murdock George Cazenavette Simon Lucey BDL 41 4 0 10 Mar 2021
$Private Stochastic Convex Optimization: Optimal Rates in $\ell_1$ Geometry$ Private Stochastic Convex Optimization: Optimal Rates in $\ell_1$ Geometry Hilal Asi Vitaly Feldman Tomer Koren Kunal Talwar 30 91 0 02 Mar 2021
Nonlinear Projection Based Gradient Estimation for Query Efficient Blackbox Attacks Huichen Li Linyi Li Xiaojun Xu Xiaolu Zhang Shuang Yang Bo-wen Li AAML 33 17 0 25 Feb 2021
Generalization Bounds for Meta-Learning via PAC-Bayes and Uniform Stability Alec Farid Anirudha Majumdar 27 36 0 12 Feb 2021
Stability of SGD: Tightness Analysis and Improved Bounds Yikai Zhang Wenjia Zhang Sammy Bald Vamsi Pingali Chao Chen Mayank Goswami MLT 27 37 0 10 Feb 2021
Generalization of Model-Agnostic Meta-Learning Algorithms: Recurring and Unseen Tasks Alireza Fallah Aryan Mokhtari Asuman Ozdaglar 25 49 0 07 Feb 2021
SGD Generalizes Better Than GD (And Regularization Doesn't Help) I Zaghloul Amir Tomer Koren Roi Livni 29 46 0 01 Feb 2021
Estimating informativeness of samples with Smooth Unique Information Hrayr Harutyunyan Alessandro Achille Giovanni Paolini Orchid Majumder Avinash Ravichandran Rahul Bhotika Stefano Soatto 27 25 0 17 Jan 2021
Mixed-Privacy Forgetting in Deep Networks Aditya Golatkar Alessandro Achille Avinash Ravichandran M. Polito Stefano Soatto CLL MU 130 161 0 24 Dec 2020
A case for new neural network smoothness constraints Mihaela Rosca T. Weber Arthur Gretton S. Mohamed AAML 35 48 0 14 Dec 2020
Federated Composite Optimization Honglin Yuan Manzil Zaheer Sashank J. Reddi FedML 37 58 0 17 Nov 2020
Direction Matters: On the Implicit Bias of Stochastic Gradient Descent with Moderate Learning Rate Jingfeng Wu Difan Zou Vladimir Braverman Quanquan Gu 18 18 0 04 Nov 2020
A Bayesian Perspective on Training Speed and Model Selection Clare Lyle Lisa Schut Binxin Ru Y. Gal Mark van der Wilk 44 24 0 27 Oct 2020