v1v2 (latest)

Train faster, generalize better: Stability of stochastic gradient descent

3 September 2015

Moritz Hardt

Benjamin Recht

Y. Singer

ArXiv (abs)PDF HTML

Papers citing "Train faster, generalize better: Stability of stochastic gradient descent"

50 / 679 papers shown

Title
Private Non-smooth Empirical Risk Minimization and Stochastic Convex Optimization in Subquadratic Steps Janardhan Kulkarni Y. Lee Daogao Liu 66 28 0 29 Mar 2021
Stability and Deviation Optimal Risk Bounds with Convergence Rate $O(1/n)$ Yegor Klochkov Nikita Zhivotovskiy 87 62 0 22 Mar 2021
Topology-Aware Segmentation Using Discrete Morse Theory Xiaoling Hu Yusu Wang Fuxin Li Dimitris Samaras Chao Chen 88 91 0 18 Mar 2021
Distributed Deep Learning Using Volunteer Computing-Like Paradigm Medha Atre B. Jha Ashwini Rao 91 11 0 16 Mar 2021
Pre-interpolation loss behaviour in neural networks Arthur E. W. Venter Marthinus W. Theunissen Marelie Hattingh Davel 43 3 0 14 Mar 2021
Membership Inference Attacks on Machine Learning: A Survey Hongsheng Hu Z. Salcic Lichao Sun Gillian Dobbie Philip S. Yu Xuyun Zhang MIACV 125 449 0 14 Mar 2021
Reframing Neural Networks: Deep Structure in Overcomplete Representations Calvin Murdock George Cazenavette Simon Lucey BDL 69 5 0 10 Mar 2021
Self-Regularity of Non-Negative Output Weights for Overparameterized Two-Layer Neural Networks D. Gamarnik Eren C. Kizildaug Ilias Zadik 95 1 0 02 Mar 2021
$Private Stochastic Convex Optimization: Optimal Rates in $\ell_1$ Geometry$ Private Stochastic Convex Optimization: Optimal Rates in $\ell_1$ Geometry Hilal Asi Vitaly Feldman Tomer Koren Kunal Talwar 63 94 0 02 Mar 2021
DPlis: Boosting Utility of Differentially Private Deep Learning via Randomized Smoothing Wenxiao Wang Tianhao Wang Lun Wang Nanqing Luo Pan Zhou Basel Alomair R. Jia 109 16 0 02 Mar 2021
Smoothness Analysis of Adversarial Training Sekitoshi Kanai Masanori Yamada Hiroshi Takahashi Yuki Yamanaka Yasutoshi Ida AAML 106 6 0 02 Mar 2021
Acceleration via Fractal Learning Rate Schedules Naman Agarwal Surbhi Goel Cyril Zhang 78 18 0 01 Mar 2021
Noisy Truncated SGD: Optimization and Generalization Yingxue Zhou Xinyan Li A. Banerjee 69 3 0 26 Feb 2021
Nonlinear Projection Based Gradient Estimation for Query Efficient Blackbox Attacks Huichen Li Linyi Li Xiaojun Xu Xiaolu Zhang Shuang Yang Yue Liu AAML 80 17 0 25 Feb 2021
A Probabilistically Motivated Learning Rate Adaptation for Stochastic Optimization Filip de Roos Carl Jidling A. Wills Thomas B. Schon Philipp Hennig 52 3 0 22 Feb 2021
Generalization bounds for graph convolutional neural networks via Rademacher complexity Shaogao Lv GNN 120 16 0 20 Feb 2021
Generalization Bounds for Meta-Learning via PAC-Bayes and Uniform Stability Alec Farid Anirudha Majumdar 81 36 0 12 Feb 2021
Stability of SGD: Tightness Analysis and Improved Bounds Yikai Zhang Wenjia Zhang Sammy Bald Vamsi Pingali Chao Chen Mayank Goswami MLT 59 38 0 10 Feb 2021
Generalization of Model-Agnostic Meta-Learning Algorithms: Recurring and Unseen Tasks Alireza Fallah Aryan Mokhtari Asuman Ozdaglar 123 51 0 07 Feb 2021
Algorithmic Instabilities of Accelerated Gradient Descent Amit Attia Tomer Koren 33 8 0 03 Feb 2021
Stability and Generalization of the Decentralized Stochastic Gradient Descent Tao Sun Dongsheng Li Bao Wang 43 0 0 02 Feb 2021
SGD Generalizes Better Than GD (And Regularization Doesn't Help) I Zaghloul Amir Tomer Koren Roi Livni 75 46 0 01 Feb 2021
Information-Theoretic Generalization Bounds for Stochastic Gradient Descent Gergely Neu Gintare Karolina Dziugaite Mahdi Haghifam Daniel M. Roy 133 90 0 01 Feb 2021
Painless step size adaptation for SGD I. Kulikovskikh Tarzan Legović 74 0 0 01 Feb 2021
On Data Efficiency of Meta-learning Maruan Al-Shedivat Liam Li Eric Xing Ameet Talwalkar FedML 70 25 0 30 Jan 2021
Differentially Private SGD with Non-Smooth Losses Puyu Wang Yunwen Lei Yiming Ying Hai Zhang 83 28 0 22 Jan 2021
Faster Convergence in Deep-Predictive-Coding Networks to Learn Deeper Representations I. Sledge José C. Príncipe 71 2 0 18 Jan 2021
Estimating informativeness of samples with Smooth Unique Information Hrayr Harutyunyan Alessandro Achille Giovanni Paolini Orchid Majumder Avinash Ravichandran Rahul Bhotika Stefano Soatto 95 25 0 17 Jan 2021
Learning with Gradient Descent and Weakly Convex Losses Dominic Richards Michael G. Rabbat MLT 71 15 0 13 Jan 2021
Target Detection and Segmentation in Circular-Scan Synthetic-Aperture-Sonar Images using Semi-Supervised Convolutional Encoder-Decoders I. Sledge Matthew S. Emigh Jonathan L. King Denton L. Woods J. T. Cobb José C. Príncipe 76 17 0 10 Jan 2021
Reinforcement Learning for Control of Valves Rajesh Siraskar AI4CE 44 33 0 29 Dec 2020
Robustness, Privacy, and Generalization of Adversarial Training Fengxiang He Shaopeng Fu Bohan Wang Dacheng Tao 125 10 0 25 Dec 2020
Mixed-Privacy Forgetting in Deep Networks Aditya Golatkar Alessandro Achille Avinash Ravichandran M. Polito Stefano Soatto CLL MU 232 169 0 24 Dec 2020
A Tight Lower Bound for Uniformly Stable Algorithms Qinghua Liu Zhou Lu 25 0 0 24 Dec 2020
Recent advances in deep learning theory Fengxiang He Dacheng Tao AI4CE 132 51 0 20 Dec 2020
A case for new neural network smoothness constraints Mihaela Rosca T. Weber Arthur Gretton S. Mohamed AAML 145 50 0 14 Dec 2020
Analyzing Finite Neural Networks: Can We Trust Neural Tangent Kernel Theory? Mariia Seleznova Gitta Kutyniok AAML 88 30 0 08 Dec 2020
Generalization bounds for deep learning Guillermo Valle Pérez A. Louis BDL 84 45 0 07 Dec 2020
Characterization of Excess Risk for Locally Strongly Convex Population Risk Mingyang Yi Ruoyu Wang Zhi-Ming Ma 46 2 0 04 Dec 2020
Learning with Knowledge of Structure: A Neural Network-Based Approach for MIMO-OFDM Detection Zhou Zhou Shashank Jere Lizhong Zheng Lingjia Liu 29 4 0 01 Dec 2020
On the Overlooked Pitfalls of Weight Decay and How to Mitigate Them: A Gradient-Norm Perspective Zeke Xie Zhiqiang Xu Jingzhao Zhang Issei Sato Masashi Sugiyama 91 25 0 23 Nov 2020
Federated Composite Optimization Honglin Yuan Manzil Zaheer Sashank J. Reddi FedML 89 61 0 17 Nov 2020
SALR: Sharpness-aware Learning Rate Scheduler for Improved Generalization Xubo Yue Maher Nouiehed Raed Al Kontar ODL 40 4 0 10 Nov 2020
Direction Matters: On the Implicit Bias of Stochastic Gradient Descent with Moderate Learning Rate Jingfeng Wu Difan Zou Vladimir Braverman Quanquan Gu 102 18 0 04 Nov 2020
Cross-Lingual Document Retrieval with Smooth Learning Jiapeng Liu Xiao Zhang Dan Goldwasser Xiao Wang 47 9 0 02 Nov 2020
Faster Differentially Private Samplers via Rényi Divergence Analysis of Discretized Langevin MCMC Arun Ganesh Kunal Talwar FedML 84 41 0 27 Oct 2020
A Bayesian Perspective on Training Speed and Model Selection Clare Lyle Lisa Schut Binxin Ru Y. Gal Mark van der Wilk 102 24 0 27 Oct 2020
Toward Better Generalization Bounds with Locally Elastic Stability Zhun Deng Hangfeng He Weijie J. Su 71 44 0 27 Oct 2020
Stochastic Optimization with Laggard Data Pipelines Naman Agarwal Rohan Anil Tomer Koren Kunal Talwar Cyril Zhang 35 12 0 26 Oct 2020
Stochastic Gradient Descent Meets Distribution Regression Nicole Mücke 67 5 0 24 Oct 2020