Unified Optimal Analysis of the (Stochastic) Gradient Method

9 July 2019

Papers citing "Unified Optimal Analysis of the (Stochastic) Gradient Method"

33 / 33 papers shown

Title
Trial and Trust: Addressing Byzantine Attacks with Comprehensive Defense Strategy Gleb Molodtsov Daniil Medyakov Sergey Skorik Nikolas Khachaturov Shahane Tigranyan Vladimir Aletov A. Avetisyan Martin Takáč Aleksandr Beznosikov AAML 35 0 0 12 May 2025
Variance Reduction Methods Do Not Need to Compute Full Gradients: Improved Efficiency through Shuffling Daniil Medyakov Gleb Molodtsov S. Chezhegov Alexey Rebrikov Aleksandr Beznosikov 103 0 0 21 Feb 2025
An Enhanced Zeroth-Order Stochastic Frank-Wolfe Framework for Constrained Finite-Sum Optimization Haishan Ye Yinghui Huang Hao Di Xiangyu Chang 38 0 0 13 Jan 2025
Demystifying SGD with Doubly Stochastic Gradients Kyurae Kim Joohwan Ko Yian Ma Jacob R. Gardner 53 0 0 03 Jun 2024
The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication Kumar Kshitij Patel Margalit Glasgow Ali Zindari Lingxiao Wang Sebastian U. Stich Ziheng Cheng Nirmit Joshi Nathan Srebro 48 6 0 19 May 2024
Towards Efficient Risk-Sensitive Policy Gradient: An Iteration Complexity Analysis Rui Liu Erfaun Noorani Pratap Tokekar John S. Baras 23 1 0 13 Mar 2024
Provably Scalable Black-Box Variational Inference with Structured Variational Families Joohwan Ko Kyurae Kim W. Kim Jacob R. Gardner BDL 30 2 0 19 Jan 2024
Provable convergence guarantees for black-box variational inference Justin Domke Guillaume Garrigos Robert Mansel Gower 18 18 0 04 Jun 2023
First Order Methods with Markovian Noise: from Acceleration to Variational Inequalities Aleksandr Beznosikov S. Samsonov Marina Sheshukova Alexander Gasnikov A. Naumov Eric Moulines 39 14 0 25 May 2023
Sarah Frank-Wolfe: Methods for Constrained Optimization with Best Rates and Practical Features Aleksandr Beznosikov David Dobre Gauthier Gidel 25 5 0 23 Apr 2023
Single-Call Stochastic Extragradient Methods for Structured Non-monotone Variational Inequalities: Improved Analysis under Weaker Conditions S. Choudhury Eduard A. Gorbunov Nicolas Loizou 25 13 0 27 Feb 2023
FedSkip: Combatting Statistical Heterogeneity with Federated Skip Aggregation Ziqing Fan Yanfeng Wang Jiangchao Yao Lingjuan Lyu Ya-Qin Zhang Qinghua Tian FedML 21 20 0 14 Dec 2022
On the effectiveness of partial variance reduction in federated learning with heterogeneous data Bo-wen Li Mikkel N. Schmidt T. S. Alstrøm Sebastian U. Stich FedML 37 9 0 05 Dec 2022
Tackling benign nonconvexity with smoothing and stochastic gradients Harsh Vardhan Sebastian U. Stich 23 8 0 18 Feb 2022
Stochastic Gradient Descent-Ascent: Unified Theory and New Efficient Methods Aleksandr Beznosikov Eduard A. Gorbunov Hugo Berard Nicolas Loizou 19 47 0 15 Feb 2022
Stochastic Extragradient: General Analysis and Improved Rates Eduard A. Gorbunov Hugo Berard Gauthier Gidel Nicolas Loizou 14 40 0 16 Nov 2021
Towards Noise-adaptive, Problem-adaptive (Accelerated) Stochastic Gradient Descent Sharan Vaswani Benjamin Dubois-Taine Reza Babanezhad 48 11 0 21 Oct 2021
A general sample complexity analysis of vanilla policy gradient Rui Yuan Robert Mansel Gower A. Lazaric 74 62 0 23 Jul 2021
Stochastic Polyak Stepsize with a Moving Target Robert Mansel Gower Aaron Defazio Michael G. Rabbat 24 17 0 22 Jun 2021
Decentralized Local Stochastic Extra-Gradient for Variational Inequalities Aleksandr Beznosikov Pavel Dvurechensky Anastasia Koloskova V. Samokhin Sebastian U. Stich Alexander Gasnikov 29 43 0 15 Jun 2021
Local Adaptivity in Federated Learning: Convergence and Consistency Jianyu Wang Zheng Xu Zachary Garrett Zachary B. Charles Luyang Liu Gauri Joshi FedML 24 39 0 04 Jun 2021
Stochastic gradient descent with noise of machine learning type. Part I: Discrete time analysis Stephan Wojtowytsch 23 50 0 04 May 2021
Consensus Control for Decentralized Deep Learning Lingjing Kong Tao R. Lin Anastasia Koloskova Martin Jaggi Sebastian U. Stich 19 75 0 09 Feb 2021
Local SGD: Unified Theory and New Efficient Methods Eduard A. Gorbunov Filip Hanzely Peter Richtárik FedML 35 108 0 03 Nov 2020
On Communication Compression for Distributed Optimization on Heterogeneous Data Sebastian U. Stich 45 22 0 04 Sep 2020
On the Convergence of SGD with Biased Gradients Ahmad Ajalloeian Sebastian U. Stich 6 83 0 31 Jul 2020
Random Reshuffling: Simple Analysis with Vast Improvements Konstantin Mishchenko Ahmed Khaled Peter Richtárik 37 131 0 10 Jun 2020
Minibatch vs Local SGD for Heterogeneous Distributed Learning Blake E. Woodworth Kumar Kshitij Patel Nathan Srebro FedML 22 198 0 08 Jun 2020
A Unified Theory of Decentralized SGD with Changing Topology and Local Updates Anastasia Koloskova Nicolas Loizou Sadra Boreiri Martin Jaggi Sebastian U. Stich FedML 41 491 0 23 Mar 2020
Is Local SGD Better than Minibatch SGD? Blake E. Woodworth Kumar Kshitij Patel Sebastian U. Stich Zhen Dai Brian Bullins H. B. McMahan Ohad Shamir Nathan Srebro FedML 34 253 0 18 Feb 2020
Better Theory for SGD in the Nonconvex World Ahmed Khaled Peter Richtárik 13 178 0 09 Feb 2020
A simpler approach to obtaining an O(1/t) convergence rate for the projected stochastic subgradient method Simon Lacoste-Julien Mark W. Schmidt Francis R. Bach 126 259 0 10 Dec 2012
Stochastic Gradient Descent for Non-smooth Optimization: Convergence Results and Optimal Averaging Schemes Ohad Shamir Tong Zhang 101 570 0 08 Dec 2012