High-probability Bounds for Non-Convex Stochastic Optimization with Heavy Tails

28 June 2021

Papers citing "High-probability Bounds for Non-Convex Stochastic Optimization with Heavy Tails"

14 / 14 papers shown

Title
Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis Zhijie Chen Qiaobo Li A. Banerjee FedML 37 0 0 11 Nov 2024
From Gradient Clipping to Normalization for Heavy Tailed SGD Florian Hübler Ilyas Fatkhullin Niao He 40 5 0 17 Oct 2024
High Probability Guarantees for Random Reshuffling Hengxu Yu Xiao Li 45 2 0 20 Nov 2023
Demystifying the Myths and Legends of Nonconvex Convergence of SGD Aritra Dutta El Houcine Bergou Soumia Boucherouite Nicklas Werge M. Kandemir Xin Li 28 0 0 19 Oct 2023
Provably Robust Temporal Difference Learning for Heavy-Tailed Rewards Semih Cayci A. Eryilmaz 23 2 0 20 Jun 2023
Clip21: Error Feedback for Gradient Clipping Sarit Khirirat Eduard A. Gorbunov Samuel Horváth Rustem Islamov Fakhri Karray Peter Richtárik 40 10 0 30 May 2023
Stochastic Nonsmooth Convex Optimization with Heavy-Tailed Noises: High-Probability Bound, In-Expectation Rate and Initial Distance Adaptation Zijian Liu Zhengyuan Zhou 30 10 0 22 Mar 2023
SGD with AdaGrad Stepsizes: Full Adaptivity with High Probability to Unknown Parameters, Unbounded Gradients and Affine Variance Amit Attia Tomer Koren ODL 22 26 0 17 Feb 2023
Breaking the Lower Bound with (Little) Structure: Acceleration in Non-Convex Stochastic Optimization with Heavy-Tailed Noise Zijian Liu Jiawei Zhang Zhengyuan Zhou 34 12 0 14 Feb 2023
Near-Optimal Non-Convex Stochastic Optimization under Generalized Smoothness Zijian Liu Srikanth Jagabathula Zhengyuan Zhou 24 5 0 13 Feb 2023
Provable Acceleration of Heavy Ball beyond Quadratics for a Class of Polyak-Łojasiewicz Functions when the Non-Convexity is Averaged-Out Jun-Kun Wang Chi-Heng Lin Andre Wibisono Bin Hu 35 20 0 22 Jun 2022
High Probability Bounds for a Class of Nonconvex Algorithms with AdaGrad Stepsize Ali Kavis Kfir Y. Levy V. Cevher 25 41 0 06 Apr 2022
Flexible risk design using bi-directional dispersion Matthew J. Holland 38 6 0 28 Mar 2022
A High Probability Analysis of Adaptive SGD with Momentum Xiaoyun Li Francesco Orabona 92 66 0 28 Jul 2020