Revisiting Gradient Clipping: Stochastic bias and tight convergence guarantees

2 May 2023

Papers citing "Revisiting Gradient Clipping: Stochastic bias and tight convergence guarantees"

12 / 12 papers shown

Title
An Improved Privacy and Utility Analysis of Differentially Private SGD with Bounded Domain and Smooth Losses Hao Liang W. Zhang Xinlei He Kaishun He Hong Xing 47 0 0 25 Feb 2025
Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis Zhijie Chen Qiaobo Li A. Banerjee FedML 35 0 0 11 Nov 2024
From Gradient Clipping to Normalization for Heavy Tailed SGD Florian Hübler Ilyas Fatkhullin Niao He 40 5 0 17 Oct 2024
A New First-Order Meta-Learning Algorithm with Convergence Guarantees El Mahdi Chayti Martin Jaggi 25 1 0 05 Sep 2024
Differential Private Stochastic Optimization with Heavy-tailed Data: Towards Optimal Rates Puning Zhao Jiafei Wu Zhe Liu Chong Wang Rongfei Fan Qingming Li 48 1 0 19 Aug 2024
You Only Accept Samples Once: Fast, Self-Correcting Stochastic Variational Inference Dominic B. Dayta TPM BDL 30 0 0 05 Jun 2024
Enhancing High-Resolution 3D Generation through Pixel-wise Gradient Clipping Zijie Pan Jiachen Lu Xiatian Zhu Li Zhang DiffM 28 11 0 19 Oct 2023
The Relative Gaussian Mechanism and its Application to Private Gradient Descent Hadrien Hendrikx Paul Mangold A. Bellet 33 1 0 29 Aug 2023
Stochastic Re-weighted Gradient Descent via Distributionally Robust Optimization Ramnath Kumar Kushal Majmundar Dheeraj M. Nagaraj A. Suggala ODL 24 6 0 15 Jun 2023
Convergence and Privacy of Decentralized Nonconvex Optimization with Gradient Clipping and Communication Compression Boyue Li Yuejie Chi 21 12 0 17 May 2023
Variants of SGD for Lipschitz Continuous Loss Functions in Low-Precision Environments Michael R. Metel 28 1 0 09 Nov 2022
Optimal Distributed Online Prediction using Mini-Batches O. Dekel Ran Gilad-Bachrach Ohad Shamir Lin Xiao 177 683 0 07 Dec 2010