Heavy-Tail Phenomenon in Decentralized SGD

13 May 2022

Papers citing "Heavy-Tail Phenomenon in Decentralized SGD"

4 / 4 papers shown

Title
Decentralized SGD and Average-direction SAM are Asymptotically Equivalent Tongtian Zhu Fengxiang He Kaixuan Chen Mingli Song Dacheng Tao 34 15 0 05 Jun 2023
Cyclic and Randomized Stepsizes Invoke Heavier Tails in SGD than Constant Stepsize Mert Gurbuzbalaban Yuanhan Hu Umut Simsekli Lingjiong Zhu LRM 23 1 0 10 Feb 2023
Distributionally Robust Learning with Weakly Convex Losses: Convergence Rates and Finite-Sample Guarantees Landi Zhu Mert Gurbuzbalaban A. Ruszczynski 29 7 0 16 Jan 2023
BlueFog: Make Decentralized Algorithms Practical for Optimization and Deep Learning Bicheng Ying Kun Yuan Hanbin Hu Yiming Chen W. Yin FedML 31 27 0 08 Nov 2021