Towards Practical Adam: Non-Convexity, Convergence Theory, and
Mini-Batch Acceleration

Towards Practical Adam: Non-Convexity, Convergence Theory, and Mini-Batch Acceleration

14 January 2021

Li Shen

Wei Liu

Papers citing "Towards Practical Adam: Non-Convexity, Convergence Theory, and Mini-Batch Acceleration"

7 / 7 papers shown

Title
Adaptive Extrapolated Proximal Gradient Methods with Variance Reduction for Composite Nonconvex Finite-Sum Minimization Ganzhao Yuan 43 0 0 28 Feb 2025
Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedup Yan Sun Li Shen Hao Sun Liang Ding Dacheng Tao FedML 24 17 0 30 Jul 2023
AdaTask: A Task-aware Adaptive Learning Rate Approach to Multi-task Learning Enneng Yang Junwei Pan Ximei Wang Haibin Yu Li Shen Xihua Chen Lei Xiao Jie Jiang G. Guo 38 43 0 28 Nov 2022
Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approach Peng Mi Li Shen Tianhe Ren Yiyi Zhou Xiaoshuai Sun Rongrong Ji Dacheng Tao AAML 38 69 0 11 Oct 2022
Adam Can Converge Without Any Modification On Update Rules Yushun Zhang Congliang Chen Naichen Shi Ruoyu Sun Zhimin Luo 18 63 0 20 Aug 2022
Efficient-Adam: Communication-Efficient Distributed Adam Congliang Chen Li Shen Wei Liu Zhi-Quan Luo 34 19 0 28 May 2022
A Simple Convergence Proof of Adam and Adagrad Alexandre Défossez Léon Bottou Francis R. Bach Nicolas Usunier 56 144 0 05 Mar 2020