A general system of differential equations to model first order adaptive algorithms

31 October 2018

Papers citing "A general system of differential equations to model first order adaptive algorithms"

7 / 7 papers shown

Title
A Langevin sampling algorithm inspired by the Adam optimizer B. Leimkuhler René Lohmann P. Whalley 79 0 0 26 Apr 2025
Understanding the robustness difference between stochastic gradient descent and adaptive gradient methods A. Ma Yangchen Pan Amir-massoud Farahmand AAML 25 5 0 13 Aug 2023
A Dynamical View on Optimization Algorithms of Overparameterized Neural Networks Zhiqi Bu Shiyun Xu Kan Chen 33 17 0 25 Oct 2020
A Qualitative Study of the Dynamic Behavior for Adaptive Gradient Algorithms Chao Ma Lei Wu E. Weinan ODL 19 23 0 14 Sep 2020
LaProp: Separating Momentum and Adaptivity in Adam Liu Ziyin Zhikang T.Wang Masahito Ueda ODL 13 18 0 12 Feb 2020
First-order Methods Almost Always Avoid Saddle Points J. Lee Ioannis Panageas Georgios Piliouras Max Simchowitz Michael I. Jordan Benjamin Recht ODL 95 83 0 20 Oct 2017
A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights Weijie Su Stephen P. Boyd Emmanuel J. Candes 108 1,157 0 04 Mar 2015