A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights

4 March 2015

Weijie Su

Emmanuel J. Candes

Papers citing "A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights"

50 / 73 papers shown

Title
Accelerated Stein Variational Gradient Flow Viktor Stein Wuchen Li 56 0 0 30 Mar 2025
Functional multi-armed bandit and the best function identification problems Yuriy Dorn Aleksandr Katrutsa Ilgam Latypov Anastasiia Soboleva 32 0 0 01 Mar 2025
Nesterov Acceleration for Ensemble Kalman Inversion and Variants Sydney Vernon Eviatar Bach Oliver R. A. Dunbar 39 1 0 15 Jan 2025
Nesterov acceleration in benignly non-convex landscapes Kanan Gupta Stephan Wojtowytsch 36 2 0 10 Oct 2024
Distributed Stochastic Gradient Descent with Staleness: A Stochastic Delay Differential Equation Based Framework Siyuan Yu Wei Chen H. V. Poor 24 0 0 17 Jun 2024
Distributed Event-Based Learning via ADMM Güner Dilsad Er Sebastian Trimpe Michael Muehlebach FedML 41 2 0 17 May 2024
A General Continuous-Time Formulation of Stochastic ADMM and Its Variants Chris Junchi Li 20 0 0 22 Apr 2024
Quantum Langevin Dynamics for Optimization Zherui Chen Yuchen Lu Hao Wang Yizhou Liu Tongyang Li AI4CE 21 9 0 27 Nov 2023
Accelerating optimization over the space of probability measures Shi Chen Wenxuan Wu Yuhang Yao Stephen J. Wright 26 4 0 06 Oct 2023
An Element-wise RSAV Algorithm for Unconstrained Optimization Problems Shiheng Zhang Jiahao Zhang Jie Shen Guang Lin 26 2 0 07 Sep 2023
Diffusion Sampling with Momentum for Mitigating Divergence Artifacts Suttisak Wizadwongsa Worameth Chinchuthakun Pramook Khungurn Amit Raj Supasorn Suwajanakorn DiffM 39 2 0 20 Jul 2023
A Passivity-Based Method for Accelerated Convex Optimisation Namhoon Cho Hyo-Sang Shin 13 0 0 20 Jun 2023
On Underdamped Nesterov's Acceleration Shu Chen Bin Shi Ya-xiang Yuan 19 5 0 28 Apr 2023
Reducing Discretization Error in the Frank-Wolfe Method Zhaoyue Chen Yifan Sun 13 1 0 04 Apr 2023
Sublinear Convergence Rates of Extragradient-Type Methods: A Survey on Classical and Recent Developments Quoc Tran-Dinh 30 7 0 30 Mar 2023
Quantum Hamiltonian Descent Jiaqi Leng Ethan Hickman Joseph Li Xiaodi Wu 18 12 0 02 Mar 2023
Accelerated First-Order Optimization under Nonlinear Constraints Michael Muehlebach Michael I. Jordan 40 3 0 01 Feb 2023
An SDE for Modeling SAM: Theory and Insights Enea Monzio Compagnoni Luca Biggio Antonio Orvieto F. Proske Hans Kersting Aurélien Lucchi 23 13 0 19 Jan 2023
A Nonstochastic Control Approach to Optimization Xinyi Chen Elad Hazan 47 5 0 19 Jan 2023
Restarts subject to approximate sharpness: A parameter-free and optimal scheme for first-order methods Ben Adcock Matthew J. Colbrook Maksym Neyra-Nesterenko 27 2 0 05 Jan 2023
Effects of Data Geometry in Early Deep Learning Saket Tiwari G. Konidaris 69 7 0 29 Dec 2022
Gradient Descent-Type Methods: Background and Simple Unified Convergence Analysis Quoc Tran-Dinh Marten van Dijk 28 0 0 19 Dec 2022
Proximal Subgradient Norm Minimization of ISTA and FISTA Bowen Li Bin Shi Ya-xiang Yuan 21 11 0 03 Nov 2022
Toward Equation of Motion for Deep Neural Networks: Continuous-time Gradient Descent and Discretization Error Analysis Taiki Miyagawa 35 9 0 28 Oct 2022
Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models Hong Liu Sang Michael Xie Zhiyuan Li Tengyu Ma AI4CE 32 49 0 25 Oct 2022
From Gradient Flow on Population Loss to Learning with Stochastic Gradient Descent Satyen Kale Jason D. Lee Chris De Sa Ayush Sekhari Karthik Sridharan 24 4 0 13 Oct 2022
On Quantum Speedups for Nonconvex Optimization via Quantum Tunneling Walks Yizhou Liu Weijie J. Su Tongyang Li 16 17 0 29 Sep 2022
Gradient Norm Minimization of Nesterov Acceleration: $o(1/k^3)$ Shu Chen Bin Shi Ya-xiang Yuan 28 15 0 19 Sep 2022
Multilevel Geometric Optimization for Regularised Constrained Linear Inverse Problems Sebastian Müller Stefania Petra Matthias Zisler AI4CE 13 1 0 11 Jul 2022
Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents Alessandro Barp Lancelot Da Costa G. Francca Karl J. Friston Mark Girolami Michael I. Jordan G. Pavliotis 28 25 0 20 Mar 2022
Sparse Neural Additive Model: Interpretable Deep Learning with Feature Selection via Group Sparsity Shiyun Xu Zhiqi Bu Pratik Chaudhari Ian J. Barnett 19 21 0 25 Feb 2022
Convergence Rates of Two-Time-Scale Gradient Descent-Ascent Dynamics for Solving Nonconvex Min-Max Problems Thinh T. Doan 18 15 0 17 Dec 2021
A More Stable Accelerated Gradient Method Inspired by Continuous-Time Perspective Yasong Feng Weiguo Gao 15 0 0 09 Dec 2021
Breaking the Convergence Barrier: Optimization via Fixed-Time Convergent Flows Param Budhraja Mayank Baranwal Kunal Garg A. Hota 13 9 0 02 Dec 2021
Convergence and Stability of the Stochastic Proximal Point Algorithm with Momentum J. Kim Panos Toulis Anastasios Kyrillidis 22 8 0 11 Nov 2021
On Accelerating Distributed Convex Optimizations Kushal Chakrabarti Nirupam Gupta Nikhil Chopra 16 7 0 19 Aug 2021
On Constraints in First-Order Optimization: A View from Non-Smooth Dynamical Systems Michael Muehlebach Michael I. Jordan 26 18 0 17 Jul 2021
ANITA: An Optimal Loopless Accelerated Variance-Reduced Gradient Method Zhize Li 30 14 0 21 Mar 2021
Revisiting the Role of Euler Numerical Integration on Acceleration and Stability in Convex Optimization Peiyuan Zhang Antonio Orvieto Hadi Daneshmand Thomas Hofmann Roy S. Smith 11 9 0 23 Feb 2021
Advances in Electron Microscopy with Deep Learning Jeffrey M. Ede 29 2 0 04 Jan 2021
First-Order Methods for Convex Optimization Pavel Dvurechensky Mathias Staudigl Shimrit Shtern ODL 16 25 0 04 Jan 2021
AEGD: Adaptive Gradient Descent with Energy Hailiang Liu Xuping Tian ODL 25 11 0 10 Oct 2020
Federated Learning with Nesterov Accelerated Gradient Zhengjie Yang Wei Bao Dong Yuan Nguyen H. Tran Albert Y. Zomaya FedML 19 29 0 18 Sep 2020
Review: Deep Learning in Electron Microscopy Jeffrey M. Ede 26 79 0 17 Sep 2020
Obtaining Adjustable Regularization for Free via Iterate Averaging Jingfeng Wu Vladimir Braverman Lin F. Yang 25 2 0 15 Aug 2020
Variance Reduction via Accelerated Dual Averaging for Finite-Sum Optimization Chaobing Song Yong Jiang Yi-An Ma 40 23 0 18 Jun 2020
On dissipative symplectic integration with applications to gradient-based optimization G. Francca Michael I. Jordan René Vidal 11 47 0 15 Apr 2020
Stochastic Modified Equations for Continuous Limit of Stochastic ADMM Xiang Zhou Huizhuo Yuan C. J. Li Qingyun Sun 15 6 0 07 Mar 2020
A Newton Frank-Wolfe Method for Constrained Self-Concordant Minimization Deyi Liu V. Cevher Quoc Tran-Dinh 21 15 0 17 Feb 2020
On the Effectiveness of Richardson Extrapolation in Machine Learning Francis R. Bach 11 9 0 07 Feb 2020