Scheduled Restart Momentum for Accelerated Stochastic Gradient Descent

Scheduled Restart Momentum for Accelerated Stochastic Gradient Descent

24 February 2020

Bao Wang

Andrea L. Bertozzi

Richard G. Baraniuk

Stanley J. Osher

Papers citing "Scheduled Restart Momentum for Accelerated Stochastic Gradient Descent"

12 / 12 papers shown

Title
MomentumSMoE: Integrating Momentum into Sparse Mixture of Experts R. Teo Tan M. Nguyen MoE 40 3 0 18 Oct 2024
Momentum Transformer: Closing the Performance Gap Between Self-attention and Its Linearization T. Nguyen Richard G. Baraniuk Robert M. Kirby Stanley J. Osher Bao Wang 44 9 0 01 Aug 2022
An Adaptive Gradient Method with Energy and Momentum Hailiang Liu Xuping Tian ODL 21 9 0 23 Mar 2022
Learning POD of Complex Dynamics Using Heavy-ball Neural ODEs Justin Baker E. Cherkaev A. Narayan Bao Wang AI4CE 24 4 0 24 Feb 2022
Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimization Tao Sun Huaming Ling Zuoqiang Shi Dongsheng Li Bao Wang ODL 27 13 0 18 Oct 2021
AIR-Net: Adaptive and Implicit Regularization Neural Network for Matrix Completion Zhemin Li Tao Sun Hongxia Wang Bao Wang 52 6 0 12 Oct 2021
Heavy Ball Neural Ordinary Differential Equations Hedi Xia Vai Suliafu H. Ji T. Nguyen Andrea L. Bertozzi Stanley J. Osher Bao Wang 45 56 0 10 Oct 2021
Accelerated Componentwise Gradient Boosting using Efficient Data Representation and Momentum-based Optimization Daniel Schalk B. Bischl David Rügamer 27 3 0 07 Oct 2021
Accelerated Gradient Descent Learning over Multiple Access Fading Channels Raz Paul Yuval Friedman Kobi Cohen 32 30 0 26 Jul 2021
SMG: A Shuffling Gradient-Based Method with Momentum Trang H. Tran Lam M. Nguyen Quoc Tran-Dinh 23 21 0 24 Nov 2020
Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers Robin M. Schmidt Frank Schneider Philipp Hennig ODL 47 162 0 03 Jul 2020
A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights Weijie Su Stephen P. Boyd Emmanuel J. Candes 108 1,157 0 04 Mar 2015