A Continuized View on Nesterov Acceleration for Stochastic Gradient
Descent and Randomized Gossip

A Continuized View on Nesterov Acceleration for Stochastic Gradient Descent and Randomized Gossip

10 June 2021

Raphael Berthier

Francis R. Bach

Nicolas Flammarion

Pierre Gaillard

Hadrien Hendrikx

Laurent Massoulié

Adrien B. Taylor

Papers citing "A Continuized View on Nesterov Acceleration for Stochastic Gradient Descent and Randomized Gossip"

15 / 15 papers shown

Title
Understanding the Generalization Error of Markov algorithms through Poissonization Benjamin Dupuis Maxime Haddouche George Deligiannidis Umut Simsekli 54 0 0 11 Feb 2025
Nesterov acceleration in benignly non-convex landscapes Kanan Gupta Stephan Wojtowytsch 42 2 0 10 Oct 2024
Generalized Continuous-Time Models for Nesterov's Accelerated Gradient Methods Chanwoong Park Youngchae Cho Insoon Yang 44 1 0 02 Sep 2024
ACCO: Accumulate While You Communicate for Communication-Overlapped Sharded LLM Training Adel Nabli Louis Fournier Pierre Erbacher Louis Serrano Eugene Belilovsky Edouard Oyallon FedML 54 1 0 03 Jun 2024
Fine-grained Analysis and Faster Algorithms for Iteratively Solving Linear Systems Michal Dereziñski Daniel LeJeune Deanna Needell E. Rebrova 52 4 0 09 May 2024
Faster Convergence of Stochastic Accelerated Gradient Descent under Interpolation Aaron Mishkin Mert Pilanci Mark Schmidt 71 1 0 03 Apr 2024
$$\textbf{A}^2\textbf{CiD}^2$: Accelerating Asynchronous Communication in Decentralized Deep Learning$ $\textbf{A}^2\textbf{CiD}^2$ : Accelerating Asynchronous Communication in Decentralized Deep Learning Adel Nabli Eugene Belilovsky Edouard Oyallon 24 6 0 14 Jun 2023
Continuized Acceleration for Quasar Convex Functions in Non-Convex Optimization Jun-Kun Wang Andre Wibisono 25 9 0 15 Feb 2023
DADAO: Decoupled Accelerated Decentralized Asynchronous Optimization Adel Nabli Edouard Oyallon 53 8 0 26 Jul 2022
Resonance in Weight Space: Covariate Shift Can Drive Divergence of SGD with Momentum Kirby Banman Liam Peet-Paré N. Hegde Alona Fyshe Martha White 31 0 0 22 Mar 2022
Acceleration of Gossip Algorithms through the Euler-Poisson-Darboux Equation Raphael Berthier Mufan Li 22 0 0 22 Feb 2022
The Implicit Regularization of Momentum Gradient Descent with Early Stopping Li Wang Yingcong Zhou Zhiguo Fu 34 2 0 14 Jan 2022
Asynchronous speedup in decentralized optimization Mathieu Even Hadrien Hendrikx Laurent Massoulie 34 4 0 07 Jun 2021
Acceleration Methods Alexandre d’Aspremont Damien Scieur Adrien B. Taylor 254 97 0 23 Jan 2021
A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights Weijie Su Stephen P. Boyd Emmanuel J. Candes 110 1,157 0 04 Mar 2015