Quasi-hyperbolic momentum and Adam for deep learning

16 October 2018

Papers citing "Quasi-hyperbolic momentum and Adam for deep learning"

17 / 17 papers shown

Title
On the Performance Analysis of Momentum Method: A Frequency Domain Perspective Xianliang Li Jun Luo Zhiwei Zheng Hanxiao Wang Li Luo Lingkun Wen Linlong Wu Sheng Xu 72 0 0 29 Nov 2024
Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance Dimitris Oikonomou Nicolas Loizou 55 4 0 06 Jun 2024
Symbolic Discovery of Optimization Algorithms Xiangning Chen Chen Liang Da Huang Esteban Real Kaiyuan Wang ... Xuanyi Dong Thang Luong Cho-Jui Hsieh Yifeng Lu Quoc V. Le 55 350 0 13 Feb 2023
T2G-Former: Organizing Tabular Features into Relation Graphs Promotes Heterogeneous Feature Interaction Jiahuan Yan Jintai Chen YiXuan Wu D. Z. Chen Jian Wu 22 35 0 30 Nov 2022
Self-Supervised Visual Representation Learning via Residual Momentum T. Pham Axi Niu Zhang Kang Sultan Rizky Hikmawan Madjid Jiajing Hong Daehyeok Kim Joshua Tian Jin Tee Chang-Dong Yoo SSL 46 6 0 17 Nov 2022
Multilevel-in-Layer Training for Deep Neural Network Regression Colin Ponce Ruipeng Li Christina Mao P. Vassilevski AI4CE 19 1 0 11 Nov 2022
On the Pros and Cons of Momentum Encoder in Self-Supervised Visual Representation Learning T. Pham Chaoning Zhang Axi Niu Kang Zhang Chang-Dong Yoo 36 11 0 11 Aug 2022
Gradient Temporal Difference with Momentum: Stability and Convergence Rohan Deb S. Bhatnagar 11 5 0 22 Nov 2021
Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimization Tao Sun Huaming Ling Zuoqiang Shi Dongsheng Li Bao Wang ODL 19 13 0 18 Oct 2021
Revisiting the Role of Euler Numerical Integration on Acceleration and Stability in Convex Optimization Peiyuan Zhang Antonio Orvieto Hadi Daneshmand Thomas Hofmann Roy S. Smith 13 9 0 23 Feb 2021
Noise and Fluctuation of Finite Learning Rate Stochastic Gradient Descent Kangqiao Liu Liu Ziyin Masakuni Ueda MLT 59 37 0 07 Dec 2020
Review: Deep Learning in Electron Microscopy Jeffrey M. Ede 29 79 0 17 Sep 2020
Flexible numerical optimization with ensmallen Ryan R. Curtin Marcus Edel Rahul Prabhu S. Basak Zhihao Lou Conrad Sanderson 14 1 0 09 Mar 2020
Statistical Adaptive Stochastic Gradient Methods Pengchuan Zhang Hunter Lang Qiang Liu Lin Xiao ODL 11 11 0 25 Feb 2020
Understanding the Role of Momentum in Stochastic Gradient Methods Igor Gitman Hunter Lang Pengchuan Zhang Lin Xiao 17 93 0 30 Oct 2019
Demon: Improved Neural Network Training with Momentum Decay John Chen Cameron R. Wolfe Zhaoqi Li Anastasios Kyrillidis ODL 16 15 0 11 Oct 2019
On the adequacy of untuned warmup for adaptive optimization Jerry Ma Denis Yarats 51 70 0 09 Oct 2019