On the Convergence of Adam and Beyond

19 April 2019

Papers citing "On the Convergence of Adam and Beyond"

33 / 33 papers shown

Title
Implicit Neural Shape Optimization for 3D High-Contrast Electrical Impedance Tomography Junqing Chen Haibo Liu 142 0 0 22 May 2025
HOME-3: High-Order Momentum Estimator with Third-Power Gradient for Convex and Smooth Nonconvex Optimization Wei Zhang Arif Hassan Zidan Afrar Jahin Wei Zhang Tianming Liu ODL 42 0 0 16 May 2025
A Langevin sampling algorithm inspired by the Adam optimizer Benedict Leimkuhler René Lohmann Peter Whalley 104 0 0 26 Apr 2025
Understanding Gradient Orthogonalization for Deep Learning via Non-Euclidean Trust-Region Optimization Dmitry Kovalev 91 3 0 16 Mar 2025
Generative Adversarial Networks for High-Dimensional Item Factor Analysis: A Deep Adversarial Learning Algorithm Nanyu Luo Feng Ji DRL 57 0 0 15 Feb 2025
Automatic Live Music Song Identification Using Multi-level Deep Sequence Similarity Learning Aapo Hakala Trevor Kincy Tuomas Virtanen 132 0 0 14 Jan 2025
On the Performance Analysis of Momentum Method: A Frequency Domain Perspective Xianliang Li Jun Luo Zhiwei Zheng Hanxiao Wang Li Luo Lingkun Wen Linlong Wu Sheng Xu 103 0 0 29 Nov 2024
Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis Zhijie Chen Qiaobo Li A. Banerjee FedML 46 0 0 11 Nov 2024
Inductive Gradient Adjustment For Spectral Bias In Implicit Neural Representations Kexuan Shi Hai Chen Leheng Zhang Shuhang Gu 58 1 0 17 Oct 2024
Metamizer: a versatile neural optimizer for fast and accurate physics simulations Nils Wandel Stefan Schulz Reinhard Klein PINN AI4CE 64 1 0 10 Oct 2024
An Attention-Based Algorithm for Gravity Adaptation Zone Calibration Chen Yu 36 0 0 06 Oct 2024
Can Learned Optimization Make Reinforcement Learning Less Difficult? Alexander David Goldie Chris Xiaoxuan Lu Matthew Jackson Shimon Whiteson Jakob N. Foerster 67 3 0 09 Jul 2024
Loss Gradient Gaussian Width based Generalization and Optimization Guarantees A. Banerjee Qiaobo Li Yingxue Zhou 87 0 0 11 Jun 2024
Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance Dimitris Oikonomou Nicolas Loizou 64 5 0 06 Jun 2024
AdaFisher: Adaptive Second Order Optimization via Fisher Information Damien Martins Gomes Yanlei Zhang Eugene Belilovsky Guy Wolf Mahdi S. Hosseini ODL 108 2 0 26 May 2024
Dinomaly: The Less Is More Philosophy in Multi-Class Unsupervised Anomaly Detection Jia Guo Shuai Lu Weihang Zhang Huiqi Li Huiqi Li Hongen Liao ViT 94 10 0 23 May 2024
Q-Newton: Hybrid Quantum-Classical Scheduling for Accelerating Neural Network Training with Newton's Gradient Descent Pingzhi Li Junyu Liu Hanrui Wang Tianlong Chen 121 1 0 30 Apr 2024
Regularized Gradient Clipping Provably Trains Wide and Deep Neural Networks Matteo Tucat Anirbit Mukherjee Procheta Sen Mingfei Sun Omar Rivasplata MLT 51 1 0 12 Apr 2024
Variational Stochastic Gradient Descent for Deep Neural Networks Haotian Chen Anna Kuzina Babak Esmaeili Jakub M. Tomczak 59 0 0 09 Apr 2024
Conjugate-Gradient-like Based Adaptive Moment Estimation Optimization Algorithm for Deep Learning Jiawu Tian Liwei Xu Xiaowei Zhang Yongqi Li ODL 69 0 0 02 Apr 2024
Controlled Training Data Generation with Diffusion Models Teresa Yeo Andrei Atanov Harold Benoit Aleksandr Alekseev Ruchira Ray Pooya Esmaeil Akhoondi Amir Zamir 57 6 0 22 Mar 2024
M-HOF-Opt: Multi-Objective Hierarchical Output Feedback Optimization via Multiplier Induced Loss Landscape Scheduling Xudong Sun Nutan Chen Alexej Gossmann Yu Xing Carla Feistner ... Felix Drost Daniele Scarcella Lisa Beer Carsten Marr Carsten Marr 64 1 0 20 Mar 2024
Remove that Square Root: A New Efficient Scale-Invariant Version of AdaGrad Sayantan Choudhury N. Tupitsa Nicolas Loizou Samuel Horváth Martin Takáč Eduard A. Gorbunov 51 1 0 05 Mar 2024
On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions Yusu Hong Junhong Lin 76 13 0 06 Feb 2024
Non-asymptotic Analysis of Biased Adaptive Stochastic Approximation Sobihan Surendran Antoine Godichon-Baggioni Adeline Fermanian Sylvain Le Corff 73 1 0 05 Feb 2024
Efficient Computation of Sparse and Robust Maximum Association Estimators Pia Pfeiffer Andreas Alfons P. Filzmoser 29 0 0 29 Nov 2023
KOALA: A Kalman Optimization Algorithm with Loss Adaptivity A. Davtyan Sepehr Sameni L. Cerkezi Givi Meishvili Adam Bielski Paolo Favaro ODL 91 2 0 07 Jul 2021
Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning Sai Praneeth Karimireddy Martin Jaggi Satyen Kale M. Mohri Sashank J. Reddi Sebastian U. Stich A. Suresh FedML 61 216 0 08 Aug 2020
Deep Learning at the Edge Sahar Voghoei N. Tonekaboni Jason G. Wallace H. Arabnia 76 41 0 22 Oct 2019
Torchreid: A Library for Deep Learning Person Re-Identification in Pytorch Kaiyang Zhou Tao Xiang 69 118 0 22 Oct 2019
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 678 149,474 0 22 Dec 2014
ADADELTA: An Adaptive Learning Rate Method Matthew D. Zeiler ODL 106 6,619 0 22 Dec 2012
Adaptive Bound Optimization for Online Convex Optimization H. B. McMahan Matthew J. Streeter ODL 75 386 0 26 Feb 2010