Title
Cautious Optimizers: Improving Training with One Line of Code Kaizhao Liang Lizhang Chen B. Liu Qiang Liu ODL 195 9 0 25 Nov 2024
On the Convergence of Adam under Non-uniform Smoothness: Separability from SGDM and Beyond Bohan Wang Huishuai Zhang Qi Meng Ruoyu Sun Zhi-Ming Ma Wei Chen 66 11 0 22 Mar 2024
Closing the Gap Between the Upper Bound and the Lower Bound of Adam's Iteration Complexity Bohan Wang Jingwen Fu Huishuai Zhang Nanning Zheng Wei Chen 49 19 0 27 Oct 2023
A Gradient-based Approach for Online Robust Deep Neural Network Training with Noisy Labels Yifan Yang Alec Koppel Zheng Zhang NoLa 53 3 0 08 Jun 2023
Convergence of Adam Under Relaxed Assumptions Haochuan Li Alexander Rakhlin Ali Jadbabaie 80 65 0 27 Apr 2023
Symbolic Discovery of Optimization Algorithms Xiangning Chen Chen Liang Da Huang Esteban Real Kaiyuan Wang ... Xuanyi Dong Thang Luong Cho-Jui Hsieh Yifeng Lu Quoc V. Le 152 374 0 13 Feb 2023
Adam Can Converge Without Any Modification On Update Rules Yushun Zhang Congliang Chen Naichen Shi Ruoyu Sun Zhimin Luo 51 68 0 20 Aug 2022
Adaptive Online Incremental Learning for Evolving Data Streams Siyun Zhang Jian-wei Liu Xin Zuo CLL 51 30 0 05 Jan 2022
Learning Robust Recommender from Noisy Implicit Feedback Wenjie Wang Fuli Feng Xiangnan He Liqiang Nie Tat-Seng Chua NoLa 62 3 0 02 Dec 2021
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 657 41,103 0 22 Oct 2020
AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients Juntang Zhuang Tommy M. Tang Yifan Ding S. Tatikonda Nicha Dvornek X. Papademetris James S. Duncan ODL 162 517 0 15 Oct 2020
BARS-CTR: Open Benchmarking for Click-Through Rate Prediction Jieming Zhu Jinyang Liu Shuai Yang Qi Zhang Xiuqiang He 71 129 0 12 Sep 2020
Online Robust and Adaptive Learning from Data Streams Shintaro Fukushima Atsushi Nitanda Kenji Yamanishi 59 3 0 23 Jul 2020
Learning under Concept Drift: A Review Jie Lu Anjin Liu Fan Dong Feng Gu João Gama Guangquan Zhang AI4TS 65 1,282 0 13 Apr 2020
A new regret analysis for Adam-type algorithms Ahmet Alacaoglu Yura Malitsky P. Mertikopoulos Volkan Cevher ODL 58 42 0 21 Mar 2020
A Simple Convergence Proof of Adam and Adagrad Alexandre Défossez Léon Bottou Francis R. Bach Nicolas Usunier 109 155 0 05 Mar 2020
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 514 42,449 0 03 Dec 2019
On the Variance of the Adaptive Learning Rate and Beyond Liyuan Liu Haoming Jiang Pengcheng He Weizhu Chen Xiaodong Liu Jianfeng Gao Jiawei Han ODL 287 1,905 0 08 Aug 2019
On the Convergence of Adam and Beyond Sashank J. Reddi Satyen Kale Surinder Kumar 96 2,499 0 19 Apr 2019
Robust Loss Functions under Label Noise for Deep Neural Networks Aritra Ghosh Himanshu Kumar P. Sastry NoLa OOD 70 957 0 27 Dec 2017
Deep & Cross Network for Ad Click Predictions Ruoxi Wang Bin Fu Gang Fu Mingliang Wang 104 1,233 0 17 Aug 2017
Proximal Policy Optimization Algorithms John Schulman Filip Wolski Prafulla Dhariwal Alec Radford Oleg Klimov OffRL 499 19,065 0 20 Jul 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 701 131,652 0 12 Jun 2017
DeepFM: A Factorization-Machine based Neural Network for CTR Prediction Huifeng Guo Ruiming Tang Yunming Ye Zhenguo Li Xiuqiang He 120 2,650 0 13 Mar 2017
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 823 11,909 0 09 Mar 2017
Product-based Neural Networks for User Response Prediction Yanru Qu Han Cai Kan Ren Weinan Zhang Yong Yu Ying Wen Jun Wang 86 716 0 01 Nov 2016
Densely Connected Convolutional Networks Gao Huang Zhuang Liu Laurens van der Maaten Kilian Q. Weinberger PINN 3DV 772 36,813 0 25 Aug 2016
Wide & Deep Learning for Recommender Systems Heng-Tze Cheng L. Koc Jeremiah Harmsen T. Shaked Tushar Chandra ... Zakaria Haque Lichan Hong Vihan Jain Xiaobing Liu Hemal Shah HAI VLM 179 3,659 0 24 Jun 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 194,020 0 10 Dec 2015
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 1.6K 100,386 0 04 Sep 2014
ADADELTA: An Adaptive Learning Rate Method Matthew D. Zeiler ODL 155 6,625 0 22 Dec 2012