AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights

15 June 2020

Papers citing "AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights"

36 / 36 papers shown

Title
Proxy Anchor Loss for Deep Metric Learning Sungyeon Kim Dongwon Kim Minsu Cho Suha Kwak 54 356 0 31 Mar 2020
An Empirical Evaluation on Robustness and Uncertainty of Regularization Methods Sanghyuk Chun Seong Joon Oh Sangdoo Yun Dongyoon Han Junsuk Choe Y. Yoo AAML OOD 393 53 0 09 Mar 2020
Learning De-biased Representations with Biased Representations Hyojin Bahng Sanghyuk Chun Sangdoo Yun Jaegul Choo Seong Joon Oh OOD 372 279 0 07 Oct 2019
On the Variance of the Adaptive Learning Rate and Beyond Liyuan Liu Haoming Jiang Pengcheng He Weizhu Chen Xiaodong Liu Jianfeng Gao Jiawei Han ODL 253 1,900 0 08 Aug 2019
Lookahead Optimizer: k steps forward, 1 step back Michael Ruogu Zhang James Lucas Geoffrey E. Hinton Jimmy Ba ODL 118 728 0 19 Jul 2019
Natural Adversarial Examples Dan Hendrycks Kevin Zhao Steven Basart Jacob Steinhardt D. Song OODD 193 1,465 0 16 Jul 2019
Toward Interpretable Music Tagging with Self-Attention Minz Won Sanghyuk Chun Xavier Serra ViT 46 82 0 12 Jun 2019
CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features Sangdoo Yun Dongyoon Han Seong Joon Oh Sanghyuk Chun Junsuk Choe Y. Yoo OOD 604 4,766 0 13 May 2019
On the Convergence of Adam and Beyond Sashank J. Reddi Satyen Kale Surinder Kumar 87 2,494 0 19 Apr 2019
Objects as Points Xingyi Zhou Dequan Wang Philipp Krahenbuhl 3DPC 101 3,249 0 16 Apr 2019
Theoretical Analysis of Auto Rate-Tuning by Batch Normalization Sanjeev Arora Zhiyuan Li Kaifeng Lyu 72 131 0 10 Dec 2018
ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness Robert Geirhos Patricia Rubisch Claudio Michaelis Matthias Bethge Felix Wichmann Wieland Brendel 96 2,662 0 29 Nov 2018
Three Mechanisms of Weight Decay Regularization Guodong Zhang Chaoqi Wang Bowen Xu Roger C. Grosse 62 258 0 29 Oct 2018
NSML: Meet the MLaaS platform with a real-world case study Hanjoo Kim Minkyu Kim Dongjoo Seo Jinwoong Kim Heungseok Park ... KyungHyun Kim Youngil Yang Youngkwan Kim Nako Sung Jung-Woo Ha 52 131 0 08 Oct 2018
Robustness May Be at Odds with Accuracy Dimitris Tsipras Shibani Santurkar Logan Engstrom Alexander Turner Aleksander Madry AAML 93 1,776 0 30 May 2018
How Does Batch Normalization Help Optimization? Shibani Santurkar Dimitris Tsipras Andrew Ilyas Aleksander Madry ODL 92 1,537 0 29 May 2018
Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition Pete Warden 74 1,615 0 09 Apr 2018
Group Normalization Yuxin Wu Kaiming He 204 3,644 0 22 Mar 2018
Norm matters: efficient and accurate normalization schemes in deep networks Elad Hoffer Ron Banner Itay Golan Daniel Soudry OffRL 59 179 0 05 Mar 2018
MobileNetV2: Inverted Residuals and Linear Bottlenecks Mark Sandler Andrew G. Howard Menglong Zhu A. Zhmoginov Liang-Chieh Chen 169 19,204 0 13 Jan 2018
The Implicit Bias of Gradient Descent on Separable Data Daniel Soudry Elad Hoffer Mor Shpigel Nacson Suriya Gunasekar Nathan Srebro 141 914 0 27 Oct 2017
Towards Deep Learning Models Resistant to Adversarial Attacks Aleksander Madry Aleksandar Makelov Ludwig Schmidt Dimitris Tsipras Adrian Vladu SILM OOD 277 12,029 0 19 Jun 2017
Deep Pyramidal Residual Networks Dongyoon Han Jiwhan Kim Junmo Kim 93 692 0 10 Oct 2016
SGDR: Stochastic Gradient Descent with Warm Restarts I. Loshchilov Frank Hutter ODL 288 8,091 0 13 Aug 2016
Instance Normalization: The Missing Ingredient for Fast Stylization Dmitry Ulyanov Andrea Vedaldi Victor Lempitsky OOD 156 3,701 0 27 Jul 2016
Layer Normalization Jimmy Lei Ba J. Kiros Geoffrey E. Hinton 346 10,467 0 21 Jul 2016
Wide Residual Networks Sergey Zagoruyko N. Komodakis 320 7,971 0 23 May 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.0K 193,426 0 10 Dec 2015
SSD: Single Shot MultiBox Detector Wen Liu Dragomir Anguelov D. Erhan Christian Szegedy Scott E. Reed Cheng-Yang Fu Alexander C. Berg ObjD BDL 200 29,742 0 08 Dec 2015
Deep Metric Learning via Lifted Structured Feature Embedding Hyun Oh Song Yu Xiang Stefanie Jegelka Silvio Savarese FedML SSL DML 94 1,641 0 19 Nov 2015
FaceNet: A Unified Embedding for Face Recognition and Clustering Florian Schroff Dmitry Kalenichenko James Philbin 3DH 342 13,123 0 12 Mar 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 430 43,234 0 11 Feb 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 1.5K 149,842 0 22 Dec 2014
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 1.5K 100,213 0 04 Sep 2014
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 1.5K 39,472 0 01 Sep 2014
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 379 43,524 0 01 May 2014