Does the Adam Optimizer Exacerbate Catastrophic Forgetting?

v1v2v3v4 (latest)

Does the Adam Optimizer Exacerbate Catastrophic Forgetting?

15 February 2021

Dylan R. Ashley

ArXiv (abs)PDF HTML

Papers citing "Does the Adam Optimizer Exacerbate Catastrophic Forgetting?"

11 / 11 papers shown

Title
Continual learning with the neural tangent ensemble Ari S. Benjamin Christian Pehle Kyle Daruwalla UQCV 122 1 0 30 Aug 2024
Understanding the Role of Training Regimes in Continual Learning Seyed Iman Mirzadeh Mehrdad Farajtabar Razvan Pascanu H. Ghasemzadeh CLL 76 226 0 12 Jun 2020
Improving Performance in Reinforcement Learning by Breaking Generalization in Neural Networks Sina Ghiassian Banafsheh Rafiee Y. Lo Adam White 42 31 0 16 Mar 2020
Similarity of Neural Network Representations Revisited Simon Kornblith Mohammad Norouzi Honglak Lee Geoffrey E. Hinton 141 1,429 0 01 May 2019
Towards Robust Evaluations of Continual Learning Sebastian Farquhar Y. Gal CLL 93 307 0 24 May 2018
Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms Han Xiao Kashif Rasul Roland Vollgraf 283 8,904 0 25 Aug 2017
Overcoming Catastrophic Forgetting by Incremental Moment Matching Sang-Woo Lee Jin-Hwa Kim Jaehyun Jun Jung-Woo Ha Byoung-Tak Zhang CLL 70 679 0 24 Mar 2017
Overcoming catastrophic forgetting in neural networks J. Kirkpatrick Razvan Pascanu Neil C. Rabinowitz J. Veness Guillaume Desjardins ... A. Grabska-Barwinska Demis Hassabis Claudia Clopath D. Kumaran R. Hadsell CLL 372 7,547 0 02 Dec 2016
OpenAI Gym Greg Brockman Vicki Cheung Ludwig Pettersson Jonas Schneider John Schulman Jie Tang Wojciech Zaremba OffRL ODL 223 5,085 0 05 Jun 2016
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification Kaiming He Xinming Zhang Shaoqing Ren Jian Sun VLM 332 18,647 0 06 Feb 2015
An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks Ian Goodfellow M. Berk Mirza Xia Da Aaron Courville Yoshua Bengio 149 1,449 0 21 Dec 2013