An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks

21 December 2013

Aaron Courville

Papers citing "An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks"

50 / 306 papers shown

Title
CLeaR: An Adaptive Continual Learning Framework for Regression Tasks Yujiang He Bernhard Sick CLL 54 38 0 04 Jan 2021
Efficient Continual Learning with Modular Networks and Task-Driven Priors Tom Véniat Ludovic Denoyer MarcÁurelio Ranzato CLL 30 97 0 23 Dec 2020
Positive-Congruent Training: Towards Regression-Free Model Updates Sijie Yan Yuanjun Xiong Kaustav Kundu Shuo Yang Siqi Deng Meng Wang Wei Xia Stefano Soatto BDL 31 49 0 18 Nov 2020
Artificial Neural Variability for Deep Learning: On Overfitting, Noise Memorization, and Catastrophic Forgetting Zeke Xie Fengxiang He Shaopeng Fu Issei Sato Dacheng Tao Masashi Sugiyama 21 60 0 12 Nov 2020
AbdomenCT-1K: Is Abdominal Organ Segmentation A Solved Problem? Jun Ma Yao Zhang Song Gu Cheng Zhu Cheng Ge ... Shangqing Liu Yunpeng Wang Yuhui Li Jian He Xiaoping Yang SSeg OOD 26 328 0 28 Oct 2020
The LMU Munich System for the WMT 2020 Unsupervised Machine Translation Shared Task Alexandra Chronopoulou Dario Stojanovski Viktor Hangya Alexander Fraser 37 5 0 25 Oct 2020
Regularizing Towards Permutation Invariance in Recurrent Models Edo Cohen-Karlik Avichai Ben David Amir Globerson OOD 24 16 0 25 Oct 2020
Continual Learning in Low-rank Orthogonal Subspaces Arslan Chaudhry Naeemullah Khan P. Dokania Philip Torr CLL 33 115 0 22 Oct 2020
CLAR: Contrastive Learning of Auditory Representations Haider Al-Tahan Y. Mohsenzadeh SSL 125 56 0 19 Oct 2020
FedAT: A High-Performance and Communication-Efficient Federated Learning System with Asynchronous Tiers Zheng Chai Yujing Chen Ali Anwar Liang Zhao Yue Cheng Huzefa Rangwala FedML 26 122 0 12 Oct 2020
Linear Mode Connectivity in Multitask and Continual Learning Seyed Iman Mirzadeh Mehrdad Farajtabar Dilan Görür Razvan Pascanu H. Ghasemzadeh CLL 37 139 0 09 Oct 2020
A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrix T. Doan Mehdi Abbana Bennani Bogdan Mazoure Guillaume Rabusseau Pierre Alquier CLL 20 80 0 07 Oct 2020
Robust High-dimensional Memory-augmented Neural Networks G. Karunaratne Manuel Schmuck Manuel Le Gallo G. Cherubini Luca Benini Abu Sebastian Abbas Rahimi 26 99 0 05 Oct 2020
Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT Alexandra Chronopoulou Dario Stojanovski Alexander Fraser 21 33 0 16 Sep 2020
An Empirical Analysis of Backward Compatibility in Machine Learning Systems Megha Srivastava Besmira Nushi Ece Kamar S. Shah Eric Horvitz AAML 34 44 0 11 Aug 2020
Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics V. Ramasesh Ethan Dyer M. Raghu CLL 24 174 0 14 Jul 2020
RATT: Recurrent Attention to Transient Tasks for Continual Image Captioning Riccardo Del Chiaro Bartlomiej Twardowski Andrew D. Bagdanov Joost van de Weijer CLL VLM 30 40 0 13 Jul 2020
Graph-Based Continual Learning Binh Tang David S. Matteson BDL CLL 34 37 0 09 Jul 2020
Gradient-based Editing of Memory Examples for Online Task-free Continual Learning Xisen Jin Arka Sadhu Junyi Du Xiang Ren CLL KELM BDL 28 96 0 27 Jun 2020
Supermasks in Superposition Mitchell Wortsman Vivek Ramanujan Rosanne Liu Aniruddha Kembhavi Mohammad Rastegari J. Yosinski Ali Farhadi SSL CLL 33 281 0 26 Jun 2020
Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descent Mehdi Abbana Bennani Thang Doan Masashi Sugiyama CLL 50 61 0 21 Jun 2020
Understanding the Role of Training Regimes in Continual Learning Seyed Iman Mirzadeh Mehrdad Farajtabar Razvan Pascanu H. Ghasemzadeh CLL 21 219 0 12 Jun 2020
Coresets via Bilevel Optimization for Continual Learning and Streaming Zalan Borsos Mojmír Mutný Andreas Krause CLL 38 227 0 06 Jun 2020
Learning to Recognize Code-switched Speech Without Forgetting Monolingual Speech Recognition Sanket Shah Basil Abraham M. GurunathReddy Sunayana Sitaram Vikas Joshi 21 18 0 01 Jun 2020
Common Sense or World Knowledge? Investigating Adapter-Based Knowledge Injection into Pretrained Transformers Anne Lauscher Olga Majewska Leonardo F. R. Ribeiro Iryna Gurevych Nikolai Rozanov Goran Glavaš KELM 39 79 0 24 May 2020
Spiking Neural Networks Hardware Implementations and Challenges: a Survey Maxence Bouvier A. Valentian T. Mesquida F. Rummens M. Reyboz Elisa Vianello E. Beigné 56 113 0 04 May 2020
Explaining How Deep Neural Networks Forget by Deep Visualization Giang Nguyen Shuan Chen Tae Joon Jun Daeyoung Kim AAML 8 10 0 03 May 2020
Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting Sanyuan Chen Yutai Hou Yiming Cui Wanxiang Che Ting Liu Xiangzhan Yu KELM CLL 21 214 0 27 Apr 2020
Generative Data Augmentation for Commonsense Reasoning Yiben Yang Chaitanya Malaviya Jared Fernandez Swabha Swayamdipta Ronan Le Bras Ji-ping Wang Chandra Bhagavatula Yejin Choi Doug Downey LRM 30 91 0 24 Apr 2020
Continual Learning with Extended Kronecker-factored Approximate Curvature Janghyeon Lee H. Hong Donggyu Joo Junmo Kim CLL 20 52 0 16 Apr 2020
Unpacking Information Bottlenecks: Unifying Information-Theoretic Objectives in Deep Learning Andreas Kirsch Clare Lyle Y. Gal 27 16 0 27 Mar 2020
An Energy-Aware Online Learning Framework for Resource Management in Heterogeneous Platforms Sumit K. Mandal Ganapati Bhat J. Doppa P. Pande Ümit Y. Ogras 19 39 0 20 Mar 2020
Synaptic Metaplasticity in Binarized Neural Networks Axel Laborieux M. Ernoult T. Hirtzlin D. Querlioz CLL 26 62 0 07 Mar 2020
Training Binary Neural Networks using the Bayesian Learning Rule Xiangming Meng Roman Bachmann Mohammad Emtiyaz Khan BDL MQ 32 40 0 25 Feb 2020
Three Approaches for Personalization with Applications to Federated Learning Yishay Mansour M. Mohri Jae Hun Ro A. Suresh FedML 45 566 0 25 Feb 2020
Using Hindsight to Anchor Past Knowledge in Continual Learning Arslan Chaudhry Albert Gordo P. Dokania Philip Torr David Lopez-Paz KELM CLL 30 234 0 19 Feb 2020
Effective Data Augmentation with Multi-Domain Learning GANs Shin'ya Yamaguchi Sekitoshi Kanai Takeharu Eda 27 27 0 25 Dec 2019
Direction Concentration Learning: Enhancing Congruency in Machine Learning Yan Luo Yongkang Wong Mohan Kankanhalli Qi Zhao 23 12 0 17 Dec 2019
Large-scale Kernel Methods and Applications to Lifelong Robot Learning Raffaello Camoriano 42 1 0 11 Dec 2019
Neural Machine Translation: A Review and Survey Felix Stahlberg 3DV AI4TS MedIm 30 313 0 04 Dec 2019
Adaptive Online Planning for Continual Lifelong Learning Kevin Lu Igor Mordatch Pieter Abbeel OffRL OnRL CLL 11 15 0 03 Dec 2019
Latent Replay for Real-Time Continual Learning Lorenzo Pellegrini G. Graffieti Vincenzo Lomonaco Davide Maltoni CLL 28 163 0 02 Dec 2019
REFIT: A Unified Watermark Removal Framework For Deep Learning Systems With Limited Data Xinyun Chen Wenxiao Wang Chris Bender Yiming Ding R. Jia Bo-wen Li D. Song AAML 27 107 0 17 Nov 2019
OpenLORIS-Object: A Robotic Vision Dataset and Benchmark for Lifelong Deep Learning Qi She Fan Feng Xinyue Hao Qihan Yang Chuanlin Lan ... Zhengwei Wang Yao Guo Yimin Zhang Fei Qiao Rosa H. M. Chan VLM 38 75 0 15 Nov 2019
Knowledge Distillation for Incremental Learning in Semantic Segmentation Umberto Michieli Pietro Zanuttigh CLL VLM 28 98 0 08 Nov 2019
Adversarial Feature Alignment: Avoid Catastrophic Forgetting in Incremental Task Lifelong Learning Hang Zhou Ziwei Liu Chenglei Wu Ping Luo Lifeng Sun CLL 17 22 0 24 Oct 2019
Continual Learning Using Bayesian Neural Networks Honglin Li Payam Barnaghi Shirin Enshaeifar F. Ganz BDL 19 38 0 09 Oct 2019
Gated Linear Networks William H. Guss Tor Lattimore David Budden Avishkar Bhoopchand Christopher Mattern ... Ruslan Salakhutdinov Jianan Wang Peter Toth Simon Schmitt Marcus Hutter AI4CE 18 40 0 30 Sep 2019
Meta-learnt priors slow down catastrophic forgetting in neural networks G. Spigler CLL 25 10 0 09 Sep 2019
Domain Adaptive Text Style Transfer Dianqi Li Yizhe Zhang Zhe Gan Yu Cheng Chris Brockett Ming-Ting Sun W. Dolan OOD 31 55 0 25 Aug 2019