v1v2 (latest)

A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning

16 July 2023

Li Shen

ArXiv (abs)PDF HTML Github (305★)

Papers citing "A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning"

50 / 126 papers shown

Title
A Unified and General Framework for Continual Learning Zhenyi Wang Yan Li Li Shen Heng-Chiao Huang CLL 84 22 0 20 Mar 2024
Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks Mingqing Xiao Qingyan Meng Zongpeng Zhang D.K. He Zhouchen Lin CLL 102 8 0 19 Feb 2024
Continual Learning in Linear Classification on Separable Data Itay Evron E. Moroshko G. Buzaglo M. Khriesh B. Marjieh Nathan Srebro Daniel Soudry CLL 72 17 0 06 Jun 2023
EcoTTA: Memory-Efficient Continual Test-time Adaptation via Self-distilled Regularization Jun S. Song Jungsoo Lee In So Kweon Sungha Choi TTA 76 94 0 03 Mar 2023
Learning to Retain while Acquiring: Combating Distribution-Shift in Adversarial Data-Free Knowledge Distillation Gaurav Patel Konda Reddy Mopuri Qiang Qiu 53 28 0 28 Feb 2023
Analyzing Leakage of Personally Identifiable Information in Language Models Nils Lukas A. Salem Robert Sim Shruti Tople Lukas Wutschitz Santiago Zanella Béguelin PILM 144 233 0 01 Feb 2023
Progressive Prompts: Continual Learning for Language Models Anastasia Razdaibiedina Yuning Mao Rui Hou Madian Khabsa M. Lewis Amjad Almahairi VLM KELM CLL 106 142 0 29 Jan 2023
Robust Mean Teacher for Continual and Gradual Test-Time Adaptation Mario Döbler Robert A. Marsden Bin Yang OOD TTA 66 90 0 23 Nov 2022
Navigating Memory Construction by Global Pseudo-Task Simulation for Continual Learning Yejia Liu Wang Zhu Shaolei Ren CLL 56 3 0 16 Oct 2022
Momentum Adversarial Distillation: Handling Large Distribution Shifts in Data-Free Knowledge Distillation Kien Do Hung Le D. Nguyen Dang Nguyen Haripriya Harikumar T. Tran Santu Rana Svetha Venkatesh 50 33 0 21 Sep 2022
Model-Free Generative Replay for Lifelong Reinforcement Learning: Application to Starcraft-2 Z. Daniels Aswin Raghavan Jesse Hostetler Abrar Rahman Indranil Sur M. Piacentino Ajay Divakaran CLL OffRL 78 13 0 09 Aug 2022
S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learning Yabin Wang Zhiwu Huang Xiaopeng Hong CLL VLM 120 223 0 26 Jul 2022
Online Domain Adaptation for Semantic Segmentation in Ever-Changing Conditions Theodoros Panagiotakopoulos Pier Luigi Dovesi Linus Harenstam-Nielsen Matteo Poggi CLL 57 19 0 21 Jul 2022
Continual Variational Autoencoder Learning via Online Cooperative Memorization Fei Ye A. Bors CLL 76 17 0 20 Jul 2022
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition Jorge Armando Mendez Mendez Eric Eaton KELM CLL 103 29 0 15 Jul 2022
Improving Task-free Continual Learning by Distributionally Robust Memory Evolution Zhenyi Wang Li Shen Le Fang Qiuling Suo Tiehang Duan Mingchen Gao OOD 81 42 0 15 Jul 2022
Measuring Forgetting of Memorized Training Examples Matthew Jagielski Om Thakkar Florian Tramèr Daphne Ippolito Katherine Lee ... Eric Wallace Shuang Song Abhradeep Thakurta Nicolas Papernot Chiyuan Zhang TDI 147 111 0 30 Jun 2022
NISPA: Neuro-Inspired Stability-Plasticity Adaptation for Continual Learning in Sparse Networks Mustafa Burak Gurbuz C. Dovrolis CLL 60 48 0 18 Jun 2022
A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learning Da-Wei Zhou Qiwen Wang Han-Jia Ye De-Chuan Zhan 76 137 0 26 May 2022
Fine-tuned Language Models are Continual Learners Thomas Scialom Tuhin Chakrabarty Smaranda Muresan CLL LRM 189 122 0 24 May 2022
DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning Zifeng Wang Zizhao Zhang Sayna Ebrahimi Ruoxi Sun Han Zhang ... Xiaoqi Ren Guolong Su Vincent Perot Jennifer Dy Tomas Pfister CLL VLM VPVLM 128 502 0 10 Apr 2022
Information-theoretic Online Memory Selection for Continual Learning Shengyang Sun Daniele Calandriello Huiyi Hu Ang Li Michalis K. Titsias CLL 52 52 0 10 Apr 2022
Efficient Test-Time Model Adaptation without Forgetting Shuaicheng Niu Jiaxiang Wu Yifan Zhang Yaofo Chen S. Zheng P. Zhao Mingkui Tan OOD VLM TTA 87 350 0 06 Apr 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 531 6,293 0 05 Apr 2022
Constrained Few-shot Class-incremental Learning Michael Hersche G. Karunaratne G. Cherubini Luca Benini Abu Sebastian Abbas Rahimi CLL 106 142 0 30 Mar 2022
Continual Test-Time Domain Adaptation Qin Wang Olga Fink Luc Van Gool Dengxin Dai OOD TTA 110 431 0 25 Mar 2022
The Challenges of Continuous Self-Supervised Learning Senthil Purushwalkam Pedro Morgado Abhinav Gupta CLL 77 44 0 23 Mar 2022
Dataset Distillation by Matching Training Trajectories George Cazenavette Tongzhou Wang Antonio Torralba Alexei A. Efros Jun-Yan Zhu FedML DD 185 395 0 22 Mar 2022
Federated Class-Incremental Learning Jiahua Dong Lixu Wang Zhen Fang Gan Sun Shichao Xu Tianlin Li Qi Zhu CLL FedML 111 183 0 22 Mar 2022
Acceleration of Federated Learning with Alleviated Forgetting in Local Training Chencheng Xu Zhiwei Hong Minlie Huang Tao Jiang FedML 84 45 0 05 Mar 2022
PUMA: Performance Unchanged Model Augmentation for Training Data Removal Ga Wu Masoud Hashemi C. Srinivasa MU 47 72 0 02 Mar 2022
Quantifying Memorization Across Neural Language Models Nicholas Carlini Daphne Ippolito Matthew Jagielski Katherine Lee Florian Tramèr Chiyuan Zhang PILM 124 630 0 15 Feb 2022
Memory Replay with Data Compression for Continual Learning Liyuan Wang Xingxing Zhang Kuo Yang L. Yu Chongxuan Li Lanqing Hong Shifeng Zhang Zhenguo Li Yi Zhong Jun Zhu 88 103 0 14 Feb 2022
Fortuitous Forgetting in Connectionist Networks Hattie Zhou Ankit Vani Hugo Larochelle Aaron Courville CLL 66 45 0 01 Feb 2022
Backdoor Defense with Machine Unlearning Yang Liu Mingyuan Fan Cen Chen Ximeng Liu Zhuo Ma Li Wang Jianfeng Ma AAML 80 78 0 24 Jan 2022
Learning to Prompt for Continual Learning Zifeng Wang Zizhao Zhang Chen-Yu Lee Han Zhang Ruoxi Sun Xiaoqi Ren Guolong Su Vincent Perot Jennifer Dy Tomas Pfister CLL VPVLM KELM VLM 105 796 0 16 Dec 2021
Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning Zixuan Ke Bing-Quan Liu Nianzu Ma Hu Xu Lei Shu CLL 217 126 0 05 Dec 2021
Learning Curves for Continual Learning in Neural Networks: Self-Knowledge Transfer and Forgetting Ryo Karakida S. Akaho CLL 72 13 0 03 Dec 2021
Subspace Regularizers for Few-Shot Class Incremental Learning Afra Feyza Akyürek Ekin Akyürek Derry Wijaya Jacob Andreas CLL 86 61 0 13 Oct 2021
Representational Continuity for Unsupervised Continual Learning Divyam Madaan Jaehong Yoon Yuanchun Li Yunxin Liu Sung Ju Hwang CLL SSL 135 117 0 13 Oct 2021
Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learning Danruo Deng Guangyong Chen Jianye Hao Qiong Wang Pheng-Ann Heng CLL AAML 56 82 0 09 Oct 2021
Meta Learning on a Sequence of Imbalanced Domains with Difficulty Awareness Zhenyi Wang Tiehang Duan Le Fang Qiuling Suo Mingchen Gao 238 18 0 29 Sep 2021
Skin Deep Unlearning: Artefact and Instrument Debiasing in the Context of Melanoma Classification P. Bevan Amir Atapour-Abarghouei MU 96 18 0 20 Sep 2021
Robust fine-tuning of zero-shot models Mitchell Wortsman Gabriel Ilharco Jong Wook Kim Mike Li Simon Kornblith ... Raphael Gontijo-Lopes Hannaneh Hajishirzi Ali Farhadi Hongseok Namkoong Ludwig Schmidt VLM 158 739 0 04 Sep 2021
Preventing Catastrophic Forgetting and Distribution Mismatch in Knowledge Distillation via Synthetic Data Kuluhan Binici N. Pham T. Mitra K. Leman 81 41 0 11 Aug 2021
Generalized Source-free Domain Adaptation Shiqi Yang Yaxing Wang Joost van de Weijer Luis Herranz Shangling Jui TTA 92 259 0 03 Aug 2021
Co $^2$ L: Contrastive Continual Learning Hyuntak Cha Jaeho Lee Jinwoo Shin CLL SSL VLM 116 306 0 28 Jun 2021
Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning James Smith Yen-Chang Hsu John C. Balloch Yilin Shen Hongxia Jin Z. Kira CLL 106 168 0 17 Jun 2021
Natural continual learning: success is a journey, not (just) a destination Ta-Chu Kao Kristopher T. Jensen Gido M. van de Ven A. Bernacchia Guillaume Hennequin CLL 83 49 0 15 Jun 2021
Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learning Liangqiong Qu Yuyin Zhou Paul Pu Liang Yingda Xia Feifei Wang Ehsan Adeli L. Fei-Fei D. Rubin FedML AI4CE 83 185 0 10 Jun 2021