v1v2 (latest)

A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrix

7 October 2020

Papers citing "A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrix"

32 / 32 papers shown

Title
Theory on Mixture-of-Experts in Continual Learning Hongbo Li Sen-Fon Lin Lingjie Duan Yingbin Liang Ness B. Shroff MoE MoMe CLL 271 17 0 20 Feb 2025
CENSOR: Defense Against Gradient Inversion via Orthogonal Subspace Bayesian Sampling Kaiyuan Zhang Siyuan Cheng Guangyu Shen Bruno Ribeiro Shengwei An Pin-Yu Chen Xinming Zhang Ninghui Li 331 2 0 28 Jan 2025
Theoretical Insights into Overparameterized Models in Multi-Task and Replay-Based Continual Learning Mohammadamin Banayeeanzade Mahdi Soltanolkotabi Mohammad Rostami CLL LRM 291 4 0 29 Aug 2024
Data-dependent and Oracle Bounds on Forgetting in Continual Learning Lior Friedman Ron Meir 126 0 0 13 Jun 2024
SketchOGD: Memory-Efficient Continual Learning Benjamin Wright Youngjae Min Jeremy Bernstein Navid Azizan CLL 182 0 0 25 May 2023
Artificial Neural Variability for Deep Learning: On Overfitting, Noise Memorization, and Catastrophic Forgetting Zeke Xie Fengxiang He Shaopeng Fu Issei Sato Dacheng Tao Masashi Sugiyama 56 61 0 12 Nov 2020
Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics V. Ramasesh Ethan Dyer M. Raghu CLL 105 179 0 14 Jul 2020
Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descent Mehdi Abbana Bennani Thang Doan Masashi Sugiyama CLL 101 65 0 21 Jun 2020
Optimization and Generalization of Regularization-Based Continual Learning: a Loss Approximation Viewpoint Dong Yin Mehrdad Farajtabar Ang Li Nir Levine Alex Mott CLL 80 21 0 19 Jun 2020
Understanding the Role of Training Regimes in Continual Learning Seyed Iman Mirzadeh Mehrdad Farajtabar Razvan Pascanu H. Ghasemzadeh CLL 81 228 0 12 Jun 2020
Explaining How Deep Neural Networks Forget by Deep Visualization Giang Nguyen Shuan Chen Tae Joon Jun Daeyoung Kim AAML 53 10 0 03 May 2020
Continual Deep Learning by Functional Regularisation of Memorable Past Pingbo Pan S. Swaroop Alexander Immer Runa Eschenhagen Richard Turner Mohammad Emtiyaz Khan KELM CLL 58 143 0 29 Apr 2020
Gradient Surgery for Multi-Task Learning Tianhe Yu Saurabh Kumar Abhishek Gupta Sergey Levine Karol Hausman Chelsea Finn 191 1,230 0 19 Jan 2020
Orthogonal Gradient Descent for Continual Learning Mehrdad Farajtabar Navid Azizan Alex Mott Ang Li CLL 105 374 0 15 Oct 2019
A continual learning survey: Defying forgetting in classification tasks Matthias De Lange Rahaf Aljundi Marc Masana Sarah Parisot Xu Jia A. Leonardis Greg Slabaugh Tinne Tuytelaars CLL KELM 72 246 0 18 Sep 2019
Online Continual Learning with Maximally Interfered Retrieval Rahaf Aljundi Lucas Caccia Eugene Belilovsky Massimo Caccia Min Lin Laurent Charlin Tinne Tuytelaars CLL 85 549 0 11 Aug 2019
Toward Understanding Catastrophic Forgetting in Continual Learning Cuong V Nguyen Alessandro Achille Michael Lam Tal Hassner Vijay Mahadevan Stefano Soatto 70 94 0 02 Aug 2019
On Exact Computation with an Infinitely Wide Neural Net Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruslan Salakhutdinov Ruosong Wang 243 928 0 26 Apr 2019
Gradient based sample selection for online continual learning Rahaf Aljundi Min Lin Baptiste Goujaud Yoshua Bengio BDL CLL 118 838 0 20 Mar 2019
Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent Jaehoon Lee Lechao Xiao S. Schoenholz Yasaman Bahri Roman Novak Jascha Narain Sohl-Dickstein Jeffrey Pennington 213 1,110 0 18 Feb 2019
An Empirical Study of Example Forgetting during Deep Neural Network Learning Mariya Toneva Alessandro Sordoni Rémi Tachet des Combes Adam Trischler Yoshua Bengio Geoffrey J. Gordon 134 741 0 12 Dec 2018
Efficient Lifelong Learning with A-GEM Arslan Chaudhry MarcÁurelio Ranzato Marcus Rohrbach Mohamed Elhoseiny CLL 212 1,463 0 02 Dec 2018
Gradient Descent Provably Optimizes Over-parameterized Neural Networks S. Du Xiyu Zhai Barnabás Póczós Aarti Singh MLT ODL 243 1,276 0 04 Oct 2018
Neural Tangent Kernel: Convergence and Generalization in Neural Networks Arthur Jacot Franck Gabriel Clément Hongler 275 3,225 0 20 Jun 2018
Towards Robust Evaluations of Continual Learning Sebastian Farquhar Y. Gal CLL 93 308 0 24 May 2018
Continual Lifelong Learning with Neural Networks: A Review G. I. Parisi Ronald Kemker Jose L. Part Christopher Kanan S. Wermter KELM CLL 207 2,901 0 21 Feb 2018
PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning Arun Mallya Svetlana Lazebnik CLL 111 1,309 0 15 Nov 2017
Measuring Catastrophic Forgetting in Neural Networks Ronald Kemker Marc McClure Angelina Abitino Tyler L. Hayes Christopher Kanan CLL 127 722 0 07 Aug 2017
Gradient Episodic Memory for Continual Learning David Lopez-Paz MarcÁurelio Ranzato VLM CLL 133 2,743 0 26 Jun 2017
Incremental Learning Through Deep Adaptation Amir Rosenfeld John K. Tsotsos CLL 76 279 0 11 May 2017
Overcoming catastrophic forgetting in neural networks J. Kirkpatrick Razvan Pascanu Neil C. Rabinowitz J. Veness Guillaume Desjardins ... A. Grabska-Barwinska Demis Hassabis Claudia Clopath D. Kumaran R. Hadsell CLL 374 7,587 0 02 Dec 2016
An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks Ian Goodfellow M. Berk Mirza Xia Da Aaron Courville Yoshua Bengio 159 1,455 0 21 Dec 2013