Reminding the Incremental Language Model via Data-Free Self-Distillation

17 October 2021

Papers citing "Reminding the Incremental Language Model via Data-Free Self-Distillation"

29 / 29 papers shown

Title
Continual Learning for Recurrent Neural Networks: an Empirical Evaluation Andrea Cossu Antonio Carta Vincenzo Lomonaco D. Bacciu CLL 61 108 0 12 Mar 2021
BERT-EMD: Many-to-Many Layer Mapping for BERT Compression with Earth Mover's Distance Jianquan Li Xiaokang Liu Honghong Zhao Ruifeng Xu Min Yang Yaohong Jin 98 54 0 13 Oct 2020
Efficient Meta Lifelong-Learning with Limited Memory Zirui Wang Sanket Vaibhav Mehta Barnabás Póczós J. Carbonell CLL KELM 67 76 0 06 Oct 2020
Lifelong Language Knowledge Distillation Yung-Sung Chuang Shang-Yu Su Yun-Nung Chen KELM CLL 66 51 0 05 Oct 2020
Crossmodal Language Grounding in an Embodied Neurocognitive Model Stefan Heinrich Yuan Yao Tobias Hinz Zhiyuan Liu Thomas Hummel Matthias Kerzel C. Weber S. Wermter LM&Ro 56 19 0 24 Jun 2020
LAMOL: LAnguage MOdeling for Lifelong Language Learning Fan-Keng Sun Cheng-Hao Ho Hung-yi Lee CLL KELM 87 208 0 07 Sep 2019
Lifelong GAN: Continual Learning for Conditional Image Generation Mengyao Zhai Lei Chen Frederick Tung Jiawei He Megha Nawhal Greg Mori CLL 79 183 0 23 Jul 2019
Episodic Memory in Lifelong Language Learning Cyprien de Masson dÁutume Sebastian Ruder Lingpeng Kong Dani Yogatama CLL KELM 128 287 0 03 Jun 2019
Unsupervised Label Noise Modeling and Loss Correction Eric Arazo Sanchez Diego Ortego Paul Albert Noel E. O'Connor Kevin McGuinness NoLa 78 612 0 25 Apr 2019
Efficient Lifelong Learning with A-GEM Arslan Chaudhry MarcÁurelio Ranzato Marcus Rohrbach Mohamed Elhoseiny CLL 208 1,456 0 02 Dec 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 94,891 0 11 Oct 2018
Generative replay with feedback connections as a general strategy for continual learning Gido M. van de Ven A. Tolias CLL KELM 72 226 0 27 Sep 2018
The Natural Language Decathlon: Multitask Learning as Question Answering Bryan McCann N. Keskar Caiming Xiong R. Socher AIMat MLLM BDL 142 645 0 20 Jun 2018
Progress & Compress: A scalable framework for continual learning Jonathan Richard Schwarz Jelena Luketina Wojciech M. Czarnecki A. Grabska-Barwinska Yee Whye Teh Razvan Pascanu R. Hadsell CLL 122 889 0 16 May 2018
Born Again Neural Networks Tommaso Furlanello Zachary Chase Lipton Michael Tschannen Laurent Itti Anima Anandkumar 68 1,031 0 12 May 2018
Continual Lifelong Learning with Neural Networks: A Review G. I. Parisi Ronald Kemker Jose L. Part Christopher Kanan S. Wermter KELM CLL 187 2,888 0 21 Feb 2018
FearNet: Brain-Inspired Model for Incremental Learning Ronald Kemker Christopher Kanan CLL 112 479 0 28 Nov 2017
Memory Aware Synapses: Learning what (not) to forget Rahaf Aljundi F. Babiloni Mohamed Elhoseiny Marcus Rohrbach Tinne Tuytelaars KELM CLL 85 1,636 0 27 Nov 2017
Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning Victor Zhong Caiming Xiong R. Socher RALM 95 1,196 0 31 Aug 2017
Gradient Episodic Memory for Continual Learning David Lopez-Paz MarcÁurelio Ranzato VLM CLL 127 2,714 0 26 Jun 2017
Continual Learning with Deep Generative Replay Hanul Shin Jung Kwon Lee Jaehong Kim Jiwon Kim KELM CLL 80 2,077 0 24 May 2017
Learning to Generate Reviews and Discovering Sentiment Alec Radford Rafal Jozefowicz Ilya Sutskever 97 509 0 05 Apr 2017
Overcoming Catastrophic Forgetting by Incremental Moment Matching Sang-Woo Lee Jin-Hwa Kim Jaehyun Jun Jung-Woo Ha Byoung-Tak Zhang CLL 70 677 0 24 Mar 2017
Overcoming catastrophic forgetting in neural networks J. Kirkpatrick Razvan Pascanu Neil C. Rabinowitz J. Veness Guillaume Desjardins ... A. Grabska-Barwinska Demis Hassabis Claudia Clopath D. Kumaran R. Hadsell CLL 365 7,518 0 02 Dec 2016
Learning without Forgetting Zhizhong Li Derek Hoiem CLL OOD SSL 296 4,408 0 29 Jun 2016
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 283 8,134 0 16 Jun 2016
A Network-based End-to-End Trainable Task-oriented Dialogue System Tsung-Hsien Wen David Vandyke N. Mrksic Milica Gasic L. Rojas-Barahona Pei-hao Su Stefan Ultes S. Young 71 1,108 0 15 Apr 2016
Character-level Convolutional Networks for Text Classification Xiang Zhang Jiaqi Zhao Yann LeCun 268 6,113 0 04 Sep 2015
Training Deep Neural Networks on Noisy Labels with Bootstrapping Scott E. Reed Honglak Lee Dragomir Anguelov Christian Szegedy D. Erhan Andrew Rabinovich NoLa 113 1,022 0 20 Dec 2014