Continual Lifelong Learning in Natural Language Processing: A Survey

17 December 2020

Papers citing "Continual Lifelong Learning in Natural Language Processing: A Survey"

50 / 96 papers shown

Title
High-Dimensional Interlingual Representations of Large Language Models Bryan Wilie Samuel Cahyawijaya Junxian He Pascale Fung 74 0 0 14 Mar 2025
Knowledge Entropy Decay during Language Model Pretraining Hinders New Knowledge Acquisition Jiyeon Kim Hyunji Lee Hyowon Cho Joel Jang Hyeonbin Hwang Seungpil Won Youbin Ahn Dohaeng Lee Minjoon Seo KELM 288 4 0 02 Oct 2024
Bayesian Learning-driven Prototypical Contrastive Loss for Class-Incremental Learning Nisha Lakshmana Raichur Lucas Heublein Tobias Feigl A. Rügamer Christopher Mutschler Felix Ott CLL BDL 86 8 0 17 May 2024
Don't Half-listen: Capturing Key-part Information in Continual Instruction Tuning Yongquan He Wenyuan Zhang Xuancheng Huang Peng Zhang Lingxun Meng Wei Lin Wenyuan Zhang Yifu Gao CLL ALM 81 5 0 15 Mar 2024
Investigating Continual Pretraining in Large Language Models: Insights and Implications cCaugatay Yildiz Nishaanth Kanna Ravichandran Prishruit Punia Matthias Bethge Beyza Ermis CLL KELM LRM 71 26 0 27 Feb 2024
Credit Risk Meets Large Language Models: Building a Risk Indicator from Loan Descriptions in P2P Lending Mario Sanz-Guerrero Javier Arroyo 45 5 0 29 Jan 2024
Unsupervised Pretraining for Neural Machine Translation Using Elastic Weight Consolidation Dusan Varis Ondrej Bojar 43 16 0 19 Oct 2020
CausaLM: Causal Model Explanation Through Counterfactual Language Models Amir Feder Nadav Oved Uri Shalit Roi Reichart CML LRM 56 158 0 27 May 2020
English Intermediate-Task Training Improves Zero-Shot Cross-Lingual Transfer Too Jason Phang Iacer Calixto Phu Mon Htut Yada Pruksachatkun Haokun Liu Clara Vania Katharina Kann Samuel R. Bowman LRM 44 66 0 26 May 2020
XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning Edoardo Ponti Goran Glavaš Olga Majewska Qianchu Liu Ivan Vulić Anna Korhonen LRM 51 315 0 01 May 2020
Don't Stop Pretraining: Adapt Language Models to Domains and Tasks Suchin Gururangan Ana Marasović Swabha Swayamdipta Kyle Lo Iz Beltagy Doug Downey Noah A. Smith VLM AI4CE CLL 90 2,380 0 23 Apr 2020
Multilingual Machine Translation: Closing the Gap between Shared and Language-specific Encoder-Decoders Carlos Escolano Marta R. Costa-jussá José A. R. Fonollosa Mikel Artetxe 27 54 0 14 Apr 2020
Meta-Learning in Neural Networks: A Survey Timothy M. Hospedales Antreas Antoniou P. Micaelli Amos Storkey OOD 266 1,950 0 11 Apr 2020
XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization Junjie Hu Sebastian Ruder Aditya Siddhant Graham Neubig Orhan Firat Melvin Johnson ELM 128 966 0 24 Mar 2020
Learning to Continually Learn Shawn L. E. Beaulieu Lapo Frati Thomas Miconi Joel Lehman Kenneth O. Stanley Jeff Clune Nick Cheney KELM CLL 63 147 0 21 Feb 2020
BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning Yeming Wen Dustin Tran Jimmy Ba OOD FedML UQCV 75 487 0 17 Feb 2020
Latent Replay for Real-Time Continual Learning Lorenzo Pellegrini G. Graffieti Vincenzo Lomonaco Davide Maltoni CLL 30 163 0 02 Dec 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 204 19,824 0 23 Oct 2019
Simple, Scalable Adaptation for Neural Machine Translation Ankur Bapna N. Arivazhagan Orhan Firat AI4CE 76 413 0 18 Sep 2019
LAMOL: LAnguage MOdeling for Lifelong Language Learning Fan-Keng Sun Cheng-Hao Ho Hung-yi Lee CLL KELM 34 208 0 07 Sep 2019
Abductive Commonsense Reasoning Chandra Bhagavatula Ronan Le Bras Chaitanya Malaviya Keisuke Sakaguchi Ari Holtzman Hannah Rashkin Doug Downey Scott Yih Yejin Choi ReLM LRM 39 456 0 15 Aug 2019
Lifelong and Interactive Learning of Factual Knowledge in Dialogues Sahisnu Mazumder Bing-Quan Liu Shuai Wang Nianzu Ma KELM RALM 19 24 0 31 Jul 2019
Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges N. Arivazhagan Ankur Bapna Orhan Firat Dmitry Lepikhin Melvin Johnson ... George F. Foster Colin Cherry Wolfgang Macherey Zhiwen Chen Yonghui Wu 45 424 0 11 Jul 2019
Psycholinguistics meets Continual Learning: Measuring Catastrophic Forgetting in Visual Question Answering Claudio Greco Barbara Plank Raquel Fernández Raffaella Bernardi CLL KELM 25 49 0 10 Jun 2019
Practical Deep Learning with Bayesian Principles Kazuki Osawa S. Swaroop Anirudh Jain Runa Eschenhagen Richard Turner Rio Yokota Mohammad Emtiyaz Khan BDL UQCV 70 243 0 06 Jun 2019
Uncertainty-guided Continual Learning with Bayesian Neural Networks Sayna Ebrahimi Mohamed Elhoseiny Trevor Darrell Marcus Rohrbach CLL BDL 28 196 0 06 Jun 2019
Episodic Memory in Lifelong Language Learning Cyprien de Masson dÁutume Sebastian Ruder Lingpeng Kong Dani Yogatama CLL KELM 95 285 0 03 Jun 2019
A comprehensive, application-oriented study of catastrophic forgetting in DNNs Benedikt Pfülb A. Gepperth 34 90 0 20 May 2019
Curriculum Learning for Domain Adaptation in Neural Machine Translation Xuan Zhang Pamela Shapiro Manish Kumar Paul McNamee Marine Carpuat Kevin Duh 34 124 0 14 May 2019
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems Alex Jinpeng Wang Yada Pruksachatkun Nikita Nangia Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 148 2,287 0 02 May 2019
Continual Learning for Sentence Representations Using Conceptors Tianlin Liu Lyle Ungar João Sedoc CLL 13 52 0 18 Apr 2019
Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting Xilai Li Yingbo Zhou Tianfu Wu R. Socher Caiming Xiong CLL 46 422 0 31 Mar 2019
Competence-based Curriculum Learning for Neural Machine Translation Emmanouil Antonios Platanios Otilia Stretcu Graham Neubig Barnabás Póczós Tom Michael Mitchell 45 340 0 23 Mar 2019
Continual Learning via Neural Pruning Siavash Golkar Michael Kagan Kyunghyun Cho CLL 30 161 0 11 Mar 2019
Sentence Embedding Alignment for Lifelong Relation Extraction Hong Wang Wenhan Xiong Mo Yu Xiaoxiao Guo Shiyu Chang William Yang Wang CLL 36 133 0 06 Mar 2019
On Tiny Episodic Memories in Continual Learning Arslan Chaudhry Marcus Rohrbach Mohamed Elhoseiny Thalaiyasingam Ajanthan P. Dokania Philip Torr MarcÁurelio Ranzato CLL 61 395 0 27 Feb 2019
Multilingual Neural Machine Translation with Knowledge Distillation Xu Tan Yi Ren Di He Tao Qin Zhou Zhao Tie-Yan Liu 56 248 0 27 Feb 2019
Parameter-Efficient Transfer Learning for NLP N. Houlsby A. Giurgiu Stanislaw Jastrzebski Bruna Morrone Quentin de Laroussilhe Andrea Gesmundo Mona Attariyan Sylvain Gelly 154 4,368 0 02 Feb 2019
Learning and Evaluating General Linguistic Intelligence Dani Yogatama Cyprien de Masson dÁutume Jerome T. Connor Tomás Kociský Mike Chrzanowski ... Angeliki Lazaridou Wang Ling Lei Yu Chris Dyer Phil Blunsom ELM AI4CE 112 210 0 31 Jan 2019
Efficient Lifelong Learning with A-GEM Arslan Chaudhry MarcÁurelio Ranzato Marcus Rohrbach Mohamed Elhoseiny CLL 149 1,438 0 02 Dec 2018
An Empirical Exploration of Curriculum Learning for Neural Machine Translation Xuan Zhang Manish Kumar Huda Khayrallah Kenton W. Murray Jeremy Gwinnup Marianna J. Martindale Paul McNamee Kevin Duh Marine Carpuat ODL 49 110 0 02 Nov 2018
Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference Matthew D Riemer Ignacio Cases R. Ajemian Miao Liu Irina Rish Y. Tu Gerald Tesauro CLL 55 776 0 29 Oct 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 782 93,936 0 11 Oct 2018
Open-world Learning and Application to Product Classification Hu Xu Bing-Quan Liu Lei Shu P. Yu CLL VLM 41 110 0 17 Sep 2018
Active Learning for Interactive Neural Machine Translation of Data Streams Álvaro Peris F. Casacuberta AI4CE 43 60 0 30 Jul 2018
The Natural Language Decathlon: Multitask Learning as Question Answering Bryan McCann N. Keskar Caiming Xiong R. Socher AIMat MLLM BDL 45 641 0 20 Jun 2018
Selfless Sequential Learning Rahaf Aljundi Marcus Rohrbach Tinne Tuytelaars CLL 48 114 0 14 Jun 2018
Projecting Embeddings for Domain Adaptation: Joint Modeling of Sentiment Analysis in Diverse Domains Jeremy Barnes Roman Klinger Sabine Schulte im Walde 32 28 0 12 Jun 2018
Diachronic word embeddings and semantic shifts: a survey Andrey Kutuzov Lilja Øvrelid Terrence Szymanski Erik Velldal AI4TS 30 301 0 09 Jun 2018
Reinforced Continual Learning Ju Xu Zhanxing Zhu CLL 76 377 0 31 May 2018