Online Continual Knowledge Learning for Language Models

16 November 2023

Papers citing "Online Continual Knowledge Learning for Language Models"

21 / 21 papers shown

Title
Knowledge Entropy Decay during Language Model Pretraining Hinders New Knowledge Acquisition Jiyeon Kim Hyunji Lee Hyowon Cho Joel Jang Hyeonbin Hwang Seungpil Won Youbin Ahn Dohaeng Lee Minjoon Seo KELM 330 5 0 02 Oct 2024
Time Waits for No One! Analysis and Challenges of Temporal Misalignment Kelvin Luu Daniel Khashabi Suchin Gururangan Karishma Mandyam Noah A. Smith 69 88 0 14 Nov 2021
Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora Xisen Jin Dejiao Zhang Henghui Zhu Wei Xiao Shang-Wen Li Xiaokai Wei Andrew O. Arnold Xiang Ren KELM CLL 83 116 0 16 Oct 2021
Entity-Based Knowledge Conflicts in Question Answering Shayne Longpre Kartik Perisetla Anthony Chen Nikhil Ramesh Chris DuBois Sameer Singh HILM 318 257 0 10 Sep 2021
Time-Aware Language Models as Temporal Knowledge Bases Bhuwan Dhingra Jeremy R. Cole Julian Martin Eisenschlos D. Gillick Jacob Eisenstein William W. Cohen KELM 90 278 0 29 Jun 2021
Online Coreset Selection for Rehearsal-based Continual Learning Jaehong Yoon Divyam Madaan Eunho Yang Sung Ju Hwang CLL 62 141 0 02 Jun 2021
Carbon Emissions and Large Neural Network Training David A. Patterson Joseph E. Gonzalez Quoc V. Le Chen Liang Lluís-Miquel Munguía D. Rothchild David R. So Maud Texier J. Dean AI4CE 331 665 0 21 Apr 2021
Editing Factual Knowledge in Language Models Nicola De Cao Wilker Aziz Ivan Titov KELM 104 503 0 16 Apr 2021
New Insights on Reducing Abrupt Representation Change in Online Continual Learning Lucas Caccia Rahaf Aljundi Nader Asadi Tinne Tuytelaars Joelle Pineau Eugene Belilovsky CLL 46 200 0 11 Apr 2021
Experience Replay with Likelihood-free Importance Weights Samarth Sinha Jiaming Song Animesh Garg Stefano Ermon OffRL 54 56 0 23 Jun 2020
Coresets via Bilevel Optimization for Continual Learning and Streaming Zalan Borsos Mojmír Mutný Andreas Krause CLL 73 232 0 06 Jun 2020
Don't Stop Pretraining: Adapt Language Models to Domains and Tasks Suchin Gururangan Ana Marasović Swabha Swayamdipta Kyle Lo Iz Beltagy Doug Downey Noah A. Smith VLM AI4CE CLL 134 2,414 0 23 Apr 2020
REALM: Retrieval-Augmented Language Model Pre-Training Kelvin Guu Kenton Lee Zora Tung Panupong Pasupat Ming-Wei Chang RALM 103 2,090 0 10 Feb 2020
K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters Ruize Wang Duyu Tang Nan Duan Zhongyu Wei Xuanjing Huang Jianshu Ji Guihong Cao Daxin Jiang Ming Zhou KELM 87 553 0 05 Feb 2020
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 367 20,053 0 23 Oct 2019
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 558 2,660 0 03 Sep 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 514 24,351 0 26 Jul 2019
Learning Loss for Active Learning Donggeun Yoo In So Kweon UQCV 77 660 0 09 May 2019
Overcoming catastrophic forgetting in neural networks J. Kirkpatrick Razvan Pascanu Neil C. Rabinowitz J. Veness Guillaume Desjardins ... A. Grabska-Barwinska Demis Hassabis Claudia Clopath D. Kumaran R. Hadsell CLL 313 7,478 0 02 Dec 2016
iCaRL: Incremental Classifier and Representation Learning Sylvestre-Alvise Rebuffi Alexander Kolesnikov G. Sperl Christoph H. Lampert CLL OOD 129 3,741 0 23 Nov 2016
Learning without Forgetting Zhizhong Li Derek Hoiem CLL OOD SSL 282 4,391 0 29 Jun 2016