Towards Continual Knowledge Learning of Language Models

7 October 2021

Papers citing "Towards Continual Knowledge Learning of Language Models"

38 / 38 papers shown

Title
OBLIVIATE: Robust and Practical Machine Unlearning for Large Language Models Xiaoyu Xu Minxin Du Qingqing Ye Haibo Hu MU 57 0 0 07 May 2025
TiC-LM: A Web-Scale Benchmark for Time-Continual LLM Pretraining Jeffrey Li Mohammadreza Armandpour Iman Mirzadeh Sachin Mehta Vaishaal Shankar ... Samy Bengio Oncel Tuzel Mehrdad Farajtabar Hadi Pouransari Fartash Faghri CLL KELM 61 0 0 02 Apr 2025
Privacy Ripple Effects from Adding or Removing Personal Information in Language Model Training Jaydeep Borkar Matthew Jagielski Katherine Lee Niloofar Mireshghallah David A. Smith Christopher A. Choquette-Choo PILM 83 1 0 24 Feb 2025
Gradient Localization Improves Lifelong Pretraining of Language Models Jared Fernandez Yonatan Bisk Emma Strubell KELM 36 1 0 07 Nov 2024
Latent Paraphrasing: Perturbation on Layers Improves Knowledge Injection in Language Models Minki Kang Sung Ju Hwang Gibbeum Lee Jaewoong Cho KELM 40 0 0 01 Nov 2024
ChroKnowledge: Unveiling Chronological Knowledge of Language Models in Multiple Domains Yein Park Chanwoong Yoon Jungwoo Park Donghyeon Lee Minbyul Jeong Jaewoo Kang KELM 58 1 0 13 Oct 2024
Knowledge Entropy Decay during Language Model Pretraining Hinders New Knowledge Acquisition Jiyeon Kim Hyunji Lee Hyowon Cho Joel Jang Hyeonbin Hwang Seungpil Won Youbin Ahn Dohaeng Lee Minjoon Seo KELM 99 3 0 02 Oct 2024
Enhancing elusive clues in knowledge learning by contrasting attention of language models Jian Gao Xiao Zhang Ji Wu Miao Li 38 0 0 26 Sep 2024
Towards LifeSpan Cognitive Systems Yu Wang Chi Han Tongtong Wu Xiaoxin He Wangchunshu Zhou ... Zexue He Wei Wang Gholamreza Haffari Heng Ji Julian McAuley KELM CLL 141 1 0 20 Sep 2024
Knowledge in Superposition: Unveiling the Failures of Lifelong Knowledge Editing for Large Language Models Chenhui Hu Pengfei Cao Yubo Chen Kang Liu Jun Zhao KELM 61 2 0 14 Aug 2024
Train-Attention: Meta-Learning Where to Focus in Continual Knowledge Learning Yeongbin Seo Dongha Lee Jinyoung Yeo CLL KELM 90 1 0 24 Jul 2024
Language Modeling with Editable External Knowledge Belinda Z. Li Emmy Liu Alexis Ross Abbas Zeitoun Graham Neubig Jacob Andreas KELM 32 4 0 17 Jun 2024
Self-Tuning: Instructing LLMs to Effectively Acquire New Knowledge through Self-Teaching Xiaoying Zhang Baolin Peng Ye Tian Jingyan Zhou Yipeng Zhang Haitao Mi Helen Meng CLL KELM 75 5 0 10 Jun 2024
The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models Seungone Kim Juyoung Suk Ji Yong Cho Shayne Longpre Chaeeun Kim ... Sean Welleck Graham Neubig Moontae Lee Kyungjae Lee Minjoon Seo ELM ALM LM&MA 97 31 0 09 Jun 2024
From Matching to Generation: A Survey on Generative Information Retrieval Xiaoxi Li Jiajie Jin Yujia Zhou Yuyao Zhang Peitian Zhang Yutao Zhu Zhicheng Dou 3DV 78 46 0 23 Apr 2024
Simple and Scalable Strategies to Continually Pre-train Large Language Models Adam Ibrahim Benjamin Thérien Kshitij Gupta Mats L. Richter Quentin Anthony Timothée Lesort Eugene Belilovsky Irina Rish KELM CLL 44 52 0 13 Mar 2024
Where is the answer? Investigating Positional Bias in Language Model Knowledge Extraction Kuniaki Saito Kihyuk Sohn Chen-Yu Lee Yoshitaka Ushiku 62 2 0 16 Feb 2024
Online Continual Knowledge Learning for Language Models Yuhao Wu Tongjun Shi Karthick Sharma Chun Seah Shuhao Zhang CLL KELM 28 4 0 16 Nov 2023
Goodtriever: Adaptive Toxicity Mitigation with Retrieval-augmented Models Luiza Amador Pozzobon B. Ermiş Patrick Lewis Sara Hooker 30 20 0 11 Oct 2023
Continual Pre-Training of Large Language Models: How to (re)warm your model? Kshitij Gupta Benjamin Thérien Adam Ibrahim Mats L. Richter Quentin G. Anthony Eugene Belilovsky Irina Rish Timothée Lesort KELM 24 99 0 08 Aug 2023
Evaluating the Ripple Effects of Knowledge Editing in Language Models Roi Cohen Eden Biran Ori Yoran Amir Globerson Mor Geva KELM 42 155 0 24 Jul 2023
chatClimate: Grounding Conversational AI in Climate Science S. Vaghefi Qian Wang V. Muccione Jingwei Ni Mathias Kraus ... Tobias Schimanski Chiara Colesanti-Senni Nicolas Webersinke Christrian Huggel Markus Leippold KELM AI4MH HILM 23 67 0 11 Apr 2023
Dynamic Benchmarking of Masked Language Models on Temporal Concept Drift with Multiple Views Katerina Margatina Shuai Wang Yogarshi Vyas Neha Ann John Yassine Benajiba Miguel Ballesteros 17 15 0 23 Feb 2023
Large Language Models with Controllable Working Memory Daliang Li A. S. Rawat Manzil Zaheer Xin Wang Michal Lukasik Andreas Veit Felix X. Yu Surinder Kumar KELM 50 152 0 09 Nov 2022
Time-aware Prompting for Text Generation Shuyang Cao Lu Wang 21 11 0 03 Nov 2022
RealTime QA: What's the Answer Right Now? Jungo Kasai Keisuke Sakaguchi Yoichi Takahashi Ronan Le Bras Akari Asai Xinyan Velocity Yu Dragomir R. Radev Noah A. Smith Yejin Choi Kentaro Inui KELM 45 165 0 27 Jul 2022
Language Anisotropic Cross-Lingual Model Editing Yang Xu Yutai Hou Wanxiang Che Min Zhang KELM 94 24 0 25 May 2022
Entity Cloze By Date: What LMs Know About Unseen Entities Yasumasa Onoe Michael J.Q. Zhang Eunsol Choi Greg Durrett KELM 21 49 0 05 May 2022
On Continual Model Refinement in Out-of-Distribution Data Streams Bill Yuchen Lin Sida I. Wang Xi Lin Robin Jia Lin Xiao Xiang Ren Wen-tau Yih CLL 25 29 0 04 May 2022
TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models Joel Jang Seonghyeon Ye Changho Lee Sohee Yang Joongbo Shin Janghoon Han Gyeonghun Kim Minjoon Seo CLL KELM 27 91 0 29 Apr 2022
ELLE: Efficient Lifelong Pre-training for Emerging Data Yujia Qin Jiajie Zhang Yankai Lin Zhiyuan Liu Peng Li Maosong Sun Jie Zhou 24 67 0 12 Mar 2022
TimeLMs: Diachronic Language Models from Twitter Daniel Loureiro Francesco Barbieri Leonardo Neves Luis Espinosa Anke Jose Camacho-Collados 33 247 0 08 Feb 2022
Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora Xisen Jin Dejiao Zhang Henghui Zhu Wei Xiao Shang-Wen Li Xiaokai Wei Andrew O. Arnold Xiang Ren KELM CLL 25 110 0 16 Oct 2021
Entity-Based Knowledge Conflicts in Question Answering Shayne Longpre Kartik Perisetla Anthony Chen Nikhil Ramesh Chris DuBois Sameer Singh HILM 245 237 0 10 Sep 2021
Internet-Augmented Dialogue Generation M. Komeili Kurt Shuster Jason Weston RALM 238 280 0 15 Jul 2021
Deduplicating Training Data Makes Language Models Better Katherine Lee Daphne Ippolito A. Nystrom Chiyuan Zhang Douglas Eck Chris Callison-Burch Nicholas Carlini SyDa 242 592 0 14 Jul 2021
Carbon Emissions and Large Neural Network Training David A. Patterson Joseph E. Gonzalez Quoc V. Le Chen Liang Lluís-Miquel Munguía D. Rothchild David R. So Maud Texier J. Dean AI4CE 244 644 0 21 Apr 2021
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 415 2,586 0 03 Sep 2019