How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advances

11 October 2023

Zihan Zhang

Meng Fang

Lingxi Chen

Mohammad-Reza Namazi-Rad

Jun Wang

KELM

ArXiv (abs)PDF HTML Github (134★)

Papers citing "How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advances"

50 / 151 papers shown

Title
ChroKnowledge: Unveiling Chronological Knowledge of Language Models in Multiple Domains Yein Park Chanwoong Yoon Jungwoo Park Donghyeon Lee Minbyul Jeong Jaewoo Kang KELM 139 2 0 13 Oct 2024
Towards LifeSpan Cognitive Systems Yu Wang Chi Han Tongtong Wu Xiaoxin He Wangchunshu Zhou ... Zexue He Wei Wang Gholamreza Haffari Heng Ji Julian McAuley KELM CLL 461 2 0 20 Sep 2024
Adversarial Attacks on Data Attribution Xinhe Wang Pingbang Hu Junwei Deng Jiaqi W. Ma TDI 136 0 0 09 Sep 2024
KLoB: a Benchmark for Assessing Knowledge Locating Methods in Language Models Yiming Ju Zheng Zhang KELM 56 9 0 28 Sep 2023
Knowledge Solver: Teaching LLMs to Search for Domain Knowledge from Knowledge Graphs Chao Feng Xinyu Zhang Zichu Fei KELM 61 49 0 06 Sep 2023
Journey to the Center of the Knowledge Neurons: Discoveries of Language-Independent Knowledge Neurons and Degenerate Knowledge Neurons Yuheng Chen Pengfei Cao Yubo Chen Kang Liu Jun Zhao KELM 83 49 0 25 Aug 2023
Eva-KELLM: A New Benchmark for Evaluating Knowledge Editing of LLMs Suhang Wu Minlong Peng Yue Chen Jinsong Su Mingming Sun KELM 74 39 0 19 Aug 2023
PMET: Precise Model Editing in a Transformer Xiaopeng Li Shasha Li Shangwen Wang Jing Yang Jun Ma Jie Yu KELM 110 135 0 17 Aug 2023
Continual Pre-Training of Large Language Models: How to (re)warm your model? Kshitij Gupta Benjamin Thérien Adam Ibrahim Mats L. Richter Quentin G. Anthony Eugene Belilovsky Irina Rish Timothée Lesort KELM 111 107 0 08 Aug 2023
Evaluating the Ripple Effects of Knowledge Editing in Language Models Roi Cohen Eden Biran Ori Yoran Amir Globerson Mor Geva KELM 97 180 0 24 Jul 2023
A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning Zhenyi Wang Enneng Yang Li Shen Heng-Chiao Huang KELM MU 86 51 0 16 Jul 2023
Propagating Knowledge Updates to LMs Through Distillation Shankar Padmanabhan Yasumasa Onoe Michael J.Q. Zhang Greg Durrett Eunsol Choi KELM 67 20 0 15 Jun 2023
Unifying Large Language Models and Knowledge Graphs: A Roadmap Shirui Pan Linhao Luo Yufei Wang Chen Chen Jiapu Wang Xindong Wu KELM 133 784 0 14 Jun 2023
ModuleFormer: Modularity Emerges from Mixture-of-Experts Songlin Yang Zheyu Zhang Tianyou Cao Shawn Tan Zhenfang Chen Chuang Gan KELM MoE 52 10 0 07 Jun 2023
Information Association for Language Model Updating by Mitigating LM-Logical Discrepancy Pengfei Yu Heng Ji KELM 66 10 0 29 May 2023
Do Large Language Models Know What They Don't Know? Zhangyue Yin Qiushi Sun Qipeng Guo Jiawen Wu Xipeng Qiu Xuanjing Huang ELM AI4MH 78 163 0 29 May 2023
Detecting Edit Failures In Large Language Models: An Improved Specificity Benchmark J. Hoelscher-Obermaier Julia Persson Esben Kran Ioannis Konstas Fazl Barez KELM 75 62 0 27 May 2023
Augmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In Zichun Yu Chenyan Xiong S. Yu Zhiyuan Liu KELM VLM 93 69 0 27 May 2023
MultiTool-CoT: GPT-3 Can Use Multiple External Tools with Chain of Thought Prompting Tatsuro Inaba Hirokazu Kiyomaru Fei Cheng Sadao Kurohashi KELM LRM 98 23 0 26 May 2023
Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy Zhihong Shao Yeyun Gong Yelong Shen Minlie Huang Nan Duan Weizhu Chen RALM LRM KELM 104 260 0 24 May 2023
Meta-Learning Online Adaptation of Language Models Nathan J. Hu E. Mitchell Christopher D. Manning Chelsea Finn KELM 84 37 0 24 May 2023
Mitigating Temporal Misalignment by Discarding Outdated Facts Michael J.Q. Zhang Eunsol Choi KELM HILM 89 20 0 24 May 2023
MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions Zexuan Zhong Zhengxuan Wu Christopher D. Manning Christopher Potts Danqi Chen KELM 86 217 0 24 May 2023
WikiChat: Stopping the Hallucination of Large Language Model Chatbots by Few-Shot Grounding on Wikipedia Sina J. Semnani Violet Z. Yao He Zhang M. Lam KELM AI4MH 109 80 0 23 May 2023
Question Answering as Programming for Solving Time-Sensitive Questions Xinyu Zhu Cheng Yang B. Chen Siheng Li Jian-Guang Lou Yujiu Yang KELM 80 11 0 23 May 2023
Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Tasks Tiedong Liu K. H. Low ALM 74 86 0 23 May 2023
Improving Language Models via Plug-and-Play Retrieval Feedback Wenhao Yu Zhihan Zhang Zhenwen Liang Meng Jiang Ashish Sabharwal KELM LRM 101 78 0 23 May 2023
Interactive Natural Language Processing Zekun Wang Ge Zhang Kexin Yang Ning Shi Wangchunshu Zhou ... Wenhu Chen Ke Xu Dayiheng Liu Yi-Ting Guo Jie Fu KELM 49 37 0 22 May 2023
Editing Large Language Models: Problems, Methods, and Opportunities Yunzhi Yao Peng Wang Bo Tian Shuyang Cheng Zhoubo Li Shumin Deng Huajun Chen Ningyu Zhang KELM 113 311 0 22 May 2023
Can We Edit Factual Knowledge by In-Context Learning? Ce Zheng Lei Li Qingxiu Dong Yuxuan Fan Zhiyong Wu Jingjing Xu Baobao Chang KELM 79 216 0 22 May 2023
Lifelong Language Pretraining with Distribution-Specialized Experts Wuyang Chen Yan-Quan Zhou Nan Du Yanping Huang James Laudon Zhiwen Chen Claire Cu KELM 95 50 0 20 May 2023
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing Zhibin Gou Zhihong Shao Yeyun Gong Yelong Shen Yujiu Yang Nan Duan Weizhu Chen KELM LRM 91 396 0 19 May 2023
Decouple knowledge from parameters for plug-and-play language modeling Xin Cheng Yankai Lin Preslav Nakov Dongyan Zhao Rui Yan KELM 60 2 0 19 May 2023
PaLM 2 Technical Report Rohan Anil Andrew M. Dai Orhan Firat Melvin Johnson Dmitry Lepikhin ... Ce Zheng Wei Zhou Denny Zhou Slav Petrov Yonghui Wu ReLM LRM 241 1,205 0 17 May 2023
Large Language Models are Built-in Autoregressive Search Engines Noah Ziems Wenhao Yu Zhihan Zhang Meng Jiang KELM 133 42 0 16 May 2023
Evaluating Open-Domain Question Answering in the Era of Large Language Models Ehsan Kamalloo Nouha Dziri C. Clarke Davood Rafiei ELM 72 110 0 11 May 2023
Active Retrieval Augmented Generation Zhengbao Jiang Frank F. Xu Luyu Gao Zhiqing Sun Qian Liu Jane Dwivedi-Yu Yiming Yang Jamie Callan Graham Neubig RALM 88 290 0 11 May 2023
WebCPM: Interactive Web Search for Chinese Long-form Question Answering Yujia Qin Zihan Cai Di Jin Lan Yan Shi Liang ... Ruobing Xie Fanchao Qi Zhiyuan Liu Maosong Sun Jie Zhou RALM 61 93 0 11 May 2023
RECKONING: Reasoning through Dynamic Knowledge Encoding Zeming Chen Gail Weiss E. Mitchell Asli Celikyilmaz Antoine Bosselut KELM LRM 87 13 0 10 May 2023
Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Framework Ruochen Zhao Xingxuan Li Shafiq Joty Chengwei Qin Lidong Bing LRM KELM 78 169 0 05 May 2023
Can LMs Learn New Entities from Descriptions? Challenges in Propagating Injected Knowledge Yasumasa Onoe Michael J.Q. Zhang Shankar Padmanabhan Greg Durrett Eunsol Choi KELM 271 78 0 02 May 2023
Dissecting Recall of Factual Associations in Auto-Regressive Language Models Mor Geva Jasmijn Bastings Katja Filippova Amir Globerson KELM 274 324 0 28 Apr 2023
Evaluating Verifiability in Generative Search Engines Nelson F. Liu Tianyi Zhang Percy Liang HILM 81 249 0 19 Apr 2023
Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models Pan Lu Baolin Peng Hao Cheng Michel Galley Kai-Wei Chang Ying Nian Wu Song-Chun Zhu Jianfeng Gao KELM MLLM LRM 120 324 0 19 Apr 2023
Interpretable Unified Language Checking Tianhua Zhang Hongyin Luo Yung-Sung Chuang Wei Fang Luc Gaitskell Thomas Hartvigsen Xixin Wu D. Fox Helen M. Meng James R. Glass 65 22 0 07 Apr 2023
Inspecting and Editing Knowledge Representations in Language Models Evan Hernandez Belinda Z. Li Jacob Andreas KELM 89 91 0 03 Apr 2023
TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIs Yaobo Liang Chenfei Wu Ting Song Wenshan Wu Yan Xia ... Shaoguang Mao Yuntao Wang Linjun Shou Ming Gong Nan Duan LLMAG CLL 79 205 0 29 Mar 2023
MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action Zhengyuan Yang Linjie Li Jianfeng Wang Kevin Qinghong Lin E. Azarnasab Faisal Ahmed Zicheng Liu Ce Liu Michael Zeng Lijuan Wang ReLM KELM LRM 111 394 0 20 Mar 2023
Context-faithful Prompting for Large Language Models Wenxuan Zhou Sheng Zhang Hoifung Poon Muhao Chen KELM 59 63 0 20 Mar 2023
The Life Cycle of Knowledge in Big Language Models: A Survey Boxi Cao Hongyu Lin Xianpei Han Le Sun KELM 84 28 0 14 Mar 2023