v1v2 (latest)

Large Language Models Struggle to Learn Long-Tail Knowledge

15 November 2022

Papers citing "Large Language Models Struggle to Learn Long-Tail Knowledge"

50 / 260 papers shown

Title
CPopQA: Ranking Cultural Concept Popularity by LLMs Ming Jiang Mansi Joshi 84 7 0 14 Nov 2023
Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applications Zhangyin Feng Weitao Ma Weijiang Yu Lei Huang Haotian Wang Qianglong Chen Weihua Peng Xiaocheng Feng Bing Qin Ting Liu KELM 80 40 0 10 Nov 2023
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions Lei Huang Weijiang Yu Weitao Ma Weihong Zhong Zhangyin Feng ... Qianglong Chen Weihua Peng Xiaocheng Feng Bing Qin Ting Liu LRM HILM 142 939 0 09 Nov 2023
The Effect of Scaling, Retrieval Augmentation and Form on the Factual Consistency of Language Models Lovisa Hagström Denitsa Saynova Tobias Norlund Moa Johansson Richard Johansson KELM HILM 63 9 0 02 Nov 2023
Diffusion Models for Reinforcement Learning: A Survey Zhengbang Zhu Hanye Zhao Haoran He Yichao Zhong Shenyu Zhang Haoquan Guo Tingting Chen Weinan Zhang 154 68 0 02 Nov 2023
Blending Reward Functions via Few Expert Demonstrations for Faithful and Accurate Knowledge-Grounded Dialogue Generation Wanyu Du Yangfeng Ji 64 1 0 02 Nov 2023
The Mystery of In-Context Learning: A Comprehensive Survey on Interpretation and Analysis Yuxiang Zhou Jiazheng Li Yanzheng Xiang Hanqi Yan Lin Gui Yulan He 93 19 0 01 Nov 2023
Filter bubbles and affective polarization in user-personalized large language model outputs Tomo Lazovich 22 4 0 31 Oct 2023
Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Models Paul Youssef Osman Alperen Koracs Meijie Li Jorg Schlotterer Christin Seifert KELM 81 19 0 25 Oct 2023
Emptying the Ocean with a Spoon: Should We Edit Models? Yuval Pinter Michael Elhadad KELM 129 29 0 18 Oct 2023
The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Models Aviv Slobodkin Omer Goldman Avi Caciularu Ido Dagan Shauli Ravfogel HILM LRM 108 33 0 18 Oct 2023
Document-Level In-Context Few-Shot Relation Extraction via Pre-Trained Language Models Yilmazcan Ozyurt Stefan Feuerriegel Ce Zhang 123 1 0 17 Oct 2023
A Comprehensive Evaluation of Tool-Assisted Generation Strategies Alon Jacovi Avi Caciularu Jonathan Herzig Roee Aharoni Bernd Bohnet Mor Geva ELM 126 7 0 16 Oct 2023
User Inference Attacks on Large Language Models Nikhil Kandpal Krishna Pillutla Alina Oprea Peter Kairouz Christopher A. Choquette-Choo Zheng Xu SILM AAML 132 20 0 13 Oct 2023
Impact of Co-occurrence on Factual Knowledge of Large Language Models Cheongwoong Kang Jaesik Choi KELM 91 18 0 12 Oct 2023
Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity Cunxiang Wang Xiaoze Liu Yuanhao Yue Xiangru Tang Tianhang Zhang ... Linyi Yang Jindong Wang Xing Xie Zheng Zhang Yue Zhang HILM KELM 172 201 0 11 Oct 2023
Crossing the Threshold: Idiomatic Machine Translation through Retrieval Augmentation and Loss Weighting Emmy Liu Aditi Chaudhary Graham Neubig 84 9 0 10 Oct 2023
Large Language Models can Learn Rules Zhaocheng Zhu Yuan Xue Xinyun Chen Denny Zhou Jian Tang Dale Schuurmans Hanjun Dai LRM ReLM 106 70 0 10 Oct 2023
Can Language Models be Instructed to Protect Personal Information? Yang Chen Ethan Mendes Sauvik Das Wei Xu Alan Ritter PILM 77 37 0 03 Oct 2023
Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models Mert Yuksekgonul Varun Chandrasekaran Erik Jones Suriya Gunasekar Ranjita Naik Hamid Palangi Ece Kamar Besmira Nushi HILM 60 49 0 26 Sep 2023
Ragas: Automated Evaluation of Retrieval Augmented Generation ES Shahul Jithin James Luis Espinosa-Anke Steven Schockaert 145 205 0 26 Sep 2023
The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" Lukas Berglund Meg Tong Max Kaufmann Mikita Balesni Asa Cooper Stickland Tomasz Korbak Owain Evans LRM 179 279 0 21 Sep 2023
Cognitive Mirage: A Review of Hallucinations in Large Language Models Hongbin Ye Tong Liu Aijia Zhang Wei Hua Weiqiang Jia HILM 120 81 0 13 Sep 2023
Loquacity and Visible Emotion: ChatGPT as a Policy Advisor Claudia Biancotti C. Camassa 34 2 0 07 Sep 2023
Explainability for Large Language Models: A Survey Haiyan Zhao Hanjie Chen Fan Yang Ninghao Liu Huiqi Deng Hengyi Cai Shuaiqiang Wang D. Yin Jundong Li LRM 106 471 0 02 Sep 2023
Characterizing Learning Curves During Language Model Pre-Training: Learning, Forgetting, and Stability Tyler A. Chang Zhuowen Tu Benjamin Bergen 59 13 0 29 Aug 2023
Journey to the Center of the Knowledge Neurons: Discoveries of Language-Independent Knowledge Neurons and Degenerate Knowledge Neurons Yuheng Chen Pengfei Cao Yubo Chen Kang Liu Jun Zhao KELM 95 49 0 25 Aug 2023
Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs? Kai Sun Yongjun Xu Hanwen Zha Yue Liu Xinhsuai Dong AI4MH 137 148 0 20 Aug 2023
KnowledGPT: Enhancing Large Language Models with Retrieval and Storage Access on Knowledge Bases Xintao Wang Qian Yang Yongting Qiu Jiaqing Liang Qi He Zhouhong Gu Yanghua Xiao Wei Wang KELM RALM 88 45 0 17 Aug 2023
LLaMA-E: Empowering E-commerce Authoring with Object-Interleaved Instruction Following Kaize Shi Xueyao Sun Dingxian Wang Yinlin Fu Guandong Xu Qing Li 86 4 0 09 Aug 2023
The Bias Amplification Paradox in Text-to-Image Generation P. Seshadri Sameer Singh Yanai Elazar DiffM 99 47 0 01 Aug 2023
Lost in the Middle: How Language Models Use Long Contexts Nelson F. Liu Kevin Lin John Hewitt Ashwin Paranjape Michele Bevilacqua Fabio Petroni Percy Liang RALM 134 1,663 0 06 Jul 2023
Offline Reinforcement Learning with Imbalanced Datasets Li Jiang Sijie Cheng Jielin Qiu Haoran Xu Wai Kin Victor Chan Zhao Ding OffRL 123 3 0 06 Jul 2023
Bring Your Own Data! Self-Supervised Evaluation for Large Language Models Neel Jain Khalid Saifullah Yuxin Wen John Kirchenbauer Manli Shu Aniruddha Saha Micah Goldblum Jonas Geiping Tom Goldstein ALM ELM 102 23 0 23 Jun 2023
Give Us the Facts: Enhancing Large Language Models with Knowledge Graphs for Fact-aware Language Modeling Lin F. Yang Hongyang Chen Zhao Li Xiao Ding Xindong Wu KELM 112 93 0 20 Jun 2023
Large Language Models, scientific knowledge and factuality: A systematic analysis in antibiotic discovery Magdalena Wysocka Oskar Wysocki Maxime Delmas V. Mutel André Freitas LM&MA 70 5 0 28 May 2023
FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation Sewon Min Kalpesh Krishna Xinxi Lyu M. Lewis Wen-tau Yih Pang Wei Koh Mohit Iyyer Luke Zettlemoyer Hannaneh Hajishirzi HILM ALM 157 705 0 23 May 2023
Improving Language Models via Plug-and-Play Retrieval Feedback Wenhao Yu Zhihan Zhang Zhenwen Liang Meng Jiang Ashish Sabharwal KELM LRM 114 78 0 23 May 2023
"According to ...": Prompting Language Models Improves Quoting from Pre-Training Data Orion Weller Marc Marone Nathaniel Weir Dawn J Lawrie Daniel Khashabi Benjamin Van Durme HILM 131 47 0 22 May 2023
Quantifying Association Capabilities of Large Language Models and Its Implications on Privacy Leakage Hanyin Shao Jie Huang Shen Zheng Kevin Chen-Chuan Chang PILM 58 30 0 22 May 2023
Knowledge Card: Filling LLMs' Knowledge Gaps with Plug-in Specialized Language Models Shangbin Feng Weijia Shi Yuyang Bai Vidhisha Balachandran Tianxing He Yulia Tsvetkov KELM 133 37 0 17 May 2023
Can LMs Learn New Entities from Descriptions? Challenges in Propagating Injected Knowledge Yasumasa Onoe Michael J.Q. Zhang Shankar Padmanabhan Greg Durrett Eunsol Choi KELM 275 78 0 02 May 2023
Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4 Kent K. Chang Mackenzie Cramer Sandeep Soni David Bamman RALM 249 124 0 28 Apr 2023
Search-in-the-Chain: Interactively Enhancing Large Language Models with Search for Knowledge-intensive Tasks Shicheng Xu Liang Pang Huawei Shen Xueqi Cheng Tat-Seng Chua RALM KELM LRM 175 48 0 28 Apr 2023
The MiniPile Challenge for Data-Efficient Language Models Jean Kaddour MoE ALM 133 46 0 17 Apr 2023
Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling Stella Biderman Hailey Schoelkopf Quentin G. Anthony Herbie Bradley Kyle O'Brien ... USVSN Sai Prashanth Edward Raff Aviya Skowron Lintang Sutawika Oskar van der Wal 151 1,311 0 03 Apr 2023
Recognition, recall, and retention of few-shot memories in large language models A. Orhan LRM KELM CLL 69 3 0 30 Mar 2023
Language Model Behavior: A Comprehensive Survey Tyler A. Chang Benjamin Bergen VLM LRM LM&MA 115 109 0 20 Mar 2023
Secret-Keeping in Question Answering Nathaniel W. Rollings Kent O'Sullivan Sakshum Kulshrestha KELM 41 0 0 16 Mar 2023
Testing AI on language comprehension tasks reveals insensitivity to underlying meaning Vittoria Dentella Fritz Guenther Elliot Murphy G. Marcus Evelina Leivada ELM 115 31 0 23 Feb 2023