Large Language Models Struggle to Learn Long-Tail Knowledge

15 November 2022

Papers citing "Large Language Models Struggle to Learn Long-Tail Knowledge"

50 / 254 papers shown

Title
On the Tip of the Tongue: Analyzing Conceptual Representation in Large Language Models with Reverse-Dictionary Probe Ningyu Xu Qi Zhang Menghan Zhang Peng Qian Xuanjing Huang LRM 67 3 0 22 Feb 2024
Can Language Models Act as Knowledge Bases at Scale? Qiyuan He Yizhong Wang Wenya Wang KELM LRM 29 8 0 22 Feb 2024
Kuaiji: the First Chinese Accounting Large Language Model Jiayuan Luo Songhua Yang Xiaoling Qiu Panyu Chen Yufei Nai Wenxuan Zeng Wentao Zhang Xinke Jiang RALM ALM 38 1 0 21 Feb 2024
ActiveRAG: Revealing the Treasures of Knowledge via Active Learning Zhipeng Xu Zhenghao Liu Yibin Liu Chenyan Xiong Yukun Yan Shuo Wang Shi Yu Zhiyuan Liu Ge Yu VLM 44 3 0 21 Feb 2024
Retrieval Helps or Hurts? A Deeper Dive into the Efficacy of Retrieval Augmentation to Language Models Seiji Maekawa Hayate Iso Sairam Gurajada Nikita Bhutani RALM KELM 45 14 0 21 Feb 2024
AnaloBench: Benchmarking the Identification of Abstract and Long-context Analogies Xiao Ye Andrew Wang Jacob Choi Yining Lu Shreya Sharma Lingfeng Shen Vijay Tiyyala Nicholas Andrews Daniel Khashabi ELM 39 8 0 19 Feb 2024
RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Model Jianhao Yuan Shuyang Sun Daniel Omeiza Bo Zhao Paul Newman Lars Kunze Matthew Gadd LRM 36 49 0 16 Feb 2024
AutoGPT+P: Affordance-based Task Planning with Large Language Models Timo Birr Christoph Pohl Abdelrahman Younes Tamim Asfour LM&Ro 32 15 0 16 Feb 2024
Into the Unknown: Self-Learning Large Language Models Teddy Ferdinan Jan Kocoñ P. Kazienko 33 2 0 14 Feb 2024
T-RAG: Lessons from the LLM Trenches M. Fatehkia J. Lucas Sanjay Chawla LLMAG 40 21 0 12 Feb 2024
Prompt Perturbation in Retrieval-Augmented Generation based Large Language Models Zhibo Hu Chen Wang Yanfeng Shu Helen Paik Paik Liming Zhu SILM RALM 51 8 0 11 Feb 2024
On Catastrophic Inheritance of Large Foundation Models Hao Chen Bhiksha Raj Xing Xie Jindong Wang AI4CE 56 12 0 02 Feb 2024
Improving Sequential Recommendations with LLMs Artun Boz Wouter Zorgdrager Zoe Kotti Jesse Harte Panos Louridas Dietmar Jannach Vassilios Karakoidas Marios Fragkoulis KELM LRM 70 4 0 02 Feb 2024
RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval Parth Sarthi Salman Abdullah Aditi Tuli Shubh Khanna Anna Goldie Christopher D. Manning RALM 19 123 0 31 Jan 2024
LLaMP: Large Language Model Made Powerful for High-fidelity Materials Knowledge Retrieval and Distillation Chiang Yuan Elvis Hsieh Chia-Hong Chou Janosh Riebesell 33 10 0 30 Jan 2024
Rethinking Interpretability in the Era of Large Language Models Chandan Singh J. Inala Michel Galley Rich Caruana Jianfeng Gao LRM AI4CE 77 62 0 30 Jan 2024
Blinded by Generated Contexts: How Language Models Merge Generated and Retrieved Contexts When Knowledge Conflicts? Hexiang Tan Fei Sun Wanli Yang Yuanzhuo Wang Qi Cao Xueqi Cheng 33 13 0 22 Jan 2024
Speak It Out: Solving Symbol-Related Problems with Symbol-to-Language Conversion for Language Models Yile Wang Sijie Cheng Zixin Sun Peng Li Yang Liu ReLM LRM 32 4 0 22 Jan 2024
Hallucination is Inevitable: An Innate Limitation of Large Language Models Ziwei Xu Sanjay Jain Mohan S. Kankanhalli HILM LRM 71 218 0 22 Jan 2024
Named Entity Recognition Under Domain Shift via Metric Learning for Life Sciences Hongyi Liu Qingyun Wang Payam Karisani Heng Ji 16 1 0 19 Jan 2024
AboutMe: Using Self-Descriptions in Webpages to Document the Effects of English Pretraining Data Filters L. Lucy Suchin Gururangan Luca Soldaini Emma Strubell David Bamman Lauren Klein Jesse Dodge 28 15 0 12 Jan 2024
Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models Asma Ghandeharioun Avi Caciularu Adam Pearce Lucas Dixon Mor Geva 34 88 0 11 Jan 2024
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems Tianyu Cui Yanling Wang Chuanpu Fu Yong Xiao Sijia Li ... Junwu Xiong Xinyu Kong Zujie Wen Ke Xu Qi Li 60 56 0 11 Jan 2024
Aligning Translation-Specific Understanding to General Understanding in Large Language Models Yi-Chong Huang Xiaocheng Feng Baohang Li Chengpeng Fu Wenshuai Huo Ting Liu Bing Qin 30 0 0 10 Jan 2024
Studying and Recommending Information Highlighting in Stack Overflow Answers Shahla Shaan Ahmed Shaowei Wang Yuan Tian Tse-Hsun Chen Chen Haoxiang Zhang HAI 14 5 0 03 Jan 2024
HyKGE: A Hypothesis Knowledge Graph Enhanced Framework for Accurate and Reliable Medical LLMs Responses Xinke Jiang Ruizhe Zhang Yongxin Xu Rihong Qiu Yue Fang ... Jinyi Tang Hongxin Ding Xu Chu Junfeng Zhao Yasha Wang RALM 32 20 0 26 Dec 2023
FoodLMM: A Versatile Food Assistant using Large Multi-modal Model Yuehao Yin Huiyan Qi B. Zhu Jingjing Chen Yu-Gang Jiang Chong-Wah Ngo 26 19 0 22 Dec 2023
Retrieval-Augmented Generation for Large Language Models: A Survey Yunfan Gao Yun Xiong Xinyu Gao Kangxiang Jia Jinliu Pan Yuxi Bi Yi Dai Jiawei Sun Meng Wang Haofen Wang 3DV RALM 61 1,530 1 18 Dec 2023
Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs O. Ovadia Menachem Brief Moshik Mishaeli Oren Elisha RALM 40 134 0 10 Dec 2023
A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepedia Giovanni Monea Maxime Peyrard Martin Josifoski Vishrav Chaudhary Jason Eisner Emre Kiciman Hamid Palangi Barun Patra Robert West KELM 51 12 0 04 Dec 2023
RELIC: Investigating Large Language Model Responses using Self-Consistency Furui Cheng Vilém Zouhar Simran Arora Mrinmaya Sachan Hendrik Strobelt Mennatallah El-Assady HILM 24 18 0 28 Nov 2023
A Survey of the Evolution of Language Model-Based Dialogue Systems Hongru Wang Lingzhi Wang Yiming Du Liang Chen Jing Zhou Yufei Wang Kam-Fai Wong LRM 67 21 0 28 Nov 2023
Large Language Models in Education: Vision and Opportunities Wensheng Gan Zhenlian Qi Jiayang Wu Chun-Wei Lin AI4Ed 44 71 0 22 Nov 2023
Do Smaller Language Models Answer Contextualised Questions Through Memorisation Or Generalisation? Tim Hartill Joshua Bensemann Michael Witbrock Patricia Riddle KELM 30 0 0 21 Nov 2023
Ever: Mitigating Hallucination in Large Language Models through Real-Time Verification and Rectification Haoqiang Kang Juntong Ni Huaxiu Yao HILM LRM 32 34 0 15 Nov 2023
How Well Do Large Language Models Truly Ground? Hyunji Lee Se June Joo Chaeeun Kim Joel Jang Doyoung Kim Kyoung-Woon On Minjoon Seo HILM 33 6 0 15 Nov 2023
Data Similarity is Not Enough to Explain Language Model Performance Gregory Yauney Emily Reif David M. Mimno 47 6 0 15 Nov 2023
Safer-Instruct: Aligning Language Models with Automated Preference Data Taiwei Shi Kai Chen Jieyu Zhao ALM SyDa 30 21 0 15 Nov 2023
CPopQA: Ranking Cultural Concept Popularity by LLMs Ming Jiang Mansi Joshi 22 6 0 14 Nov 2023
Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applications Zhangyin Feng Weitao Ma Weijiang Yu Lei Huang Haotian Wang Qianglong Chen Weihua Peng Xiaocheng Feng Bing Qin Ting Liu KELM 23 37 0 10 Nov 2023
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions Lei Huang Weijiang Yu Weitao Ma Weihong Zhong Zhangyin Feng ... Qianglong Chen Weihua Peng Xiaocheng Feng Bing Qin Ting Liu LRM HILM 44 732 0 09 Nov 2023
The Effect of Scaling, Retrieval Augmentation and Form on the Factual Consistency of Language Models Lovisa Hagström Denitsa Saynova Tobias Norlund Moa Johansson Richard Johansson KELM HILM 29 8 0 02 Nov 2023
Diffusion Models for Reinforcement Learning: A Survey Zhengbang Zhu Hanye Zhao Haoran He Yichao Zhong Shenyu Zhang Haoquan Guo Tingting Chen Weinan Zhang 43 60 0 02 Nov 2023
Blending Reward Functions via Few Expert Demonstrations for Faithful and Accurate Knowledge-Grounded Dialogue Generation Wanyu Du Yangfeng Ji 24 1 0 02 Nov 2023
The Mystery of In-Context Learning: A Comprehensive Survey on Interpretation and Analysis Yuxiang Zhou Jiazheng Li Yanzheng Xiang Hanqi Yan Lin Gui Yulan He 24 14 0 01 Nov 2023
Filter bubbles and affective polarization in user-personalized large language model outputs Tomo Lazovich 16 2 0 31 Oct 2023
Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Models Paul Youssef Osman Alperen Koracs Meijie Li Jorg Schlotterer Christin Seifert KELM 24 16 0 25 Oct 2023
Emptying the Ocean with a Spoon: Should We Edit Models? Yuval Pinter Michael Elhadad KELM 27 26 0 18 Oct 2023
The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Models Aviv Slobodkin Omer Goldman Avi Caciularu Ido Dagan Shauli Ravfogel HILM LRM 49 24 0 18 Oct 2023
Document-Level In-Context Few-Shot Relation Extraction via Pre-Trained Language Models Yilmazcan Ozyurt Stefan Feuerriegel Ce Zhang 26 1 0 17 Oct 2023