v1v2 (latest)

Large Language Models Struggle to Learn Long-Tail Knowledge

15 November 2022

Papers citing "Large Language Models Struggle to Learn Long-Tail Knowledge"

50 / 260 papers shown

Title
Adaptive Contrastive Decoding in Retrieval-Augmented Generation for Handling Noisy Contexts Youna Kim Sungmin Cho Cheonbok Park Choonghyun Park Hyunsoo Cho Junyeob Kim Kang Min Yoo Sang-goo Lee Taeuk Kim 79 7 0 02 Aug 2024
MCGMark: An Encodable and Robust Online Watermark for Tracing LLM-Generated Malicious Code Peng Ding Jingyu Wu Qingyuan Zhong Dan Ma Xunliang Cai ... Shi Chen Weizhe Zhang Zibin Zheng Weizhe Zhang Zibin Zheng 117 0 0 02 Aug 2024
GOProteinGNN: Leveraging Protein Knowledge Graphs for Protein Representation Learning Dan Kalifa Uriel Singer Kira Radinsky 146 1 0 31 Jul 2024
Are Large Language Models Possible to Conduct Cognitive Behavioral Therapy? Hao Shen Zihan Li Minqiang Yang Minghui Ni Yongfeng Tao Zhengyang Yu Weihao Zheng Chen Xu Bin Hu AI4MH 69 3 0 25 Jul 2024
Benchmarks as Microscopes: A Call for Model Metrology Michael Stephen Saxon Ari Holtzman Peter West William Y. Wang Naomi Saphra 112 13 0 22 Jul 2024
MoRSE: Bridging the Gap in Cybersecurity Expertise with Retrieval Augmented Generation Marco Simoni Andrea Saracino Vinod Puthuvath Maurco Conti 109 4 0 22 Jul 2024
Knowledge Mechanisms in Large Language Models: A Survey and Perspective Meng Wang Yunzhi Yao Ziwen Xu Shuofei Qiao Shumin Deng ... Yong Jiang Pengjun Xie Fei Huang Huajun Chen Ningyu Zhang 139 39 0 22 Jul 2024
Knowledge Overshadowing Causes Amalgamated Hallucination in Large Language Models Yuji Zhang Sha Li Jiateng Liu Pengfei Yu Yi R. Fung Jing Li Manling Li Heng Ji 110 12 0 10 Jul 2024
Leveraging Large Language Models for Integrated Satellite-Aerial-Terrestrial Networks: Recent Advances and Future Directions Shumaila Javaid R. A. Khalil Nasir Saeed Bin He Mohamed-Slim Alouini 109 12 0 05 Jul 2024
Why does in-context learning fail sometimes? Evaluating in-context learning on open and closed questions Xiang Li Haoran Tang Siyu Chen Ziwei Wang Ryan Chen Marcin Abram LRM 106 4 0 02 Jul 2024
Understanding Transformers via N-gram Statistics Timothy Nguyen 88 10 0 30 Jun 2024
HRDE: Retrieval-Augmented Large Language Models for Chinese Health Rumor Detection and Explainability Yanfang Chen Ding Chen Shichao Song Pengnian Qi Hanyu Wang Zeyun Tang Feiyu Xiong Zhiyu Li 50 0 0 30 Jun 2024
Learning to Correct for QA Reasoning with Black-box LLMs Jaehyung Kim Dongyoung Kim Yiming Yang LRM 116 4 0 26 Jun 2024
Few-shot Personalization of LLMs with Mis-aligned Responses Jaehyung Kim Yiming Yang 160 9 0 26 Jun 2024
A Tale of Trust and Accuracy: Base vs. Instruct LLMs in RAG Systems Florin Cuconasu Giovanni Trappolini Nicola Tonellotto Fabrizio Silvestri 83 2 0 21 Jun 2024
CodeRAG-Bench: Can Retrieval Augment Code Generation? Zora Z. Wang Akari Asai Xinyan Velocity Yu Frank F. Xu Yiqing Xie Graham Neubig Daniel Fried RALM 229 42 0 20 Jun 2024
R^2AG: Incorporating Retrieval Information into Retrieval Augmented Generation Fuda Ye Shuangyin Li Yongqi Zhang Lei Chen 74 0 0 19 Jun 2024
Multi-Stage Balanced Distillation: Addressing Long-Tail Challenges in Sequence-Level Knowledge Distillation Yuhang Zhou Jing Zhu Paiheng Xu Xiaoyu Liu Xiyao Wang Danai Koutra Wei Ai Furong Huang 137 5 0 19 Jun 2024
How Do Large Language Models Acquire Factual Knowledge During Pretraining? Hoyeon Chang Jinho Park Seonghyeon Ye Sohee Yang Youngkyung Seo Du-Seong Chang Minjoon Seo KELM 97 46 0 17 Jun 2024
Are Large Language Models True Healthcare Jacks-of-All-Trades? Benchmarking Across Health Professions Beyond Physician Exams Zheheng Luo Chenhan Yuan Qianqian Xie Sophia Ananiadou ELM AI4MH LM&MA 84 0 0 17 Jun 2024
Refiner: Restructure Retrieval Content Efficiently to Advance Question-Answering Capabilities Zhonghao Li Xuming Hu Aiwei Liu Kening Zheng Shijie Huang Hui Xiong RALM 189 8 0 17 Jun 2024
CRAG -- Comprehensive RAG Benchmark Xiao Yang Kai Sun Hao Xin Yushi Sun Nikita Bhalla ... Nirav Shah Rakesh Wanga Anuj Kumar Wen-tau Yih Xin Luna Dong 90 32 0 07 Jun 2024
AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathways Zehang Deng Yongjian Guo Changzhou Han Wanlun Ma Junwu Xiong Sheng Wen Yang Xiang 157 49 0 04 Jun 2024
DAFNet: Dynamic Auxiliary Fusion for Sequential Model Editing in Large Language Models Taolin Zhang Qizhou Chen Dongyang Li Chengyu Wang Xiaofeng He Longtao Huang Hui Xue Junyuan Huang CLL KELM 84 6 0 31 May 2024
Evaluating the External and Parametric Knowledge Fusion of Large Language Models Hao Zhang Yuyang Zhang Xiaoguang Li Wenxuan Shi Haonan Xu ... Yasheng Wang Lifeng Shang Qun Liu Yong Liu Ruiming Tang KELM 97 5 0 29 May 2024
Nearest Neighbor Speculative Decoding for LLM Generation and Attribution Minghan Li Xilun Chen Ari Holtzman Beidi Chen Jimmy Lin Wen-tau Yih Xi Lin RALM BDL 240 14 0 29 May 2024
A Textbook Remedy for Domain Shifts: Knowledge Priors for Medical Image Analysis Yue Yang Mona Gandhi Yufei Wang Yifan Wu Michael S. Yao Christopher Callison-Burch James C. Gee Mark Yatskar 129 4 0 23 May 2024
Adapting Multi-modal Large Language Model to Concept Drift From Pre-training Onwards Xiaoyu Yang Jie Lu Enshui Yu VLM 110 7 0 22 May 2024
SynthesizRR: Generating Diverse Datasets with Retrieval Augmentation Abhishek Divekar Greg Durrett 141 10 0 16 May 2024
LMD3: Language Model Data Density Dependence John Kirchenbauer Garrett Honke Gowthami Somepalli Jonas Geiping Daphne Ippolito Katherine Lee Tom Goldstein David Andre 97 7 0 10 May 2024
Can large language models understand uncommon meanings of common words? Jinyang Wu Feihu Che Xinxin Zheng Shuai Zhang Ruihan Jin Shuai Nie Pengpeng Shao Jianhua Tao 80 4 0 09 May 2024
Enhancing Contextual Understanding in Large Language Models through Contrastive Decoding Zheng Zhao Emilio Monti Jens Lehmann H. Assem 99 33 0 04 May 2024
Recall Them All: Retrieval-Augmented Language Models for Long Object List Extraction from Long Documents Sneha Singhania Simon Razniewski Gerhard Weikum RALM 129 1 0 04 May 2024
FLAME: Factuality-Aware Alignment for Large Language Models Sheng-Chieh Lin Luyu Gao Barlas Oğuz Wenhan Xiong Jimmy Lin Wen-tau Yih Xilun Chen HILM 95 20 0 02 May 2024
Automated Construction of Theme-specific Knowledge Graphs Linyi Ding Sizhe Zhou Jinfeng Xiao Jiawei Han 142 12 0 29 Apr 2024
Tool Calling: Enhancing Medication Consultation via Retrieval-Augmented Large Language Models Zhongzhen Huang Kui Xue Yongqi Fan Linjie Mu Ruoyu Liu Tong Ruan Shaoting Zhang Xiaofan Zhang LM&MA RALM 93 5 0 27 Apr 2024
Language in Vivo vs. in Silico: Size Matters but Larger Language Models Still Do Not Comprehend Language on a Par with Humans Due to Impenetrable Semantic Reference Vittoria Dentella Fritz Guenther Evelina Leivada ELM 96 2 0 23 Apr 2024
U Can't Gen This? A Survey of Intellectual Property Protection Methods for Data in Generative AI Tanja Sarcevic Alicja Karlowicz Rudolf Mayer Ricardo A. Baeza-Yates Andreas Rauber 103 7 0 22 Apr 2024
AmbigDocs: Reasoning across Documents on Different Entities under the Same Name Yoonsang Lee Xi Ye Eunsol Choi 79 14 0 18 Apr 2024
Unifying Bias and Unfairness in Information Retrieval: A Survey of Challenges and Opportunities with Large Language Models Sunhao Dai Chen Xu Shicheng Xu Liang Pang Zhenhua Dong Jun Xu 117 83 0 17 Apr 2024
MemLLM: Finetuning LLMs to Use An Explicit Read-Write Memory Ali Modarressi Abdullatif Köksal Ayyoob Imani Mohsen Fayyaz Hinrich Schütze KELM 196 11 0 17 Apr 2024
Fewer Truncations Improve Language Modeling Hantian Ding Zijian Wang Giovanni Paolini Varun Kumar Anoop Deoras Dan Roth Stefano Soatto 111 14 0 16 Apr 2024
RAR-b: Reasoning as Retrieval Benchmark Chenghao Xiao G. Thomas Al Moubayed LRM RALM 143 12 0 09 Apr 2024
No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance Vishaal Udandarao Ameya Prabhu Adhiraj Ghosh Yash Sharma Philip Torr Adel Bibi Samuel Albanie Matthias Bethge VLM 221 55 0 04 Apr 2024
AI and the Problem of Knowledge Collapse Andrew J. Peterson 105 23 0 04 Apr 2024
On Large Language Models' Hallucination with Regard to Known Facts Che Jiang Biqing Qi Xiangyu Hong Dayuan Fu Yang Cheng Fandong Meng Mo Yu Bowen Zhou Jie Zhou HILM LRM 75 22 0 29 Mar 2024
FIT-RAG: Black-Box RAG with Factual Information and Token Reduction Yuren Mao Xuemei Dong Wenyi Xu Yunjun Gao Bin Wei Ying Zhang 62 10 0 21 Mar 2024
WikiFactDiff: A Large, Realistic, and Temporally Adaptable Dataset for Atomic Factual Knowledge Update in Causal Language Models Hichem Ammar Khodja Frédéric Béchet Quentin Brabant Alexis Nasr Gwénolé Lecorvé HILM KELM SyDa 54 8 0 21 Mar 2024
Re-Search for The Truth: Multi-round Retrieval-augmented Large Language Models are Strong Fake News Detectors Guanghua Li Wensheng Lu Wei Zhang Defu Lian Kezhong Lu Rui Mao Kai Shu Hao Liao HILM 72 5 0 14 Mar 2024
Unfamiliar Finetuning Examples Control How Language Models Hallucinate Katie Kang Eric Wallace Claire Tomlin Aviral Kumar Sergey Levine HILM LRM 106 58 0 08 Mar 2024