Large Language Models Struggle to Learn Long-Tail Knowledge

15 November 2022

Papers citing "Large Language Models Struggle to Learn Long-Tail Knowledge"

50 / 254 papers shown

Title
A Novel Psychometrics-Based Approach to Developing Professional Competency Benchmark for Large Language Models Elena Kardanova Alina Ivanova Ksenia Tarasova Taras Pashchenko Aleksei Tikhoniuk Elen Yusupova Anatoly Kasprzhak Yaroslav Kuzminov Ekaterina Kruchinskaia Irina Brun 47 1 0 29 Oct 2024
Beyond Text: Optimizing RAG with Multimodal Inputs for Industrial Applications Monica Riedler Stefan Langer VLM 44 13 0 29 Oct 2024
Exploring Local Memorization in Diffusion Models via Bright Ending Attention Chong Chen Daochang Liu M. Shah Chang Xu 65 3 0 29 Oct 2024
Belief in the Machine: Investigating Epistemological Blind Spots of Language Models Mirac Suzgun Tayfun Gur Federico Bianchi Daniel E. Ho Thomas Icard Dan Jurafsky James Zou 31 1 0 28 Oct 2024
Mask-based Membership Inference Attacks for Retrieval-Augmented Generation Mingrui Liu Sixiao Zhang Cheng Long AAML 65 2 0 26 Oct 2024
DeCoRe: Decoding by Contrasting Retrieval Heads to Mitigate Hallucinations Aryo Pradipta Gema Chen Jin Ahmed Abdulaal Tom Diethe Philip Teare Beatrice Alex Pasquale Minervini Amrutha Saseendran 26 5 0 24 Oct 2024
LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answering Qingfei Zhao Ruobing Wang Yukuo Cen Daren Zha Shicheng Tan Yuxiao Dong Jie Tang RALM 41 9 0 23 Oct 2024
Scalable Influence and Fact Tracing for Large Language Model Pretraining Tyler A. Chang Dheeraj Rajagopal Tolga Bolukbasi Lucas Dixon Ian Tenney TDI 35 2 0 22 Oct 2024
Fact Recall, Heuristics or Pure Guesswork? Precise Interpretations of Language Models for Fact Completion Denitsa Saynova Lovisa Hagström Moa Johansson Richard Johansson Marco Kuhlmann HILM 43 0 0 18 Oct 2024
Evaluating Self-Generated Documents for Enhancing Retrieval-Augmented Generation with Large Language Models Jiatao Li Xinyu Hu Xunjian Yin Xiaojun Wan RALM 53 0 0 17 Oct 2024
RAG-DDR: Optimizing Retrieval-Augmented Generation Using Differentiable Data Rewards Xinze Li Sen Mei Zhenghao Liu Yukun Yan Shuo Wang ... Huimin Chen Ge Yu Zhiyuan Liu Maosong Sun Chenyan Xiong 50 7 0 17 Oct 2024
Telco-DPR: A Hybrid Dataset for Evaluating Retrieval Models of 3GPP Technical Specifications Thaina Saraiva Marco Sousa Pedro Vieira António Rodrigues 29 0 0 15 Oct 2024
A Multi-LLM Orchestration Engine for Personalized, Context-Rich Assistance Sumedh Rasal 25 0 0 13 Oct 2024
NoVo: Norm Voting off Hallucinations with Attention Heads in Large Language Models Zheng Yi Ho Siyuan Liang Sen Zhang Yibing Zhan Dacheng Tao 31 2 0 11 Oct 2024
PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoning Tingchen Fu Mrinank Sharma Philip Torr Shay B. Cohen David M. Krueger Fazl Barez AAML 50 7 0 11 Oct 2024
Large Language Models in Qualitative Research: Can We Do the Data Justice? Hope Schroeder Marianne Aubin Le Quéré Casey Randazzo David Mimno Sarita Schoenebeck 23 2 0 09 Oct 2024
Deciphering the Interplay of Parametric and Non-parametric Memory in Retrieval-augmented Language Models M. Farahani Richard Johansson RALM 33 2 0 07 Oct 2024
ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detection Yibo Yan Shen Wang Jiahao Huo Hang Li Yangqiu Song ... Kun Wang Hui Xiong Philip S. Yu Xuming Hu Qingsong Wen LRM 36 14 0 06 Oct 2024
Upsample or Upweight? Balanced Training on Heavily Imbalanced Datasets Tianjian Li Haoran Xu Weiting Tan Kenton Murray Daniel Khashabi 35 1 0 06 Oct 2024
Reward-RAG: Enhancing RAG with Reward Driven Supervision Thang Nguyen Peter Chin Yu-Wing Tai RALM 42 4 0 03 Oct 2024
Mitigating Memorization In Language Models Mansi Sakarvadia Aswathy Ajith Arham Khan Nathaniel Hudson Caleb Geniesse Kyle Chard Yaoqing Yang Ian Foster Michael W. Mahoney KELM MU 58 1 0 03 Oct 2024
Quantifying Generalization Complexity for Large Language Models Zhenting Qi Hongyin Luo Xuliang Huang Zhuokai Zhao Yibo Jiang Xiangjun Fan Himabindu Lakkaraju James Glass LRM ELM 31 5 0 02 Oct 2024
Knowledge-Driven Feature Selection and Engineering for Genotype Data with Large Language Models Joseph Lee Shu Yang Jae Young Baik Xiaoxi Liu Zhen Tan ... Zixuan Wen Bojian Hou D. Duong-Tran Tianlong Chen Li Shen 52 1 0 02 Oct 2024
Reasoning Elicitation in Language Models via Counterfactual Feedback Alihan Hüyük Xinnuo Xu Jacqueline Maasch Aditya V. Nori Javier González ReLM LRM 175 1 0 02 Oct 2024
Decoding Large-Language Models: A Systematic Overview of Socio-Technical Impacts, Constraints, and Emerging Questions Zeyneb N. Kaya Souvick Ghosh 42 0 0 25 Sep 2024
Controlling Risk of Retrieval-augmented Generation: A Counterfactual Prompting Framework Lu Chen Ruqing Zhang Jiafeng Guo Yixing Fan Xueqi Cheng 29 3 0 24 Sep 2024
RACOON: An LLM-based Framework for Retrieval-Augmented Column Type Annotation with a Knowledge Graph Linxi Wei Guorui Xiao Magdalena Balazinska 33 1 0 22 Sep 2024
Synthetic continued pretraining Zitong Yang Neil Band Shuangping Li Emmanuel Candès Tatsunori Hashimoto CLL SyDa 41 11 0 11 Sep 2024
Retrieval Augmented Correction of Named Entity Speech Recognition Errors Ernest Pusateri Anmol Walia Anirudh Kashi Bortik Bandyopadhyay Nadia Hyder Sayantan Mahinder R. Anantha Daben Liu Sashank Gondala RALM 3DV 31 2 0 09 Sep 2024
Pairing Analogy-Augmented Generation with Procedural Memory for Procedural Q&A K Roth Rushil Gupta Simon Halle Bang Liu RALM 45 0 0 02 Sep 2024
Pandora's Box or Aladdin's Lamp: A Comprehensive Analysis Revealing the Role of RAG Noise in Large Language Models Jinyang Wu Feihu Che Chuyuan Zhang Jianhua Tao Shuai Zhang Pengpeng Shao 41 2 0 24 Aug 2024
Ancient Wisdom, Modern Tools: Exploring Retrieval-Augmented LLMs for Ancient Indian Philosophy Priyanka Mandikal RALM VLM 45 0 0 21 Aug 2024
Multilingual Needle in a Haystack: Investigating Long-Context Behavior of Multilingual Large Language Models Amey Hengle Prasoon Bajpai Soham Dan Tanmoy Chakraborty LRM 34 2 0 19 Aug 2024
Training Language Models on the Knowledge Graph: Insights on Hallucinations and Their Detectability Jiri Hron Laura J. Culp Gamaleldin F. Elsayed Rosanne Liu Ben Adlam ... T. Warkentin Lechao Xiao Kelvin Xu Jasper Snoek Simon Kornblith 45 1 0 14 Aug 2024
A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning Prateek Yadav Colin Raffel Mohammed Muqeeth Lucas Caccia Haokun Liu Tianlong Chen Joey Tianyi Zhou Leshem Choshen Alessandro Sordoni MoMe 49 21 0 13 Aug 2024
KnowPO: Knowledge-aware Preference Optimization for Controllable Knowledge Selection in Retrieval-Augmented Language Models Ruizhe Zhang Yongxin Xu Yuzhen Xiao Runchuan Zhu Xinke Jiang Xu Chu Junfeng Zhao Yasha Wang 40 2 0 06 Aug 2024
Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neurons Yifei Wang Yuheng Chen Wanting Wen Yu Sheng Linjing Li D. Zeng KELM 44 5 0 06 Aug 2024
Knowledge Prompting: How Knowledge Engineers Use Large Language Models Elisavet Koutsiana Johanna Walker Michelle Nwachukwu Albert Meroño-Peñuela Elena Simperl 45 1 0 02 Aug 2024
Adaptive Contrastive Decoding in Retrieval-Augmented Generation for Handling Noisy Contexts Youna Kim Hyuhng Joon Kim Cheonbok Park Choonghyun Park Hyunsoo Cho Junyeob Kim Kang Min Yoo Sang-goo Lee Taeuk Kim 36 5 0 02 Aug 2024
MCGMark: An Encodable and Robust Online Watermark for Tracing LLM-Generated Malicious Code Peng Ding Jingyu Wu Qingyuan Zhong Dan Ma Xunliang Cai ... Shi Chen Weizhe Zhang Zibin Zheng Weizhe Zhang Zibin Zheng 48 0 0 02 Aug 2024
GOProteinGNN: Leveraging Protein Knowledge Graphs for Protein Representation Learning Dan Kalifa Uriel Singer Kira Radinsky 49 1 0 31 Jul 2024
Are Large Language Models Possible to Conduct Cognitive Behavioral Therapy? Hao Shen Zihan Li Minqiang Yang Minghui Ni Yongfeng Tao Zhengyang Yu Weihao Zheng Chen Xu Bin Hu AI4MH 20 3 0 25 Jul 2024
Benchmarks as Microscopes: A Call for Model Metrology Michael Stephen Saxon Ari Holtzman Peter West William Y. Wang Naomi Saphra 39 10 0 22 Jul 2024
MoRSE: Bridging the Gap in Cybersecurity Expertise with Retrieval Augmented Generation Marco Simoni Andrea Saracino Vinod Puthuvath Maurco Conti 60 2 0 22 Jul 2024
Knowledge Mechanisms in Large Language Models: A Survey and Perspective Meng Wang Yunzhi Yao Ziwen Xu Shuofei Qiao Shumin Deng ... Yong-jia Jiang Pengjun Xie Fei Huang Huajun Chen Ningyu Zhang 55 28 0 22 Jul 2024
Knowledge Overshadowing Causes Amalgamated Hallucination in Large Language Models Yuji Zhang Sha Li Jiateng Liu Pengfei Yu Yi R. Fung Jing Li Manling Li Heng Ji 39 11 0 10 Jul 2024
Leveraging Large Language Models for Integrated Satellite-Aerial-Terrestrial Networks: Recent Advances and Future Directions Shumaila Javaid R. A. Khalil Nasir Saeed Bin He Mohamed-Slim Alouini 39 9 0 05 Jul 2024
Why does in-context learning fail sometimes? Evaluating in-context learning on open and closed questions Xiang Li Haoran Tang Siyu Chen Ziwei Wang Ryan Chen Marcin Abram LRM 31 1 0 02 Jul 2024
Understanding Transformers via N-gram Statistics Timothy Nguyen 41 10 0 30 Jun 2024
HRDE: Retrieval-Augmented Large Language Models for Chinese Health Rumor Detection and Explainability Yanfang Chen Ding Chen Shichao Song Simin Niu Hanyu Wang Zeyun Tang Zhiyu Li Zhiyu Li 24 0 0 30 Jun 2024