Evaluation of Retrieval-Augmented Generation: A Survey

13 May 2024

Kai Zhang

Qi Liu

Papers citing "Evaluation of Retrieval-Augmented Generation: A Survey"

50 / 57 papers shown

Title
Optimizing Retrieval-Augmented Generation: Analysis of Hyperparameter Impact on Performance and Efficiency Adel Ammar Anis Koubaa Omer Nacar W. Boulila RALM 3DV 35 0 0 13 May 2025
Can LLMs Be Trusted for Evaluating RAG Systems? A Survey of Methods and Datasets Lorenz Brehme Thomas Ströhle Ruth Breu 59 0 0 28 Apr 2025
DualRAG: A Dual-Process Approach to Integrate Reasoning and Retrieval for Multi-Hop Question Answering Rong Cheng J. Liu Yan Zheng Fei Ni Jiazhen Du Hangyu Mao Fuzheng Zhang Bo-Lan Wang Jianye Hao LRM 56 0 0 25 Apr 2025
Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey Aoran Gan Hao Yu Kai Zhang Qi Liu Wenyu Yan Zhenya Huang Shiwei Tong Guoping Hu RALM 3DV 38 0 0 21 Apr 2025
Support Evaluation for the TREC 2024 RAG Track: Comparing Human versus LLM Judges Nandan Thakur Ronak Pradeep Shivani Upadhyay Daniel Fernando Campos Nick Craswell Jimmy Lin ELM 38 0 0 21 Apr 2025
Benchmarking Biopharmaceuticals Retrieval-Augmented Generation Evaluation Hanmeng Zhong Linqing Chen Weilei Wang Wentao Wu 28 0 0 15 Apr 2025
A System for Comprehensive Assessment of RAG Frameworks Mattia Rengo Senad Beadini Domenico Alfano Roberto Abbruzzese 40 1 0 10 Apr 2025
Affordable AI Assistants with Knowledge Graph of Thoughts Maciej Besta Lorenzo Paleari Jia Hao Andrea Jiang Robert Gerstenberger You Wu ... Jón Gunnar Hannesson Grzegorz Kwa'sniewski Marcin Copik H. Niewiadomski Torsten Hoefler LLMAG RALM 145 0 0 03 Apr 2025
Hyper-RAG: Combating LLM Hallucinations using Hypergraph-Driven Retrieval-Augmented Generation Yifan Feng Hao Hu Xingliang Hou Shiquan Liu Shihui Ying S. Du Han Hu Yue Gao 37 0 0 30 Mar 2025
MHTS: Multi-Hop Tree Structure Framework for Generating Difficulty-Controllable QA Datasets for RAG Evaluation Jeongsoo Lee Daeyong Kwon Kyohoon Jin Junnyeong Jeong Minwoo Sim Minwoo Kim 29 0 0 29 Mar 2025
Retrieval Augmented Generation and Understanding in Vision: A Survey and New Outlook Xu Zheng Ziqiao Weng Yuanhuiyi Lyu Lutao Jiang Haiwei Xue Bin Ren Danda Pani Paudel N. Sebe Luc Van Gool Xuming Hu 3DV 39 1 0 23 Mar 2025
KG-IRAG: A Knowledge Graph-Based Iterative Retrieval-Augmented Generation Framework for Temporal Reasoning Ruiyi Yang Hao Xue Imran Razzak Hakim Hacid Flora D. Salim RALM 88 0 0 18 Mar 2025
A Survey on Transformer Context Extension: Approaches and Evaluation Yijun Liu Jinzheng Yu Yang Xu Zhongyang Li Qingfu Zhu LLMAG 66 0 0 17 Mar 2025
A Survey on Knowledge-Oriented Retrieval-Augmented Generation Mingyue Cheng Yucong Luo Jie Ouyang Q. Liu Huijie Liu ... Bohou Zhang Jiawei Cao Jie Ma Daoyu Wang Enhong Chen 3DV 70 3 0 11 Mar 2025
In-depth Analysis of Graph-based RAG in a Unified Framework Yingli Zhou Yaodong Su Youran Sun Shu Wang Taotao Wang ... Yongwei Zhang Sicong Liang Xilin Liu Yuchi Ma Yixiang Fang 42 0 0 06 Mar 2025
KidneyTalk-open: No-code Deployment of a Private Large Language Model with Medical Documentation-Enhanced Knowledge Database for Kidney Disease Yongchao Long Chao Yang Gongzheng Tang Jinwei Wang Zhun Sui Yuxi Zhou Shenda Hong Luxia Zhang RALM 56 0 0 06 Mar 2025
LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted Contrastive Learning Zhibin Lan Liqiang Niu Fandong Meng Jie Zhou Jinsong Su VLM 69 0 0 04 Mar 2025
Optimizing open-domain question answering with graph-based retrieval augmented generation Joyce Cahoon Prerna Singh Nick Litombe Jonathan Larson Ha Trinh Yiwen Zhu A. Mueller Fotis Psallidas Carlo Curino 29 0 0 04 Mar 2025
Do Retrieval-Augmented Language Models Adapt to Varying User Needs? Peilin Wu Xinlu Zhang Wenhao Yu Xingyu Liu Xinya Du Zhiyu Zoey Chen RALM 43 0 0 27 Feb 2025
Trustworthy Answers, Messier Data: Bridging the Gap in Low-Resource Retrieval-Augmented Generation for Domain Expert Systems Nayoung Choi Grace Byun Andrew Chung Ellie S. Paek S. Lee Jinho D. Choi RALM 86 1 0 26 Feb 2025
MMRAG: Multi-Mode Retrieval-Augmented Generation with Large Language Models for Biomedical In-Context Learning Zaifu Zhan J. Wang Shuang Zhou Jiawen Deng Rui Zhang 40 4 0 21 Feb 2025
Enhancing Domain-Specific Retrieval-Augmented Generation: Synthetic Data Generation and Evaluation using Reasoning Models Aryan Jadon Avinash Patil Shashank Kumar SyDa 45 1 0 21 Feb 2025
CondAmbigQA: A Benchmark and Dataset for Conditional Ambiguous Question Answering Zongxi Li Y. Li Haoran Xie S. J. Qin 68 0 0 03 Feb 2025
Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation Satyapriya Krishna Kalpesh Krishna Anhad Mohananey Steven Schwarcz Adam Stambler Shyam Upadhyay Manaal Faruqui ReLM 3DV LRM RALM 37 13 0 28 Jan 2025
CG-RAG: Research Question Answering by Citation Graph Retrieval-Augmented LLMs Yuntong Hu Zhihan Lei Zhongjie Dai Allen Zhang Abhinav Angirekula Zheng Zhang Liang Zhao 34 0 0 28 Jan 2025
ASTRID -- An Automated and Scalable TRIaD for the Evaluation of RAG-based Clinical Question Answering Systems Mohita Chowdhury Yajie Vera He Aisling Higham Ernest Lim 58 1 0 14 Jan 2025
Unimib Assistant: designing a student-friendly RAG-based chatbot for all their needs Chiara Antico Stefano Giordano Cansu Koyuturk D. Ognibene 61 2 0 29 Nov 2024
Efficient Learning Content Retrieval with Knowledge Injection Batuhan Sariturk Rabia Bayraktar Merve Elmas Erdem 81 0 0 28 Nov 2024
ML-Promise: A Multilingual Dataset for Corporate Promise Verification Yohei Seki Hakusen Shu Anaïs Lhuissier Hanwool Lee Juyeon Kang Min-Yuh Day Chung-Chi Chen 23 0 0 07 Nov 2024
Is Our Chatbot Telling Lies? Assessing Correctness of an LLM-based Dutch Support Chatbot Herman Lassche Michiel Overeem Ayushi Rastogi 45 0 0 29 Oct 2024
Do RAG Systems Cover What Matters? Evaluating and Optimizing Responses with Sub-Question Coverage Kaige Xie Philippe Laban Prafulla Kumar Choubey Caiming Xiong C. Wu 29 1 0 20 Oct 2024
HEALTH-PARIKSHA: Assessing RAG Models for Health Chatbots in Real-World Multilingual Settings Varun Gumma Anandhita Raghunath Mohit Jain Sunayana Sitaram LM&MA 32 1 0 17 Oct 2024
Quebec Automobile Insurance Question-Answering With Retrieval-Augmented Generation David Beauchemin Zachary Gagnon Ricahrd Khoury AILaw 31 1 0 12 Oct 2024
Enterprise Benchmarks for Large Language Model Evaluation Bing Zhang Mikio Takeuchi Ryo Kawahara Shubhi Asthana Md. Maruf Hossain Guang-Jie Ren Kate Soule Yada Zhu ELM 31 2 0 11 Oct 2024
Aligning Human and LLM Judgments: Insights from EvalAssist on Task-Specific Evaluations and AI-assisted Assessment Strategy Preferences Zahra Ashktorab Michael Desmond Qian Pan James M. Johnson Martin Santillan Cooper Elizabeth M. Daly Rahul Nair Tejaswini Pedapati Swapnaja Achintalwar Werner Geyer ELM 44 4 0 01 Oct 2024
IRSC: A Zero-shot Evaluation Benchmark for Information Retrieval through Semantic Comprehension in Retrieval-Augmented Generation Scenarios Hai Lin Shaoxiong Zhan Junyou Su Haitao Zheng Hui Wang RALM 29 1 0 24 Sep 2024
HALO: Hallucination Analysis and Learning Optimization to Empower LLMs with Retrieval-Augmented Context for Guided Clinical Decision Making Sumera Anjum Hanzhi Zhang Wenjun Zhou Eun Jin Paek Xiaopeng Zhao Yunhe Feng 29 1 0 16 Sep 2024
GroUSE: A Benchmark to Evaluate Evaluators in Grounded Question Answering Sacha Muller António Loison Bilel Omrani Gautier Viaud RALM ELM 36 1 0 10 Sep 2024
LegalBench-RAG: A Benchmark for Retrieval-Augmented Generation in the Legal Domain Nicholas Pipitone Ghita Houir Alami AILaw RALM VLM ELM 29 23 0 19 Aug 2024
Graph Retrieval-Augmented Generation: A Survey Boci Peng Yun Zhu Yongchao Liu Xiaohe Bo Haizhou Shi Chuntao Hong Yan Zhang Siliang Tang 3DV 45 63 0 15 Aug 2024
RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generation Dongyu Ru Lin Qiu Xiangkun Hu Tianhang Zhang Peng Shi ... Tong He Zhiguo Wang Pengfei Liu Yue Zhang Zheng Zhang 49 12 0 15 Aug 2024
A RAG-Based Question-Answering Solution for Cyber-Attack Investigation and Attribution Sampath Rajapaksha Ruby Rani Erisa Karafili 43 3 0 12 Aug 2024
RAG Foundry: A Framework for Enhancing LLMs for Retrieval Augmented Generation Daniel Fleischer Moshe Berchansky Moshe Wasserblat Peter Izsak 3DV 44 4 0 05 Aug 2024
ABC Align: Large Language Model Alignment for Safety & Accuracy Gareth Seneque Lap-Hang Ho Peter W. Glynn Yinyu Ye Jeffrey Molendijk 41 1 0 01 Aug 2024
Adaptive Retrieval-Augmented Generation for Conversational Systems Xi Wang Procheta Sen Ruizhe Li Emine Yilmaz RALM 28 5 0 31 Jul 2024
A Tale of Trust and Accuracy: Base vs. Instruct LLMs in RAG Systems Florin Cuconasu Giovanni Trappolini Nicola Tonellotto Fabrizio Silvestri 51 2 0 21 Jun 2024
Evaluating the Efficacy of Open-Source LLMs in Enterprise-Specific RAG Systems: A Comparative Study of Performance and Scalability Gautam B A. Purwar 22 11 0 17 Jun 2024
Multi-Head RAG: Solving Multi-Aspect Problems with LLMs Maciej Besta Aleš Kubíček Roman Niggli Robert Gerstenberger Lucas Weitzendorf ... Jürgen Müller H. Niewiadomski Marcin Chrapek Michał Podstawski Torsten Hoefler 41 15 0 07 Jun 2024
A Survey on Retrieval-Augmented Text Generation for Large Language Models Yizheng Huang Jimmy X. Huang 3DV RALM 58 44 0 17 Apr 2024
Retrieval-Augmented Generation for AI-Generated Content: A Survey Penghao Zhao Hailin Zhang Qinhan Yu Zhengren Wang Yunteng Geng Fangcheng Fu Ling Yang Wentao Zhang Jie Jiang Bin Cui 3DV 115 224 0 29 Feb 2024