Benchmarking Retrieval-Augmented Generation for Medicine

20 February 2024

Aidong Zhang

Papers citing "Benchmarking Retrieval-Augmented Generation for Medicine"

28 / 28 papers shown

Title
Can LLMs Be Trusted for Evaluating RAG Systems? A Survey of Methods and Datasets Lorenz Brehme Thomas Ströhle Ruth Breu 62 0 0 28 Apr 2025
Exploring the Role of Knowledge Graph-Based RAG in Japanese Medical Question Answering with Small-Scale LLMs Yingjian Chen Feiyang Li Xingyu Song Tianxiao Li Zixin Xu Xiujie Chen Issey Sukeda Irene Z Li 23 0 0 15 Apr 2025
Benchmarking Biopharmaceuticals Retrieval-Augmented Generation Evaluation Hanmeng Zhong Linqing Chen Weilei Wang Wentao Wu 28 0 0 15 Apr 2025
AI Hiring with LLMs: A Context-Aware and Explainable Multi-Agent Framework for Resume Screening F. P. Lo Jianing Qiu Zeyu Wang Haibao Yu Yeming Chen Gao Zhang Benny P. L. Lo 28 0 0 01 Apr 2025
MedAgentsBench: Benchmarking Thinking Models and Agent Frameworks for Complex Medical Reasoning Xiangru Tang Daniel Shao Jiwoong Sohn Jiapeng Chen Jiayi Zhang ... Yilun Zhao Chenglin Wu Wenqi Shi Arman Cohan Mark B. Gerstein AI4MH LRM ELM LM&MA 62 4 0 10 Mar 2025
SearchRAG: Can Search Engines Be Helpful for LLM-based Medical Question Answering? Yucheng Shi Tianze Yang Canyu Chen Quanzheng Li Tianming Liu X. Li Ninghao Liu MedIm 46 2 0 18 Feb 2025
SimRAG: Self-Improving Retrieval-Augmented Generation for Adapting Large Language Models to Specialized Domains Ran Xu Hui Liu Sreyashi Nag Zhenwei Dai Yaochen Xie ... Chen Luo Yang Li Joyce C. Ho Carl Yang Qi He RALM 68 8 0 28 Jan 2025
ASTRID -- An Automated and Scalable TRIaD for the Evaluation of RAG-based Clinical Question Answering Systems Mohita Chowdhury Yajie Vera He Aisling Higham Ernest Lim 58 1 0 14 Jan 2025
MEG: Medical Knowledge-Augmented Large Language Models for Question Answering Laura Cabello Carmen Martin-Turrero Uchenna Akujuobi Anders Søgaard Carlos Bobed AI4MH 133 1 0 06 Nov 2024
Data Extraction Attacks in Retrieval-Augmented Generation via Backdoors Yuefeng Peng Junda Wang Hong-ye Yu Amir Houmansadr SILM 57 2 0 03 Nov 2024
From Tokens to Words: On the Inner Lexicon of LLMs Guy Kaplan Matanel Oren Yuval Reif Roy Schwartz 48 12 0 08 Oct 2024
CasiMedicos-Arg: A Medical Question Answering Dataset Annotated with Explanatory Argumentative Structures Ekaterina Sviridova Anar Yeginbergen A. Estarrona Elena Cabrio S. Villata Rodrigo Agerri 42 2 0 07 Oct 2024
Lessons Learned on Information Retrieval in Electronic Health Records: A Comparison of Embedding Models and Pooling Strategies Skatje Myers Timothy A. Miller Yanjun Gao M. Churpek Anoop Mayampurath Dmitriy Dligach Majid Afshar 23 3 0 23 Sep 2024
What is the Role of Small Models in the LLM Era: A Survey Lihu Chen Gaël Varoquaux ALM 63 23 0 10 Sep 2024
MedTrinity-25M: A Large-scale Multimodal Dataset with Multigranular Annotations for Medicine Yunfei Xie Ce Zhou Lang Gao Juncheng Wu Xianhang Li ... Sheng Liu Lei Xing James Zou Cihang Xie Yuyin Zhou LM&MA MedIm 74 23 0 06 Aug 2024
A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations Md Tahmid Rahman Laskar Sawsan Alqahtani M Saiful Bari Mizanur Rahman Mohammad Abdullah Matin Khan ... Chee Wei Tan Md. Rizwan Parvez Enamul Hoque Shafiq R. Joty Jimmy Huang ELM ALM 27 27 0 04 Jul 2024
A Textbook Remedy for Domain Shifts: Knowledge Priors for Medical Image Analysis Yue Yang Mona Gandhi Yufei Wang Yifan Wu Michael S. Yao Christopher Callison-Burch James C. Gee Mark Yatskar 50 3 0 23 May 2024
Evaluation of Retrieval-Augmented Generation: A Survey Hao Yu Aoran Gan Kai Zhang Shiwei Tong Qi Liu Zhaofeng Liu 3DV 62 80 0 13 May 2024
BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrievers Ran Xu Wenqi Shi Yue Yu Yuchen Zhuang Yanqiao Zhu M. D. Wang Joyce C. Ho Chao Zhang Carl Yang LM&MA 40 19 0 29 Apr 2024
MedExpQA: Multilingual Benchmarking of Large Language Models for Medical Question Answering Inigo Alonso Maite Oronoz Rodrigo Agerri AI4MH LM&MA ELM 47 16 1 08 Apr 2024
JMLR: Joint Medical LLM and Retrieval Training for Enhancing Reasoning and Professional Question Answering Capability Junda Wang Zhichao Yang Zonghai Yao Hong-ye Yu BDL AI4MH LRM 40 30 0 27 Feb 2024
README: Bridging Medical Jargon and Lay Understanding for Patient Education through Data-Centric NLP Zonghai Yao Nandyala Siddharth Kantu Guanghao Wei Hieu Tran Zhangqi Duan Sunjae Kwon Zhichao Yang Readme annotation team Hong-ye Yu 24 7 0 24 Dec 2023
PaperQA: Retrieval-Augmented Generative Agent for Scientific Research Jakub Lála Odhran O'Donoghue Aleksandar Shtedritski Sam Cox Samuel G. Rodriques Andrew D. White RALM 75 73 0 08 Dec 2023
Learning to Break: Knowledge-Enhanced Reasoning in Multi-Agent Debate System Haotian Wang Xiyuan Du Weijiang Yu Qianglong Chen Kun Zhu Zheng Chu Lian Yan Yi Guan 30 10 0 08 Dec 2023
Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Health Shubo Tian Qiao Jin Lana Yeganova Po-Ting Lai Qingqing Zhu ... Donald C. Comeau R. Islamaj Aadit Kapoor Xin Gao Zhiyong Lu LM&MA MedIm AI4MH 109 209 0 15 Jun 2023
Almanac: Retrieval-Augmented Language Models for Clinical Medicine C. Zakka Akash Chaurasia R. Shad Alex R. Dalal Jennifer L. Kim ... Kathleen Boyd Karen Hirsch C. Langlotz Joanna Nelson W. Hiesinger LM&MA 102 143 0 01 Mar 2023
BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models Nandan Thakur Nils Reimers Andreas Rucklé Abhishek Srivastava Iryna Gurevych VLM 231 966 0 17 Apr 2021
PubMedQA: A Dataset for Biomedical Research Question Answering Qiao Jin Bhuwan Dhingra Zhengping Liu William W. Cohen Xinghua Lu 210 807 0 13 Sep 2019