Retrieving Multimodal Information for Augmented Generation: A Survey

20 March 2023

Hailin Chen

Papers citing "Retrieving Multimodal Information for Augmented Generation: A Survey"

50 / 80 papers shown

Title
ArtRAG: Retrieval-Augmented Generation with Structured Context for Visual Art Understanding Shuai Wang Ivona Najdenkoska Hongyi Zhu S. Rudinac Monika Kackovic N. Wijnberg M. Worring 228 0 0 09 May 2025
HiPerRAG: High-Performance Retrieval Augmented Generation for Scientific Insights Ozan Gokdemir Carlo Siebenschuh Alexander Brace Azton Wells Brian Hsu ... A. Anandkumar Ian Foster R. Stevens V. Vishwanath A. Ramanathan VLM 37 0 0 07 May 2025
CollEX -- A Multimodal Agentic RAG System Enabling Interactive Exploration of Scientific Collections Florian Schneider Narges Baba Ahmadi Niloufar Baba Ahmadi Iris Vogel Martin Semmann Chris Biemann 35 0 0 10 Apr 2025
A Multimedia Analytics Model for the Foundation Model Era M. Worring Jan Zahálka Stef van den Elzen M. T. Fischer Daniel A. Keim VGen HAI 37 0 0 08 Apr 2025
LiveVQA: Live Visual Knowledge Seeking Mingyang Fu Yuyang Peng Benlin Liu Yao Wan D. Z. Chen 28 0 0 07 Apr 2025
A Survey on Knowledge-Oriented Retrieval-Augmented Generation Mingyue Cheng Yucong Luo Jie Ouyang Q. Liu Huijie Liu ... Bohou Zhang Jiawei Cao Jie Ma Daoyu Wang Enhong Chen 3DV 70 3 0 11 Mar 2025
Poisoned-MRAG: Knowledge Poisoning Attacks to Multimodal Retrieval Augmented Generation Yinuo Liu Zenghui Yuan Guiyao Tie Jiawen Shi Lichao Sun Lichao Sun Neil Zhenqiang Gong 40 1 0 08 Mar 2025
MCiteBench: A Benchmark for Multimodal Citation Text Generation in MLLMs Caiyu Hu Yikai Zhang Tinghui Zhu Yiwei Ye Yanghua Xiao 83 0 0 04 Mar 2025
SuperRAG: Beyond RAG with Layout-Aware Graph Modeling Jeff Yang Duy-Khanh Vu Minh-Tien Nguyen Xuan-Quang Nguyen Linh Nguyen H. Le 3DV 68 0 0 28 Feb 2025
Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation Mohammad Mahdi Abootorabi Amirhosein Zobeiri Mahdi Dehghani Mohammadali Mohammadkhani Bardia Mohammadi Omid Ghahroodi M. Baghshah Ehsaneddin Asgari RALM 100 4 0 12 Feb 2025
Enhanced Multimodal RAG-LLM for Accurate Visual Question Answering Junxiao Xue Quan Deng Fei Yu Yanhao Wang Jun Wang Y. Li VLM 43 3 0 31 Dec 2024
AlzheimerRAG: Multimodal Retrieval Augmented Generation for PubMed articles A. Lahiri Qinmin Vivian Hu 69 5 0 21 Dec 2024
Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agent Yangning Li Yinghui Li Xinyu Wang Yong-feng Jiang Zhen Zhang ... Hui Wang Hai-Tao Zheng Pengjun Xie Philip S. Yu Fei Huang 62 15 0 05 Nov 2024
RAP: Retrieval-Augmented Personalization for Multimodal Large Language Models Haoran Hao Jiaming Han Changsheng Li Yu-Feng Li Xiangyu Yue RALM 46 1 0 17 Oct 2024
Self-adaptive Multimodal Retrieval-Augmented Generation Wenjia Zhai VLM 36 0 0 15 Oct 2024
MRAG-Bench: Vision-Centric Evaluation for Retrieval-Augmented Multimodal Models Wenbo Hu Jia-Chen Gu Zi-Yi Dou Mohsen Fayyaz Pan Lu Kai-Wei Chang Nanyun Peng VLM 66 4 0 10 Oct 2024
Enhancing Post-Hoc Attributions in Long Document Comprehension via Coarse Grained Answer Decomposition Pritika Ramu Koustava Goswami Apoorv Saxena Balaji Vasan Srinivavsan 33 1 0 25 Sep 2024
AsthmaBot: Multi-modal, Multi-Lingual Retrieval Augmented Generation For Asthma Patient Support Adil Bahaj Mounir Ghogho 38 2 0 24 Sep 2024
Investigating Context-Faithfulness in Large Language Models: The Roles of Memory Strength and Evidence Style Yuepei Li Kang Zhou Qiao Qiao Bach Nguyen Qing Wang Qi Li HILM RALM 29 2 0 17 Sep 2024
SFR-RAG: Towards Contextually Faithful LLMs Xuan-Phi Nguyen Shrey Pandit Senthil Purushwalkam Austin Xu Hailin Chen Yifei Ming Zixuan Ke Silvio Savarese Caiming Xong Shafiq Joty RALM 86 7 0 16 Sep 2024
A Comparison of LLM Finetuning Methods & Evaluation Metrics with Travel Chatbot Use Case Sonia Meyer Shreya Singh Bertha Tam Christopher Ton Angel Ren 34 3 0 07 Aug 2024
Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey) K. Kenthapadi M. Sameki Ankur Taly HILM ELM AILaw 39 12 0 10 Jul 2024
Synthetic Multimodal Question Generation Ian Wu Sravan Jayanthi Vijay Viswanathan Simon Rosenberg Sina Pakazad Tongshuang Wu Graham Neubig 42 2 0 02 Jul 2024
We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning? Runqi Qiao Qiuna Tan Guanting Dong Minhui Wu Chong Sun ... Yida Xu Muxi Diao Zhimin Bao Chen Li Honggang Zhang VLM LRM 41 31 0 01 Jul 2024
Searching for Best Practices in Retrieval-Augmented Generation Xiaohua Wang Zhenghua Wang Xuan Gao Feiran Zhang Yixin Wu ... Qi Qian Ruicheng Yin Changze Lv Xiaoqing Zheng Xuanjing Huang 52 40 0 01 Jul 2024
ColPali: Efficient Document Retrieval with Vision Language Models Manuel Faysse Hugues Sibille Tony Wu Bilel Omrani Gautier Viaud C´eline Hudelot Pierre Colombo VLM 60 21 0 27 Jun 2024
XL-HeadTags: Leveraging Multimodal Retrieval Augmentation for the Multilingual Generation of News Headlines and Tags Faisal Tareque Shohan Mir Tafseer Nayeem Samsul Islam Abu Ubaida Akash Shafiq R. Joty 34 2 0 06 Jun 2024
Jina CLIP: Your CLIP Model Is Also Your Text Retriever Andreas Koukounas Georgios Mastrapas Michael Gunther Bo Wang Scott Martens ... Saahil Ognawala Susana Guzman Maximilian Werk Nan Wang Han Xiao VLM 25 15 0 30 May 2024
Peering into the Mind of Language Models: An Approach for Attribution in Contextual Question Answering Anirudh Phukan Shwetha Somasundaram Apoorv Saxena Koustava Goswami Balaji Vasan Srinivasan 32 8 0 28 May 2024
Preparing for Black Swans: The Antifragility Imperative for Machine Learning Ming Jin 36 2 0 18 May 2024
A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models Wenqi Fan Yujuan Ding Liang-bo Ning Shijie Wang Hengyun Li Dawei Yin Tat-Seng Chua Qing Li RALM 3DV 40 185 0 10 May 2024
ChatHuman: Language-driven 3D Human Understanding with Retrieval-Augmented Tool Reasoning Jing Lin Yao Feng Weiyang Liu Michael J. Black 3DH LRM 45 6 0 07 May 2024
RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processing Yucheng Hu Yuxing Lu RALM 58 17 0 30 Apr 2024
Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMs Yu Xia Rui Wang Xu Liu Mingyan Li Tong Yu Xiang Chen Julian McAuley Shuai Li LRM 50 18 0 24 Apr 2024
PDF-MVQA: A Dataset for Multimodal Information Retrieval in PDF-based Visual Question Answering Yihao Ding Kaixuan Ren Jiabin Huang Siwen Luo S. Han 40 1 0 19 Apr 2024
Enhancing Trust in Autonomous Agents: An Architecture for Accountability and Explainability through Blockchain and Large Language Models Laura Fernández-Becerra Miguel Ángel González Santamarta Ángel Manuel Guerrero Higueras Francisco J. Rodríguez-Lera Vicente Matellán Olivera 34 0 0 14 Mar 2024
RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Horizon Generation Zihao Wang Anji Liu Haowei Lin Jiaqi Li Xiaojian Ma Yitao Liang ReLM RALM LRM 87 47 0 08 Mar 2024
Vision-Language Models for Medical Report Generation and Visual Question Answering: A Review Iryna Hartsock Ghulam Rasool 40 62 0 04 Mar 2024
Retrieval-Augmented Generation for AI-Generated Content: A Survey Penghao Zhao Hailin Zhang Qinhan Yu Zhengren Wang Yunteng Geng Fangcheng Fu Ling Yang Wentao Zhang Jie Jiang Bin Cui 3DV 115 228 0 29 Feb 2024
Exploring ChatGPT for Next-generation Information Retrieval: Opportunities and Challenges Yizheng Huang Jimmy X. Huang 33 10 0 17 Feb 2024
The Sound of Healthcare: Improving Medical Transcription ASR Accuracy with Large Language Models Ayo Adedeji Sarita Joshi Brendan Doohan LM&MA 22 14 0 12 Feb 2024
A Comprehensive Evaluation of GPT-4V on Knowledge-Intensive Visual Question Answering Yunxin Li Longyue Wang Baotian Hu Xinyu Chen Wanqi Zhong Chenyang Lyu Wei Wang Min Zhang ELM 27 21 0 13 Nov 2023
PDFTriage: Question Answering over Long, Structured Documents Jon Saad-Falcon Joe Barrow Alexa F. Siu A. Nenkova David Seunghyun Yoon Ryan A. Rossi Franck Dernoncourt RALM 22 19 0 16 Sep 2023
Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts Jian Xie Kai Zhang Jiangjie Chen Renze Lou Yu-Chuan Su RALM 201 153 0 22 May 2023
Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sources Xingxuan Li Ruochen Zhao Yew Ken Chia Bosheng Ding Shafiq R. Joty Soujanya Poria Lidong Bing HILM BDL LRM 84 87 0 22 May 2023
Panda LLM: Training Data and Evaluation for Open-Sourced Chinese Instruction-Following Large Language Models Fangkai Jiao Bosheng Ding Tianze Luo Zhanfeng Mo ELM ALM 19 3 0 04 May 2023
RAMM: Retrieval-augmented Biomedical Visual Question Answering with Multi-modal Pre-training Zheng Yuan Qiao Jin Chuanqi Tan Zhengyun Zhao Hongyi Yuan Fei Huang Songfang Huang 49 27 0 01 Mar 2023
Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models Rongjie Huang Jia-Bin Huang Dongchao Yang Yi Ren Luping Liu Mingze Li Zhenhui Ye Jinglin Liu Xiaoyue Yin Zhou Zhao DiffM 142 316 0 30 Jan 2023
Rethinking with Retrieval: Faithful Large Language Model Inference Hangfeng He Hongming Zhang Dan Roth KELM LRM 141 156 0 31 Dec 2022
Cap4Video: What Can Auxiliary Captions Do for Text-Video Retrieval? Wenhao Wu Haipeng Luo Bo Fang Jingdong Wang Wanli Ouyang 95 80 0 31 Dec 2022