v1v2v3 (latest)

Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation

12 February 2025

Mohammad Mahdi Abootorabi

Amirhosein Zobeiri

Mahdi Dehghani

Mohammadali Mohammadkhani

Papers citing "Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation"

50 / 187 papers shown

Title
MineInsight: A Multi-sensor Dataset for Humanitarian Demining Robotics in Off-Road Environments Mario Malizia Charles Hamesse Ken Hasselmann G. D. Cubber Nikolaos Tsiogkas Eric Demeester Rob Haelterman 134 0 0 05 Jun 2025
Learning to Route Queries Across Knowledge Bases for Step-wise Retrieval-Augmented Reasoning Chunyi Peng Zhipeng Xu Zhenghao Liu Yishan Li Yukun Yan ... Zhiyuan Liu Yu Gu Minghe Yu Ge Yu Maosong Sun LRM 68 0 0 28 May 2025
AudioJailbreak: Jailbreak Attacks against End-to-End Large Audio-Language Models Guangke Chen Fu Song Zhe Zhao Xiaojun Jia Yang Liu Yanchen Qiao Weizhe Zhang AuLLM AAML 76 1 0 20 May 2025
OMGM: Orchestrate Multiple Granularities and Modalities for Efficient Multimodal Retrieval Wei Yang Jingjing Fu Rongpin Wang Jinyu Wang Lei Song Jiang Bian 55 1 0 10 May 2025
VR-RAG: Open-vocabulary Species Recognition with RAG-Assisted Large Multi-Modal Models Fahad Shahbaz Khan Jun Chen Youssef Mohamed Chun-Mei Feng Mohamed Elhoseiny VLM 109 1 0 08 May 2025
UniversalRAG: Retrieval-Augmented Generation over Corpora of Diverse Modalities and Granularities Woongyeong Yeo Kangsan Kim Soyeong Jeong Jinheon Baek Sung Ju Hwang 115 1 0 29 Apr 2025
Fashion-RAG: Multimodal Fashion Image Editing via Retrieval-Augmented Generation Fulvio Sanguigni Davide Morelli Marcella Cornia Rita Cucchiara DiffM 50 1 0 18 Apr 2025
A Visual RAG Pipeline for Few-Shot Fine-Grained Product Classification Bianca Lamm J. Keuper VLM AI4TS 117 0 0 16 Apr 2025
Leveraging LLMs for Multimodal Retrieval-Augmented Radiology Report Generation via Key Phrase Extraction Kyoyun Choi Byungmu Yoon Soobum Kim Jonggwon Park 126 1 0 10 Apr 2025
CollEX -- A Multimodal Agentic RAG System Enabling Interactive Exploration of Scientific Collections Florian Schneider Narges Baba Ahmadi Niloufar Baba Ahmadi Iris Vogel Martin Semmann Chris Biemann 77 3 0 10 Apr 2025
Reasoning LLMs for User-Aware Multimodal Conversational Agents Hamed Rahimi Jeanne Cattoni Meriem Beghili Mouad Abrini Mahdi Khoramshahi Maribel Pino Mohamed Chetouani LRM 69 3 0 02 Apr 2025
SuperRAG: Beyond RAG with Layout-Aware Graph Modeling Jeff Yang Duy-Khanh Vu Minh-Tien Nguyen Xuan-Quang Nguyen Linh Nguyen H. Le 3DV 88 6 0 28 Feb 2025
Benchmarking Retrieval-Augmented Generation in Multi-Modal Contexts Zhenghao Liu Xingsheng Zhu Tianshuo Zhou Xinyi Zhang Xiaoyuan Yi Yukun Yan Yu Gu Ge Yu Maosong Sun RALM VLM 46 3 0 24 Feb 2025
WavRAG: Audio-Integrated Retrieval Augmented Generation for Spoken Dialogue Models Yifu Chen Shengpeng Ji Haoxiao Wang Ziyi Wang Siyu Chen Jinzheng He Jin Xu Zhou Zhao AuLLM VLM 101 3 0 21 Feb 2025
VideoRAG: Retrieval-Augmented Generation with Extreme Long-Context Videos Xubin Ren Lingrui Xu Long Xia Shuaiqiang Wang Dawei Yin Chao Huang VGen VLM 140 5 0 03 Feb 2025
Unveiling the Potential of Multimodal Retrieval Augmented Generation with Planning Xiaohan Yu Zhihan Yang C. L. Philip Chen LRM 49 2 0 28 Jan 2025
RAG-Check: Evaluating Multimodal Retrieval Augmented Generation Performance Matin Mortaheb M. A. Khojastepour S. Chakradhar S. Ulukus VLM RALM 61 2 0 08 Jan 2025
RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval Di Liu Meng Chen Baotong Lu Huiqiang Jiang Zhenhua Han ... Kai Zhang Chong Chen Fan Yang Yue Yang Lili Qiu 114 44 0 03 Jan 2025
Retrieval-Augmented Dynamic Prompt Tuning for Incomplete Multimodal Learning Jian Lang Zhangtao Cheng Ting Zhong Fan Zhou VLM 56 4 0 02 Jan 2025
GFG -- Gender-Fair Generation: A CALAMITA Challenge Simona Frenda Andrea Piergentili Beatrice Savoldi Marco Madeddu Martina Rosola Silvia Casola Chiara Ferrando V. Patti Matteo Negri L. Bentivogli 68 5 0 31 Dec 2024
GME: Improving Universal Multimodal Retrieval by Multimodal LLMs Xin Zhang Yanzhao Zhang Wen Xie Mingxin Li Ziqi Dai Dingkun Long Pengjun Xie Meishan Zhang Wenjie Li Hao Fei 194 20 0 22 Dec 2024
AlzheimerRAG: Multimodal Retrieval Augmented Generation for PubMed articles A. Lahiri Qinmin Vivian Hu 118 10 0 21 Dec 2024
MegaPairs: Massive Data Synthesis For Universal Multimodal Retrieval Yueze Wang Zheng Liu Ze Liu Shitao Xiao Yueze Wang Bo Zhao Chen Jason Zhang Defu Lian Y. Xiong 120 8 0 19 Dec 2024
VisDoM: Multi-Document QA with Visually Rich Elements Using Multimodal Retrieval-Augmented Generation Manan Suri Puneet Mathur Franck Dernoncourt Kanika Goswami Ryan Rossi Dinesh Manocha 131 5 0 14 Dec 2024
OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations Linke Ouyang Yuan Qu Hongbin Zhou Jiawei Zhu Rui Zhang ... Chao Xu Bo Zhang Botian Shi Zhongying Tu Zeang Sheng 146 11 0 10 Dec 2024
CADMR: Cross-Attention and Disentangled Learning for Multimodal Recommender Systems Yasser Khalafaoui M. Lovisetto Basarab Matei Nistor Grozavu 72 1 0 03 Dec 2024
Multi-modal Retrieval Augmented Multi-modal Generation: Datasets, Evaluation Metrics and Strong Baselines Zi-Ao Ma Tian Lan Rong-Cheng Tu Yong Hu Yu-Shi Zhu Tong Zhang Heyan Huang Zhijing Wu Xian-Ling Mao VLM 130 2 0 25 Nov 2024
Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehension Yongdong Luo Xiawu Zheng Xiao Yang Guilin Li Haojia Lin Jinfa Huang Jiayi Ji Chia-Wen Lin Jiebo Luo Rongrong Ji VLM 138 27 0 20 Nov 2024
M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understanding Jaemin Cho Debanjan Mahata Ozan Irsoy Yujie He Joey Tianyi Zhou VLM 75 16 0 07 Nov 2024
Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agent Yangning Li Hai-Tao Zheng Xinyu Wang Yong Jiang Zhen Zhang ... Hui Wang Hai-Tao Zheng Pengjun Xie Philip S. Yu Fei Huang 105 22 0 05 Nov 2024
MM-Embed: Universal Multimodal Retrieval with Multimodal LLMs Sheng-Chieh Lin Chankyu Lee Mohammad Shoeybi Jimmy J. Lin Bryan Catanzaro Ming-Yu Liu 241 19 0 04 Nov 2024
Beyond Text: Optimizing RAG with Multimodal Inputs for Industrial Applications Monica Riedler Stefan Langer VLM 60 18 0 29 Oct 2024
RA-BLIP: Multimodal Adaptive Retrieval-Augmented Bootstrapping Language-Image Pre-training Muhe Ding Yang Ma Pengda Qin Jianlong Wu Yuhong Li Liqiang Nie 63 1 0 18 Oct 2024
VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents S. Yu C. Tang Bokai Xu Junbo Cui Junhao Ran ... Zhenghao Liu Shuo Wang Xu Han Zhiyuan Liu Maosong Sun VLM 184 39 0 14 Oct 2024
DRCap: Decoding CLAP Latents with Retrieval-Augmented Generation for Zero-shot Audio Captioning Xiquan Li Wenxi Chen Ziyang Ma Xuenan Xu Yuzhe Liang Zhisheng Zheng Qiuqiang Kong Xie Chen VLM 100 5 0 12 Oct 2024
MRAG-Bench: Vision-Centric Evaluation for Retrieval-Augmented Multimodal Models Wenbo Hu Jia-Chen Gu Zi-Yi Dou Mohsen Fayyaz Pan Lu Kai-Wei Chang Nanyun Peng VLM 118 8 0 10 Oct 2024
Quantifying the Gaps Between Translation and Native Perception in Training for Multimodal, Multilingual Retrieval Kyle Buettner Adriana Kovashka VLM 87 4 0 02 Oct 2024
AsthmaBot: Multi-modal, Multi-Lingual Retrieval Augmented Generation For Asthma Patient Support Adil Bahaj Mounir Ghogho 120 2 0 24 Sep 2024
SURf: Teaching Large Vision-Language Models to Selectively Utilize Retrieved Information Jiashuo Sun Jihai Zhang Yucheng Zhou Zhaochen Su Xiaoye Qu Yu Cheng 73 13 0 21 Sep 2024
mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understanding Anwen Hu Haiyang Xu Liang Zhang Jiabo Ye Ming Yan Ji Zhang Qin Jin Fei Huang Jingren Zhou VLM 81 35 0 05 Sep 2024
UniFashion: A Unified Vision-Language Model for Multimodal Fashion Retrieval and Generation Xiangyu Zhao Yuehan Zhang Wenlong Zhang X. Wu 74 5 0 21 Aug 2024
MuRAR: A Simple and Effective Multimodal Retrieval and Answer Refinement Framework for Multimodal Question Answering Zhengyuan Zhu Daniel Lee Hong Zhang Sai Sree Harsha Loic Feujio Akash Maharaj Yunyao Li 58 3 0 16 Aug 2024
AppAgent v2: Advanced Agent for Flexible Mobile Interactions Yanda Li Chi Zhang Wanqi Yang Bin-Bin Fu Pei Cheng Xin Chen Ling Chen Yunchao Wei LLMAG LM&Ro 82 26 0 05 Aug 2024
MLLM Is a Strong Reranker: Advancing Multimodal Retrieval-augmented Generation via Knowledge-enhanced Reranking and Noise-injected Training Rivik Setty Chengjin Xu Vinay Setty Jian Guo 73 13 0 31 Jul 2024
Pruning Large Language Models with Semi-Structural Adaptive Sparse Training Weiyu Huang Yuezhou Hu Guohao Jian Jun Zhu Jianfei Chen 86 8 0 30 Jul 2024
An Empirical Comparison of Video Frame Sampling Methods for Multi-Modal RAG Retrieval Mahesh Kandhare Thibault Gisselbrecht 64 5 0 22 Jul 2024
Golden-Retriever: High-Fidelity Agentic Retrieval Augmented Generation for Industrial Knowledge Base Zhiyu An Xianzhong Ding Yen-Chun Fu Cheng-Chung Chu Yan Li Wan Du RALM 59 6 0 20 Jul 2024
EchoSight: Advancing Visual-Language Models with Wiki Knowledge Yibin Yan Weidi Xie RALM 126 14 0 17 Jul 2024
OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation Kepan Nan Rui Xie Penghao Zhou Tiehan Fan Zhenheng Yang Zhijie Chen Xiang Li Jian Yang Ying Tai 135 92 0 02 Jul 2024
MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizations Yubo Ma Yuhang Zang Liangyu Chen Meiqi Chen Yizhu Jiao ... Liangming Pan Yu-Gang Jiang Jiaqi Wang Yixin Cao Aixin Sun ELM RALM VLM 91 33 0 01 Jul 2024