Enhancing Multi-modal and Multi-hop Question Answering via Structured Knowledge and Unified Retrieval-Generation

16 December 2022

Qian Yang

Qian Chen

Wen Wang

Baotian Hu

Min Zhang

ArXiv PDF HTML

Papers citing "Enhancing Multi-modal and Multi-hop Question Answering via Structured Knowledge and Unified Retrieval-Generation"

23 / 23 papers shown

Title
VLMT: Vision-Language Multimodal Transformer for Multimodal Multi-hop Question Answering Qi Zhi Lim C. Lee K. Lim Kalaiarasi Sonai Muthu Anbananthen 31 0 0 11 Apr 2025
MM-PoisonRAG: Disrupting Multimodal RAG with Local and Global Poisoning Attacks Hyeonjeong Ha Qiusi Zhan Jeonghwan Kim Dimitrios Bralios Saikrishna Sanniboina Nanyun Peng Kai-Wei Chang Daniel Kang Heng Ji KELM AAML 69 1 0 25 Feb 2025
Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation Mohammad Mahdi Abootorabi Amirhosein Zobeiri Mahdi Dehghani Mohammadali Mohammadkhani Bardia Mohammadi Omid Ghahroodi M. Baghshah Ehsaneddin Asgari RALM 105 4 0 12 Feb 2025
An Entailment Tree Generation Approach for Multimodal Multi-Hop Question Answering with Mixture-of-Experts and Iterative Feedback Mechanism Qing Zhang Haocheng Lv Jie Liu Z. Chen Jianyong Duan Hao Wang Li He Mingying Xv 72 1 0 08 Dec 2024
RA-BLIP: Multimodal Adaptive Retrieval-Augmented Bootstrapping Language-Image Pre-training Muhe Ding Yang Ma Pengda Qin Jianlong Wu Yuhong Li Liqiang Nie 23 1 0 18 Oct 2024
Crafting Personalized Agents through Retrieval-Augmented Generation on Editable Memory Graphs Zheng Wang Zhongyang Li Zeren Jiang Dandan Tu Wei Shi 49 7 0 28 Sep 2024
Reasoning Factual Knowledge in Structured Data with Large Language Models Sirui Huang Yanggan Gu Xuming Hu Zhonghao Li Qing Li Guandong Xu HILM LMTD 32 1 0 22 Aug 2024
MLLM Is a Strong Reranker: Advancing Multimodal Retrieval-augmented Generation via Knowledge-enhanced Reranking and Noise-injected Training Rivik Setty Chengjin Xu Vinay Setty Jian Guo 34 12 0 31 Jul 2024
MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation? Zhaorun Chen Yichao Du Zichen Wen Yiyang Zhou Chenhang Cui ... Jiawei Zhou Zhuokai Zhao Rafael Rafailov Chelsea Finn Huaxiu Yao EGVM MLLM 58 29 0 05 Jul 2024
UniBind: LLM-Augmented Unified and Balanced Representation Space to Bind Them All Yuanhuiyi Lyu Xueye Zheng Jiazhou Zhou Lin Wang 32 16 0 19 Mar 2024
Retrieval-Augmented Generation for AI-Generated Content: A Survey Penghao Zhao Hailin Zhang Qinhan Yu Zhengren Wang Yunteng Geng Fangcheng Fu Ling Yang Wentao Zhang Jie Jiang Bin Cui 3DV 115 228 0 29 Feb 2024
Exploring Hybrid Question Answering via Program-based Prompting Qi Shi Han Cui Haofeng Wang Qingfu Zhu Wanxiang Che Ting Liu 35 4 0 16 Feb 2024
MoqaGPT : Zero-Shot Multi-modal Open-domain Question Answering with Large Language Model Le Zhang Yihong Wu Fengran Mo Jian-Yun Nie Aishwarya Agrawal MLLM RALM 34 6 0 20 Oct 2023
Progressive Evidence Refinement for Open-domain Multimodal Retrieval Question Answering Shuwen Yang Anran Wu Xingjiao Wu Luwei Xiao Tianlong Ma Cheng Jin Liang He 27 2 0 15 Oct 2023
MMHQA-ICL: Multimodal In-context Learning for Hybrid Question Answering over Text, Tables and Images Weihao Liu Fangyu Lei Tongxu Luo Jiahe Lei Shizhu He Jun Zhao Kang Liu LMTD 29 9 0 09 Sep 2023
Unified Language Representation for Question Answering over Text, Tables, and Images Yu Bowen Cheng Fu Haiyang Yu Fei Huang Yongbin Li LMTD 24 20 0 29 Jun 2023
Complex QA and language models hybrid architectures, Survey Xavier Daull P. Bellot Emmanuel Bruno Vincent Martin Elisabeth Murisasco ELM 28 15 0 17 Feb 2023
Mixed-modality Representation Learning and Pre-training for Joint Table-and-Text Retrieval in OpenQA Junjie Huang Wanjun Zhong Qianchu Liu Ming Gong Daxin Jiang Nan Duan VLM RALM LMTD 103 14 0 11 Oct 2022
Modeling Multi-hop Question Answering as Single Sequence Prediction Semih Yavuz Kazuma Hashimoto Yingbo Zhou N. Keskar Caiming Xiong 43 27 0 18 May 2022
Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skills Ori Yoran Alon Talmor Jonathan Berant ReLM LRM 183 53 0 15 Jul 2021
UniK-QA: Unified Representations of Structured and Unstructured Knowledge for Open-Domain Question Answering Barlas Oğuz Xilun Chen Vladimir Karpukhin Stanislav Peshterliev Dmytro Okhonko M. Schlichtkrull Sonal Gupta Yashar Mehdad Scott Yih 211 87 0 29 Dec 2020
Answering Open-Domain Questions of Varying Reasoning Steps from Text Peng Qi Haejun Lee OghenetegiriTGSido Christopher D. Manning KELM RALM LRM 191 55 0 23 Oct 2020
Unified Vision-Language Pre-Training for Image Captioning and VQA Luowei Zhou Hamid Palangi Lei Zhang Houdong Hu Jason J. Corso Jianfeng Gao MLLM VLM 252 927 0 24 Sep 2019