Hallucination of Multimodal Large Language Models: A Survey

29 April 2024

Tianjun Xiao

Zheng Zhang

Papers citing "Hallucination of Multimodal Large Language Models: A Survey"

50 / 115 papers shown

Title
Hallucination-Aware Multimodal Benchmark for Gastrointestinal Image Analysis with Large Vision-Language Models Bidur Khanal Sandesh Pokhrel Sanjay Bhandari Ramesh Rana Nikesh Shrestha Ram Bahadur Gurung Cristian A. Linte Angus Watson Y. Shrestha Binod Bhattarai VLM 31 0 0 11 May 2025
Adaptive Stress Testing Black-Box LLM Planners Neeloy Chakraborty John Pohovey Melkior Ornik Katherine Driggs-Campbell 28 0 0 08 May 2025
Looking Beyond Language Priors: Enhancing Visual Comprehension and Attention in Multimodal Models Aarti Ghatkesar Uddeshya Upadhyay Ganesh Venkatesh VLM 38 0 0 08 May 2025
VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations for Synthetic Videos Zongxia Li Xiyang Wu Yubin Qin Guangyao Shi Hongyang Du Dinesh Manocha Tianyi Zhou Jordan Boyd-Graber MLLM 46 0 0 02 May 2025
HyPerAlign: Hypotheses-driven Personalized Alignment Cristina Garbacea Chenhao Tan 51 0 0 29 Apr 2025
Multimodal Large Language Models for Medicine: A Comprehensive Survey Jiarui Ye Hao Tang LM&MA 84 0 0 29 Apr 2025
CAMU: Context Augmentation for Meme Understanding Girish A. Koushik Diptesh Kanojia Helen Treharne Aditya Joshi VLM 96 0 0 24 Apr 2025
Multimodal Large Language Models for Enhanced Traffic Safety: A Comprehensive Review and Future Trends M. Tami Mohammed Elhenawy Huthaifa I. Ashqar 26 0 0 21 Apr 2025
Hydra: An Agentic Reasoning Approach for Enhancing Adversarial Robustness and Mitigating Hallucinations in Vision-Language Models Chung-En Hsuan-Chih Chen Brian Jalaian Nathaniel D. Bastian AAML VLM 44 0 0 19 Apr 2025
Low-hallucination Synthetic Captions for Large-Scale Vision-Language Model Pre-training X. Zhang Yarong Zeng Xinting Huang Hu Hu Runquan Xie Han Hu Zhanhui Kang MLLM VLM 45 0 0 17 Apr 2025
AeroLite: Tag-Guided Lightweight Generation of Aerial Image Captions Xing Zi Tengjun Ni Xianjing Fan Xian Tao Jun Li Ali Braytee Mukesh Prasad 23 0 0 13 Apr 2025
Data Metabolism: An Efficient Data Design Schema For Vision Language Model Jingyuan Zhang Hongzhi Zhang Zhou Haonan Chenxi Sun Xingguang Ji Jiakang Wang Fanheng Kong Y. Liu Qi Wang Fuzheng Zhang VLM 53 1 0 10 Apr 2025
Decoupling Contrastive Decoding: Robust Hallucination Mitigation in Multimodal Large Language Models Wei Chen Xin Yan Bin Wen Fan Yang Tingting Gao Di Zhang Long Chen MLLM 92 0 0 09 Apr 2025
Patch Matters: Training-free Fine-grained Image Caption Enhancement via Local Perception Ruotian Peng Haiying He Yake Wei Yandong Wen D. Hu VLM 39 0 0 09 Apr 2025
Explaining Low Perception Model Competency with High-Competency Counterfactuals Sara Pohland Claire Tomlin DiffM AAML 51 0 0 07 Apr 2025
TARAC: Mitigating Hallucination in LVLMs via Temporal Attention Real-time Accumulative Connection C. Xie Tongxuan Liu Lei Jiang Yuting Zeng J. Guo Yunheng Shen Weizhe Huang Jing Li Xiaohua Xu VLM 61 0 0 05 Apr 2025
Towards Trustworthy GUI Agents: A Survey Yucheng Shi Wenhao Yu Wenlin Yao Wenhu Chen Ninghao Liu 39 3 0 30 Mar 2025
Exploring Hallucination of Large Multimodal Models in Video Understanding: Benchmark, Analysis and Mitigation Hongcheng Gao Jiashu Qu Jingyi Tang Baolong Bi Y. Liu Hongyu Chen Li Liang Li Su Qingming Huang MLLM VLM LRM 83 3 0 25 Mar 2025
Mitigating Object Hallucinations in MLLMs via Multi-Frequency Perturbations Shuo Li Jiajun Sun Guodong Zheng Xiaoran Fan Yujiong Shen ... Wenming Tan Tao Ji Tao Gui Qi Zhang Xuanjing Huang AAML VLM 85 0 0 19 Mar 2025
Do Multimodal Large Language Models Understand Welding? Grigorii Khvatskii Yong Suk Lee Corey Angst Maria Gibbs Robert Landers Nitesh V. Chawla AI4CE 44 1 0 18 Mar 2025
Can Large Vision Language Models Read Maps Like a Human? Shuo Xing Zezhou Sun Shuangyu Xie Kaiyuan Chen Yanjia Huang Yuping Wang Jiachen Li Dezhen Song Zhengzhong Tu 60 2 0 18 Mar 2025
Uncertainty Distillation: Teaching Language Models to Express Semantic Confidence Sophia Hager David Mueller Kevin Duh Nicholas Andrews 65 0 0 18 Mar 2025
RAG-KG-IL: A Multi-Agent Hybrid Framework for Reducing Hallucinations and Enhancing LLM Reasoning through RAG and Incremental Knowledge Graph Learning Integration Hong Qing Yu Frank McQuade 46 1 0 14 Mar 2025
Taxonomic Reasoning for Rare Arthropods: Combining Dense Image Captioning and RAG for Interpretable Classification Nathaniel Lesperance S. Ratnasingham Graham W. Taylor VLM 74 0 0 13 Mar 2025
ExtremeAIGC: Benchmarking LMM Vulnerability to AI-Generated Extremist Content Bhavik Chandna Mariam Aboujenane Usman Naseem 60 0 0 13 Mar 2025
EAZY: Eliminating Hallucinations in LVLMs by Zeroing out Hallucinatory Image Tokens Liwei Che Tony Qingze Liu Jing Jia Weiyi Qin Ruixiang Tang Vladimir Pavlovic MLLM VLM 100 1 0 10 Mar 2025
TPC: Cross-Temporal Prediction Connection for Vision-Language Model Hallucination Reduction Chao Wang Weiwei Fu Yang Zhou MLLM VLM 69 0 0 06 Mar 2025
MCiteBench: A Benchmark for Multimodal Citation Text Generation in MLLMs Caiyu Hu Yikai Zhang Tinghui Zhu Yiwei Ye Yanghua Xiao 81 0 0 04 Mar 2025
Evaluating and Predicting Distorted Human Body Parts for Generated Images Lu Ma Kaibo Cao Hao Liang Jiaxin Lin Z. Li Yuhong Liu Jihong Zhang Wentao Zhang Bin Cui MedIm 39 0 0 02 Mar 2025
HalCECE: A Framework for Explainable Hallucination Detection through Conceptual Counterfactuals in Image Captioning Maria Lymperaiou Giorgos Filandrianos Angeliki Dimitriou Athanasios Voulodimos Giorgos Stamou MLLM 35 0 0 01 Mar 2025
Towards Statistical Factuality Guarantee for Large Vision-Language Models Z. Li Chao Yan Nicholas J. Jackson Wendi Cui B. Li Jiaxin Zhang Bradley Malin 71 0 0 27 Feb 2025
Mitigating Hallucinations in Diffusion Models through Adaptive Attention Modulation Trevine Oorloff Yaser Yacoob Abhinav Shrivastava 46 0 0 24 Feb 2025
Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generation Y. Yang Ajay Patel Matt Deitke Tanmay Gupta Luca Weihs ... Mark Yatskar Chris Callison-Burch Ranjay Krishna Aniruddha Kembhavi Christopher Clark SyDa 68 2 0 21 Feb 2025
Re-Align: Aligning Vision Language Models via Retrieval-Augmented Direct Preference Optimization Shuo Xing Yuping Wang Peiran Li Ruizheng Bai Y. Wang Chengxuan Qian Huaxiu Yao Zhengzhong Tu 87 6 0 18 Feb 2025
Mitigating Visual Knowledge Forgetting in MLLM Instruction-tuning via Modality-decoupled Gradient Descent Junda Wu Yuxin Xiong Xintong Li Yu Xia Ruoyu Wang ... Sungchul Kim Ryan Rossi Lina Yao Jingbo Shang Julian McAuley CLL VLM 49 0 0 17 Feb 2025
Valuable Hallucinations: Realizable Non-realistic Propositions Qiucheng Chen Bo Wang LRM 54 0 0 16 Feb 2025
MRAMG-Bench: A Comprehensive Benchmark for Advancing Multimodal Retrieval-Augmented Multimodal Generation Qinhan Yu Zhiyou Xiao Binghui Li Zhengren Wang C. L. P. Chen W. Zhang RALM VLM 86 0 0 06 Feb 2025
Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoning Yibo Yan Shen Wang Jiahao Huo Jingheng Ye Zhendong Chu Xuming Hu Philip S. Yu Carla P. Gomes B. Selman Qingsong Wen LRM 121 9 0 05 Feb 2025
Visual Attention Never Fades: Selective Progressive Attention ReCalibration for Detailed Image Captioning in Multimodal Large Language Models Mingi Jung Saehuyng Lee Eunji Kim Sungroh Yoon 68 0 0 03 Feb 2025
Robust-LLaVA: On the Effectiveness of Large-Scale Robust Image Encoders for Multi-modal Large Language Models H. Malik Fahad Shamshad Muzammal Naseer Karthik Nandakumar F. Khan Salman Khan AAML MLLM VLM 66 0 0 03 Feb 2025
Learning to Summarize from LLM-generated Feedback Hwanjun Song Taewon Yun Yuho Lee Jihwan Oh Gihun Lee Jason (Jinglun) Cai Hang Su 73 2 0 28 Jan 2025
Supervision-free Vision-Language Alignment Giorgio Giannone Ruoteng Li Qianli Feng Evgeny Perevodchikov Rui Chen Aleix M. Martinez VLM 58 0 0 08 Jan 2025
Performance Gap in Entity Knowledge Extraction Across Modalities in Vision Language Models Ido Cohen Daniela Gottesman Mor Geva Raja Giryes VLM 87 0 1 18 Dec 2024
A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Future Shilin Sun Wenbin An Feng Tian Fang Nan Qidong Liu J. Liu N. Shah Ping Chen 91 2 0 18 Dec 2024
Nullu: Mitigating Object Hallucinations in Large Vision-Language Models via HalluSpace Projection Le Yang Ziwei Zheng Boxu Chen Zhengyu Zhao Chenhao Lin Chao Shen VLM 138 3 0 18 Dec 2024
Who Brings the Frisbee: Probing Hidden Hallucination Factors in Large Vision-Language Model via Causality Analysis Po-Hsuan Huang Jeng-Lin Li Chin-Po Chen Ming-Ching Chang Wei-Chao Chen LRM 74 1 0 04 Dec 2024
Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey Yunkai Dang Kaichen Huang Jiahao Huo Yibo Yan S. Huang ... Kun Wang Yong Liu Jing Shao Hui Xiong Xuming Hu LRM 96 14 0 03 Dec 2024
FactCheXcker: Mitigating Measurement Hallucinations in Chest X-ray Report Generation Models Alice Heiman Xiaoman Zhang E. Chen Sung Eun Kim Pranav Rajpurkar HILM MedIm 77 0 0 27 Nov 2024
A Topic-level Self-Correctional Approach to Mitigate Hallucinations in MLLMs Lehan He Zeren Chen Zhelun Shi Tianyu Yu Jing Shao Lu Sheng MLLM 111 1 0 26 Nov 2024
ICT: Image-Object Cross-Level Trusted Intervention for Mitigating Object Hallucination in Large Vision-Language Models Junzhe Chen Tianshu Zhang S. Huang Yuwei Niu Linfeng Zhang Lijie Wen Xuming Hu MLLM VLM 163 2 0 22 Nov 2024