Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning

26 June 2023

Fuxiao Liu

Papers citing "Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning"

50 / 196 papers shown

Title
Critique Before Thinking: Mitigating Hallucination through Rationale-Augmented Instruction Tuning Zexian Yang Dian Li Dayan Wu Gang Liu Weiping Wang MLLM LRM 41 0 0 12 May 2025
Hallucination-Aware Multimodal Benchmark for Gastrointestinal Image Analysis with Large Vision-Language Models Bidur Khanal Sandesh Pokhrel Sanjay Bhandari Ramesh Rana Nikesh Shrestha Ram Bahadur Gurung Cristian A. Linte Angus Watson Yash Raj Shrestha Binod Bhattarai VLM 36 0 0 11 May 2025
RoboOS: A Hierarchical Embodied Framework for Cross-Embodiment and Multi-Agent Collaboration Huajie Tan Xiaoshuai Hao Minglan Lin Pengwei Wang Yaoxu Lyu Mingyu Cao Zhongyuan Wang S. Zhang LM&Ro 48 0 0 06 May 2025
A Comprehensive Analysis for Visual Object Hallucination in Large Vision-Language Models Liqiang Jing Guiming Hardy Chen Ehsan Aghazadeh Xin Eric Wang Xinya Du 55 0 0 04 May 2025
VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations on Synthetic Video Understanding Zongxia Li Xiyang Wu Guangyao Shi Yubin Qin Hongyang Du Tianyi Zhou Dinesh Manocha Jordan Lee Boyd-Graber MLLM 57 0 0 02 May 2025
Black-Box Visual Prompt Engineering for Mitigating Object Hallucination in Large Vision Language Models Sangmin Woo Kang Zhou Yun Zhou Shuai Wang Sheng Guan Haibo Ding Lin Lee Cheong VPVLM 83 0 0 30 Apr 2025
Localizing Before Answering: A Hallucination Evaluation Benchmark for Grounded Medical Multimodal LLMs Dung Tien Nguyen Minh Khoi Ho Huy Ta T. Nguyen Qi Chen ... Zhibin Liao Minh Nguyen Nhat To Johan W. Verjans Phi Le Nguyen Vu Minh Hieu Phan 29 0 0 30 Apr 2025
Antidote: A Unified Framework for Mitigating LVLM Hallucinations in Counterfactual Presupposition and Object Perception Yuanchen Wu Lu Zhang Hang Yao Junlong Du Ke Yan Shouhong Ding Yunsheng Wu Zhaoxin Fan MLLM 71 0 0 29 Apr 2025
Multimodal Large Language Models for Medicine: A Comprehensive Survey Jiarui Ye Hao Tang LM&MA 89 0 0 29 Apr 2025
Span-Level Hallucination Detection for LLM-Generated Answers Passant Elchafei Mervet Abu-Elkheir HILM LRM 71 1 0 25 Apr 2025
A Large Vision-Language Model based Environment Perception System for Visually Impaired People Zezhou Chen Zhaoxiang Liu Ning Wang Kohou Wang Shiguo Lian 52 0 0 25 Apr 2025
AdaViP: Aligning Multi-modal LLMs via Adaptive Vision-enhanced Preference Optimization Jinda Lu Jinghan Li Yuan Gao Junkang Wu Jiancan Wu Xuben Wang Xiangnan He 120 0 0 22 Apr 2025
Efficient Contrastive Decoding with Probabilistic Hallucination Detection - Mitigating Hallucinations in Large Vision Language Models - Laura Fieback Nishilkumar Balar Jakob Spiegelberg Hanno Gottschalk MLLM VLM 80 0 0 16 Apr 2025
The Mirage of Performance Gains: Why Contrastive Decoding Fails to Address Multimodal Hallucination Hao Yin Gunagzong Si Zilei Wang 145 0 0 14 Apr 2025
Perception in Reflection Yana Wei Liang Zhao Kangheng Lin En Yu Yuang Peng ... Jianjian Sun Haoran Wei Zheng Ge Xiangyu Zhang Vishal M. Patel 31 0 0 09 Apr 2025
PaMi-VDPO: Mitigating Video Hallucinations by Prompt-Aware Multi-Instance Video Preference Learning Xinpeng Ding Kaipeng Zhang Jinahua Han Lanqing Hong Hang Xu Xiaomeng Li MLLM VLM 178 0 0 08 Apr 2025
Towards Trustworthy GUI Agents: A Survey Yucheng Shi Wenhao Yu Wenlin Yao Wenhu Chen Ninghao Liu 44 4 0 30 Mar 2025
DASH: Detection and Assessment of Systematic Hallucinations of VLMs Maximilian Augustin Yannic Neuhaus Matthias Hein VLM 55 1 0 30 Mar 2025
Breaking Language Barriers in Visual Language Models via Multilingual Textual Regularization Iñigo Pikabea Iñaki Lacunza Oriol Pareras Carlos Escolano Aitor Gonzalez-Agirre Javier Hernando Marta Villegas VLM 52 0 0 28 Mar 2025
Exploring Hallucination of Large Multimodal Models in Video Understanding: Benchmark, Analysis and Mitigation Hongcheng Gao Jiashu Qu Jingyi Tang Baolong Bi Yijia Liu Hongyu Chen Li Liang Li Su Qingming Huang MLLM VLM LRM 85 5 0 25 Mar 2025
Bridging Writing Manner Gap in Visual Instruction Tuning by Creating LLM-aligned Instructions Dong Jing Nanyi Fei Zhiwu Lu 47 0 0 24 Mar 2025
Where do Large Vision-Language Models Look at when Answering Questions? X. Xing Chia-Wen Kuo Li Fuxin Yulei Niu Fan Chen Ming Li Ying Wu Longyin Wen Sijie Zhu LRM 62 0 0 18 Mar 2025
Aligning Multimodal LLM with Human Preference: A Survey Tao Yu Yinger Zhang Chaoyou Fu Junkang Wu Jinda Lu ... Qingsong Wen Z. Zhang Yan Huang Liang Wang Tieniu Tan 167 2 0 18 Mar 2025
ClearSight: Visual Signal Enhancement for Object Hallucination Mitigation in Multimodal Large language Models Hao Yin Guangzong Si Zilei Wang 142 1 0 17 Mar 2025
Grounded Chain-of-Thought for Multimodal Large Language Models Qiong Wu Xiangcong Yang Yiyi Zhou Chenxin Fang Baiyang Song Xiaoshuai Sun Rongrong Ji LRM 83 1 0 17 Mar 2025
Mitigating Cross-Modal Distraction and Ensuring Geometric Feasibility via Affordance-Guided, Self-Consistent MLLMs for Food Preparation Task Planning Yu-Hong Shen Chuan-Yu Wu Yi-Ru Yang Yen-Ling Tai Yi-Ting Chen LRM 36 0 0 17 Mar 2025
Attention Reallocation: Towards Zero-cost and Controllable Hallucination Mitigation of MLLMs Chongjun Tu Peng Ye Dongzhan Zhou Lei Bai Gang Yu Tao Chen Wanli Ouyang 66 0 0 13 Mar 2025
Through the Magnifying Glass: Adaptive Perception Magnification for Hallucination-Free VLM Decoding Shunqi Mao Chaoyi Zhang Weidong Cai MLLM 149 0 0 13 Mar 2025
OmniMamba: Efficient and Unified Multimodal Understanding and Generation via State Space Models Jialv Zou Bencheng Liao Qian Zhang Wenyu Liu Xinggang Wang Mamba MLLM 82 1 0 11 Mar 2025
Oasis: One Image is All You Need for Multimodal Instruction Data Synthesis Letian Zhang Quan Cui Bingchen Zhao Cheng Yang MLLM SyDa 54 0 0 11 Mar 2025
Attention Hijackers: Detect and Disentangle Attention Hijacking in LVLMs for Hallucination Mitigation Beitao Chen Xinyu Lyu Lianli Gao Jingkuan Song H. Shen 70 1 0 11 Mar 2025
Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models Wenxuan Huang Bohan Jia Zijie Zhai Shaosheng Cao Zheyu Ye Fei Zhao Zhe Xu Yao Hu Shaohui Lin MU OffRL LRM MLLM ReLM VLM 59 41 0 09 Mar 2025
PerturboLLaVA: Reducing Multimodal Hallucinations with Perturbative Visual Training Cong Chen Mingyu Liu Chenchen Jing Y. Zhou Fengyun Rao Hao Chen Bo Zhang Chunhua Shen MLLM AAML VLM 65 3 0 09 Mar 2025
From Captions to Rewards (CAREVL): Leveraging Large Language Model Experts for Enhanced Reward Modeling in Large Vision-Language Models Muzhi Dai Jiashuo Sun Zhiyuan Zhao Shixuan Liu Rui Li Junyu Gao Xuelong Li VLM 53 0 0 08 Mar 2025
TPC: Cross-Temporal Prediction Connection for Vision-Language Model Hallucination Reduction Chao Wang Weiwei Fu Yang Zhou MLLM VLM 74 0 0 06 Mar 2025
HalCECE: A Framework for Explainable Hallucination Detection through Conceptual Counterfactuals in Image Captioning Maria Lymperaiou Giorgos Filandrianos Angeliki Dimitriou Athanasios Voulodimos Giorgos Stamou MLLM 40 0 0 01 Mar 2025
Octopus: Alleviating Hallucination via Dynamic Contrastive Decoding Wei Suo Lijun Zhang Mengyang Sun Lin Yuanbo Wu Peng Wang Yinger Zhang MLLM VLM 47 1 0 01 Mar 2025
MedHallTune: An Instruction-Tuning Benchmark for Mitigating Medical Hallucination in Vision-Language Models Qiao Yan Yuchen Yuan Xiaowei Hu Yihan Wang Jiaqi Xu Jinpeng Li Chi-Wing Fu Pheng Ann Heng MLLM VLM LM&MA 55 0 0 28 Feb 2025
Mitigating Hallucinations in Large Vision-Language Models by Adaptively Constraining Information Flow Jiaqi Bai Hongcheng Guo Zhongyuan Peng Jian Yang Z. Li Mingze Li Zhihong Tian VLM 62 0 0 28 Feb 2025
Vision-Encoders (Already) Know What They See: Mitigating Object Hallucination via Simple Fine-Grained CLIPScore Hongseok Oh Wonseok Hwang VLM 41 0 0 27 Feb 2025
Multimodal Large Language Models for Text-rich Image Understanding: A Comprehensive Review Pei Fu Tongkun Guan Zining Wang Zhentao Guo Chen Duan ... Boming Chen Jiayao Ma Qianyi Jiang Kai Zhou Junfeng Luo VLM 62 0 0 23 Feb 2025
Mitigating Hallucinations in Large Vision-Language Models via Summary-Guided Decoding Kyungmin Min Minbeom Kim Kang-il Lee Dongryeol Lee Kyomin Jung MLLM 89 3 0 20 Feb 2025
What are Models Thinking about? Understanding Large Language Model Hallucinations "Psychology" through Model Inner State Analysis Peiran Wang Yang Liu Yunfei Lu Jue Hong Ye Wu HILM LRM 77 0 0 20 Feb 2025
Can Hallucination Correction Improve Video-Language Alignment? Lingjun Zhao Mingyang Xie Paola Cascante-Bonilla Hal Daumé III Kwonjoon Lee HILM VLM 62 0 0 20 Feb 2025
Understanding and Evaluating Hallucinations in 3D Visual Language Models Ruiying Peng Kaiyuan Li Weichen Zhang Chen Gao Xinlei Chen Yongqian Li 45 0 0 18 Feb 2025
LowRA: Accurate and Efficient LoRA Fine-Tuning of LLMs under 2 Bits Zikai Zhou Qizheng Zhang Hermann Kumbong Kunle Olukotun MQ 246 0 0 12 Feb 2025
Refine Knowledge of Large Language Models via Adaptive Contrastive Learning Hai-Tao Zheng Haojing Huang Jiayi Kuang Yangning Li Shu Guo C. Qu Xiaoyu Tan Hai-Tao Zheng Ying Shen Philip S. Yu CLL 66 5 0 11 Feb 2025
MULTI: Multimodal Understanding Leaderboard with Text and Images Zichen Zhu Yang Xu Lu Chen Jingkai Yang Yichuan Ma ... Yingzi Ma Situo Zhang Zihan Zhao Liangtai Sun Kai Yu VLM 54 5 0 08 Jan 2025
HoVLE: Unleashing the Power of Monolithic Vision-Language Models with Holistic Vision-Language Embedding Chenxin Tao Shiqian Su X. Zhu Chenyu Zhang Zhe Chen ... Wenhai Wang Lewei Lu Gao Huang Yu Qiao Jifeng Dai MLLM VLM 104 2 0 20 Dec 2024
Nullu: Mitigating Object Hallucinations in Large Vision-Language Models via HalluSpace Projection Le Yang Ziwei Zheng Boxu Chen Zhengyu Zhao Chenhao Lin Chao Shen VLM 140 3 0 18 Dec 2024