Detecting and Preventing Hallucinations in Large Vision Language Models

11 August 2023

Papers citing "Detecting and Preventing Hallucinations in Large Vision Language Models"

50 / 123 papers shown

Title
A Unified Hallucination Mitigation Framework for Large Vision-Language Models Yue Chang Liqiang Jing Xiaopeng Zhang Yue Zhang VLM MLLM 68 2 0 24 Sep 2024
MediConfusion: Can you trust your AI radiologist? Probing the reliability of multimodal medical foundation models Mohammad Shahab Sepehri Zalan Fabian Maryam Soltanolkotabi Mahdi Soltanolkotabi MedIm 48 3 0 23 Sep 2024
Multi-Modal Generative AI: Multi-modal LLM, Diffusion and Beyond Hong Chen Xin Wang Yuwei Zhou Bin Huang Yipeng Zhang Wei Feng Houlun Chen Zeyang Zhang Siao Tang Wenwu Zhu DiffM 55 7 0 23 Sep 2024
A Survey on Multimodal Benchmarks: In the Era of Large AI Models Lin Li Guikun Chen Hanrong Shi Jun Xiao Long Chen 42 9 0 21 Sep 2024
Understanding Foundation Models: Are We Back in 1924? Alan F. Smeaton AI4CE 35 2 0 11 Sep 2024
Seeing Through Their Eyes: Evaluating Visual Perspective Taking in Vision Language Models Gracjan Góral Alicja Ziarko Michal Nauman Maciej Wołczyk LRM 33 1 0 02 Sep 2024
Understanding Multimodal Hallucination with Parameter-Free Representation Alignment Yueqian Wang Jianxin Liang Yuxuan Wang Huishuai Zhang Dongyan Zhao 46 1 0 02 Sep 2024
A Survey on Evaluation of Multimodal Large Language Models Jiaxing Huang Jingyi Zhang LM&MA ELM LRM 50 20 0 28 Aug 2024
CLIP-DPO: Vision-Language Models as a Source of Preference for Fixing Hallucinations in LVLMs Yassine Ouali Adrian Bulat Brais Martínez Georgios Tzimiropoulos VLM MLLM 32 18 0 19 Aug 2024
Visual Agents as Fast and Slow Thinkers Guangyan Sun Mingyu Jin Zhenting Wang Cheng-Long Wang Siqi Ma Qifan Wang Ying Nian Wu Ying Nian Wu Dongfang Liu Dongfang Liu LLMAG LRM 79 13 0 16 Aug 2024
Reference-free Hallucination Detection for Large Vision-Language Models Qing Li Chenyang Lyu Jiahui Geng Derui Zhu Maxim Panov Fakhri Karray 26 6 0 11 Aug 2024
Surgical-VQLA++: Adversarial Contrastive Learning for Calibrated Robust Visual Question-Localized Answering in Robotic Surgery Long Bai Guankun Wang Mobarakol Islam Lalithkumar Seenivasan An-Chi Wang Hongliang Ren 54 13 0 09 Aug 2024
Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models Fushuo Huo Wenchao Xu Zhong Zhang Yining Qi Zhicheng Chen Peilin Zhao VLM MLLM 66 19 0 04 Aug 2024
Piculet: Specialized Models-Guided Hallucination Decrease for MultiModal Large Language Models Afia Anjum Xiang Liu Zhaoxiang Liu Ning Wang Shiguo Lian VLM MLLM 41 0 0 02 Aug 2024
Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs Shiping Liu Kecheng Zheng Wei Chen MLLM 49 34 0 31 Jul 2024
Identity-Consistent Diffusion Network for Grading Knee Osteoarthritis Progression in Radiographic Imaging Wenhua Wu Kun Hu Wenxi Yue Wei Li Milena Simic ChangYang Li Wei Xiang Zhiyong Wang MedIm 19 0 0 31 Jul 2024
Interpreting and Mitigating Hallucination in MLLMs through Multi-agent Debate Zheng Lin Zhenxing Niu Zhibin Wang Yinghui Xu 39 4 0 30 Jul 2024
HaloQuest: A Visual Hallucination Dataset for Advancing Multimodal Reasoning Zhecan Wang Garrett Bingham Adams Wei Yu Quoc V. Le Thang Luong Golnaz Ghiasi MLLM LRM 45 9 0 22 Jul 2024
DOPRA: Decoding Over-accumulation Penalization and Re-allocation in Specific Weighting Layer Jinfeng Wei Xiaofeng Zhang 28 13 0 21 Jul 2024
BEAF: Observing BEfore-AFter Changes to Evaluate Hallucination in Vision-language Models Moon Ye-Bin Nam Hyeon-Woo Wonseok Choi Tae-Hyun Oh MLLM 51 6 0 18 Jul 2024
Reflective Instruction Tuning: Mitigating Hallucinations in Large Vision-Language Models Jinrui Zhang Teng Wang Haigang Zhang Ping Lu Feng Zheng MLLM LRM VLM 34 3 0 16 Jul 2024
Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey) K. Kenthapadi M. Sameki Ankur Taly HILM ELM AILaw 39 12 0 10 Jul 2024
Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness Khyathi Raghavi Chandu Linjie Li Anas Awadalla Ximing Lu Jae Sung Park Jack Hessel Lijuan Wang Yejin Choi 50 2 0 02 Jul 2024
MIA-Bench: Towards Better Instruction Following Evaluation of Multimodal LLMs Yusu Qian Hanrong Ye J. Fauconnier Peter Grasch Yinfei Yang Zhe Gan 108 13 0 01 Jul 2024
Evaluating and Analyzing Relationship Hallucinations in Large Vision-Language Models Mingrui Wu Jiayi Ji Oucheng Huang Jiale Li Yuhang Wu Xiaoshuai Sun Rongrong Ji 53 8 0 24 Jun 2024
Detecting and Evaluating Medical Hallucinations in Large Vision Language Models Jiawei Chen Dingkang Yang Tong Wu Yue Jiang Xiaolu Hou Mingcheng Li Shunli Wang Dongling Xiao Ke Li Lihua Zhang LM&MA VLM 42 18 0 14 Jun 2024
On Softmax Direct Preference Optimization for Recommendation Yuxin Chen Junfei Tan An Zhang Zhengyi Yang Leheng Sheng Enzhi Zhang Xiang Wang Tat-Seng Chua 34 23 0 13 Jun 2024
VLind-Bench: Measuring Language Priors in Large Vision-Language Models Kang-il Lee Minbeom Kim Seunghyun Yoon Minsung Kim Dongryeol Lee Hyukhun Koh Kyomin Jung CoGe VLM 92 5 0 13 Jun 2024
CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Models Peng Xia Ze Chen Juanxi Tian Yangrui Gong Ruibo Hou ... Jimeng Sun Zongyuan Ge Gang Li James Zou Huaxiu Yao MU VLM 69 31 0 10 Jun 2024
3D-GRAND: A Million-Scale Dataset for 3D-LLMs with Better Grounding and Less Hallucination Jianing Yang Xuweiyi Chen Nikhil Madaan Madhavan Iyengar Shengyi Qian David Fouhey Joyce Chai 3DV 78 11 0 07 Jun 2024
Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaboration Junyang Wang Haiyang Xu Haitao Jia Xi Zhang Ming Yan Weizhou Shen Ji Zhang Fei Huang Jitao Sang LM&Ro LLMAG 34 48 0 03 Jun 2024
Generative AI Voting: Fair Collective Choice is Resilient to LLM Biases and Inconsistencies Srijoni Majumdar Edith Elkind Evangelos Pournaras SyDa 55 1 0 31 May 2024
Visual Perception by Large Language Model's Weights Feipeng Ma Hongwei Xue Guangting Wang Yizhou Zhou Fengyun Rao Shilin Yan Yueyi Zhang Siying Wu Mike Zheng Shou Xiaoyan Sun VLM 25 5 0 30 May 2024
MetaToken: Detecting Hallucination in Image Descriptions by Meta Classification Laura Fieback Jakob Spiegelberg Hanno Gottschalk MLLM 65 5 0 29 May 2024
RITUAL: Random Image Transformations as a Universal Anti-hallucination Lever in LVLMs Sangmin Woo Jaehyuk Jang Donguk Kim Yubin Choi Changick Kim 42 1 0 28 May 2024
Don't Miss the Forest for the Trees: Attentional Vision Calibration for Large Vision Language Models Sangmin Woo Donguk Kim Jaehyuk Jang Yubin Choi Changick Kim 42 12 0 28 May 2024
Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputs Mustafa Shukor Matthieu Cord 71 5 0 26 May 2024
Embedding-Aligned Language Models Guy Tennenholtz Yinlam Chow Chih-Wei Hsu Lior Shani Ethan Liang Craig Boutilier AIFin 37 1 0 24 May 2024
Alleviating Hallucinations in Large Vision-Language Models through Hallucination-Induced Optimization Beitao Chen Xinyu Lyu Lianli Gao Jingkuan Song Hengtao Shen MLLM 58 10 0 24 May 2024
Visual Fact Checker: Enabling High-Fidelity Detailed Caption Generation Yunhao Ge Fangyin Wei Siddharth Gururani Nayeon Lee Xuan Li Huayu Chen CoGe DiffM 35 14 0 30 Apr 2024
Fake Artificial Intelligence Generated Contents (FAIGC): A Survey of Theories, Detection Methods, and Opportunities Xiaomin Yu Yezhaohui Wang Yanfang Chen Zhen Tao Dinghao Xi Shichao Song Simin Niu Zhiyu Li 67 8 0 25 Apr 2024
Exploring the Transferability of Visual Prompting for Multimodal Large Language Models Yichi Zhang Yinpeng Dong Siyuan Zhang Tianzan Min Hang Su Jun Zhu LRM VLM 52 5 0 17 Apr 2024
Direct Preference Optimization of Video Large Multimodal Models from Language Model Reward Ruohong Zhang Liangke Gui Zhiqing Sun Yihao Feng Keyang Xu ... Di Fu Chunyuan Li Alexander G. Hauptmann Yonatan Bisk Yiming Yang MLLM 56 60 0 01 Apr 2024
A Review of Multi-Modal Large Language and Vision Models Kilian Carolan Laura Fennelly Alan F. Smeaton VLM 22 23 0 28 Mar 2024
Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning Hao Shao Shengju Qian Han Xiao Guanglu Song Zhuofan Zong Letian Wang Yu Liu Hongsheng Li VGen LRM MLLM 66 38 0 25 Mar 2024
Mitigating Dialogue Hallucination for Large Vision Language Models via Adversarial Instruction Tuning Dongmin Park Zhaofang Qian Guangxing Han Ser-Nam Lim MLLM 43 0 0 15 Mar 2024
Debiasing Multimodal Large Language Models Yi-Fan Zhang Weichen Yu Qingsong Wen Xue Wang Zhang Zhang Liang Wang Rong Jin Tien-Ping Tan 45 4 0 08 Mar 2024
HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding Zhaorun Chen Zhuokai Zhao Hongyin Luo Huaxiu Yao Bo Li Jiawei Zhou MLLM 46 57 0 01 Mar 2024
ShapeLLM: Universal 3D Object Understanding for Embodied Interaction Zekun Qi Runpei Dong Shaochen Zhang Haoran Geng Chunrui Han Zheng Ge Li Yi Kaisheng Ma 41 52 0 27 Feb 2024
How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Prompts Yusu Qian Haotian Zhang Yinfei Yang Zhe Gan 91 26 0 20 Feb 2024