Analyzing and Mitigating Object Hallucination in Large Vision-Language Models

1 October 2023

Mohit Bansal

Papers citing "Analyzing and Mitigating Object Hallucination in Large Vision-Language Models"

42 / 142 papers shown

Title
Prescribing the Right Remedy: Mitigating Hallucinations in Large Vision-Language Models via Targeted Instruction Tuning Rui Hu Yahan Tu Jitao Sang MLLM 19 0 0 16 Apr 2024
FGAIF: Aligning Large Vision-Language Models with Fine-grained AI Feedback Liqiang Jing Xinya Du 79 17 0 07 Apr 2024
LVLM-Interpret: An Interpretability Tool for Large Vision-Language Models Gabriela Ben-Melech Stan Estelle Aflalo R. Y. Rohekar Anahita Bhiwandiwalla Shao-Yen Tseng Matthew Lyle Olson Yaniv Gurwicz Chenfei Wu Nan Duan Vasudev Lal 83 7 0 03 Apr 2024
Pensieve: Retrospect-then-Compare Mitigates Visual Hallucination Dingchen Yang Bowen Cao Guang Chen Changjun Jiang 56 7 0 21 Mar 2024
Multi-Modal Hallucination Control by Visual Information Grounding Alessandro Favero L. Zancato Matthew Trager Siddharth Choudhary Pramuditha Perera Alessandro Achille Ashwin Swaminathan Stefano Soatto MLLM 90 64 0 20 Mar 2024
Automated data processing and feature engineering for deep learning and big data applications: a survey A. Mumuni F. Mumuni TPM 51 51 0 18 Mar 2024
Mitigating Dialogue Hallucination for Large Vision Language Models via Adversarial Instruction Tuning Dongmin Park Zhaofang Qian Guangxing Han Ser-Nam Lim MLLM 48 0 0 15 Mar 2024
Debiasing Multimodal Large Language Models Yi-Fan Zhang Weichen Yu Qingsong Wen Xue Wang Zhang Zhang Liang Wang Rong Jin Tien-Ping Tan 58 4 0 08 Mar 2024
Effectiveness Assessment of Recent Large Vision-Language Models Yao Jiang Xinyu Yan Ge-Peng Ji Keren Fu Meijun Sun Huan Xiong Deng-Ping Fan Fahad Shahbaz Khan 49 14 0 07 Mar 2024
Quantity Matters: Towards Assessing and Mitigating Number Hallucination in Large Vision-Language Models Huixuan Zhang Junzhe Zhang Xiaojun Wan MLLM 47 0 0 03 Mar 2024
HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding Zhaorun Chen Zhuokai Zhao Hongyin Luo Huaxiu Yao Bo Li Jiawei Zhou MLLM 48 60 0 01 Mar 2024
IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decoding Lanyun Zhu Deyi Ji Tianrun Chen Peng Xu Jieping Ye Jun Liu MLLM 56 47 0 28 Feb 2024
ShapeLLM: Universal 3D Object Understanding for Embodied Interaction Zekun Qi Runpei Dong Shaochen Zhang Haoran Geng Chunrui Han Zheng Ge Li Yi Kaisheng Ma 49 52 0 27 Feb 2024
Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models Yixin Liu Kai Zhang Yuan Li Zhiling Yan Chujie Gao ... Yue Huang Hanchi Sun Jianfeng Gao Lifang He Lichao Sun VLM VGen EGVM 82 263 0 27 Feb 2024
How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Prompts Yusu Qian Haotian Zhang Yinfei Yang Zhe Gan 105 26 0 20 Feb 2024
Aligning Modalities in Vision Large Language Models via Preference Fine-tuning Yiyang Zhou Chenhang Cui Rafael Rafailov Chelsea Finn Huaxiu Yao VLM MLLM 43 91 0 18 Feb 2024
Unified Hallucination Detection for Multimodal Large Language Models Xiang Chen Chenxi Wang Yida Xue Ningyu Zhang Xiaoyan Yang Qian Li Yue Shen Lei Liang Jinjie Gu Huajun Chen HILM 38 38 0 05 Feb 2024
Position: What Can Large Language Models Tell Us about Time Series Analysis Ming Jin Yifan Zhang Wei Chen Kexin Zhang Keli Zhang Bin Yang Jindong Wang Shirui Pan Qingsong Wen AI4TS 39 16 0 05 Feb 2024
$Skip \n: A Simple Method to Reduce Hallucination in Large Vision-Language Models$ Skip \n: A Simple Method to Reduce Hallucination in Large Vision-Language Models Zongbo Han Zechen Bai Haiyang Mei Qianli Xu Changqing Zhang Mike Zheng Shou VLM 39 7 0 02 Feb 2024
A Survey on Hallucination in Large Vision-Language Models Hanchao Liu Wenyuan Xue Yifei Chen Dapeng Chen Xiutian Zhao Ke Wang Liping Hou Rong-Zhi Li Wei Peng LRM MLLM 35 117 0 01 Feb 2024
On the Audio Hallucinations in Large Audio-Video Language Models Taichi Nishimura Shota Nakada Masayoshi Kondo VLM 30 5 0 18 Jan 2024
GlitchBench: Can large multimodal models detect video game glitches? Mohammad Reza Taesiri Tianjun Feng Anh Totti Nguyen Cor-Paul Bezemer MLLM VLM LRM 48 10 0 08 Dec 2023
MLLMs-Augmented Visual-Language Representation Learning Yanqing Liu Kai Wang Wenqi Shao Ping Luo Yu Qiao Mike Zheng Shou Kaipeng Zhang Yang You VLM 29 11 0 30 Nov 2023
OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation Qidong Huang Xiao-wen Dong Pan Zhang Bin Wang Conghui He Jiaqi Wang Dahua Lin Weiming Zhang Neng H. Yu MLLM 50 173 0 29 Nov 2023
Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding Sicong Leng Hang Zhang Guanzheng Chen Xin Li Shijian Lu Chunyan Miao Li Bing VLM MLLM 100 204 0 28 Nov 2023
Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization Zhiyuan Zhao Bin Wang Linke Ouyang Xiao-wen Dong Jiaqi Wang Conghui He MLLM VLM 55 108 0 28 Nov 2023
How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMs Haoqin Tu Chenhang Cui Zijun Wang Yiyang Zhou Bingchen Zhao Junlin Han Wangchunshu Zhou Huaxiu Yao Cihang Xie MLLM 67 72 0 27 Nov 2023
Check, Locate, Rectify: A Training-Free Layout Calibration System for Text-to-Image Generation Biao Gong Siteng Huang Yutong Feng Shiwei Zhang Yuyuan Li Yu Liu DiffM 33 11 0 27 Nov 2023
AutoEval-Video: An Automatic Benchmark for Assessing Large Vision Language Models in Open-Ended Video Question Answering Xiuyuan Chen Yuan Lin Yuchen Zhang Weiran Huang ELM MLLM 31 26 0 25 Nov 2023
HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data Qifan Yu Juncheng Li Longhui Wei Liang Pang Wentao Ye Bosheng Qin Siliang Tang Qi Tian Yueting Zhuang MLLM VLM 51 68 0 22 Nov 2023
Multimodal Representation Learning by Alternating Unimodal Adaptation Xiaohui Zhang Jaehong Yoon Mohit Bansal Huaxiu Yao 48 22 0 17 Nov 2023
DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedback Yangyi Chen Karan Sikka Michael Cogswell Heng Ji Ajay Divakaran 48 59 0 16 Nov 2023
Evaluation and Analysis of Hallucination in Large Vision-Language Models Junyan Wang Yi Zhou Guohai Xu Pengcheng Shi Chenlin Zhao ... Mingshi Yan Ji Zhang Jihua Zhu Jitao Sang Haoyu Tang MLLM 37 65 0 29 Aug 2023
A Survey on Multimodal Large Language Models Shukang Yin Chaoyou Fu Sirui Zhao Ke Li Xing Sun Tong Xu Enhong Chen MLLM LRM 62 562 0 23 Jun 2023
How Language Model Hallucinations Can Snowball Muru Zhang Ofir Press William Merrill Alisa Liu Noah A. Smith HILM LRM 88 259 0 22 May 2023
mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality Qinghao Ye Haiyang Xu Guohai Xu Jiabo Ye Ming Yan ... Junfeng Tian Qiang Qi Ji Zhang Feiyan Huang Jingren Zhou VLM MLLM 215 910 0 27 Apr 2023
Exposing and Mitigating Spurious Correlations for Cross-Modal Retrieval Jae Myung Kim A. Sophia Koepke Cordelia Schmid Zeynep Akata 83 26 0 06 Apr 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 322 4,300 0 30 Jan 2023
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 450 8,699 0 28 Jan 2022
Let there be a clock on the beach: Reducing Object Hallucination in Image Captioning Ali Furkan Biten L. G. I. Bigorda Dimosthenis Karatzas 102 57 0 04 Oct 2021
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 323 1,090 0 17 Feb 2021
Bridging AIC and BIC: a new criterion for autoregression Jie Ding Vahid Tarokh Yuhong Yang 186 75 0 11 Aug 2015