Analyzing and Mitigating Object Hallucination in Large Vision-Language Models

1 October 2023

Mohit Bansal

Papers citing "Analyzing and Mitigating Object Hallucination in Large Vision-Language Models"

50 / 142 papers shown

Title
The Curse of Multi-Modalities: Evaluating Hallucinations of Large Multimodal Models across Language, Visual, and Audio Sicong Leng Yun Xing Zesen Cheng Yang Zhou Hang Zhang Xin Li Deli Zhao Shijian Lu Chunyan Miao Lidong Bing 45 8 0 16 Oct 2024
MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation Chenxi Wang Xiang Chen N. Zhang Bozhong Tian Haoming Xu Shumin Deng Huajun Chen MLLM LRM 57 4 0 15 Oct 2024
From Pixels to Tokens: Revisiting Object Hallucinations in Large Vision-Language Models Yuying Shang Xinyi Zeng Yutao Zhu Xiao Yang Zhengwei Fang Jingyuan Zhang Jiawei Chen Zinan Liu Yu Tian VLM MLLM 221 1 0 09 Oct 2024
Intriguing Properties of Large Language and Vision Models Young-Jun Lee ByungSoo Ko Han-Gyu Kim Yechan Hwang Ho-Jin Choi LRM VLM 61 0 0 07 Oct 2024
TLDR: Token-Level Detective Reward Model for Large Vision Language Models Deqing Fu Tong Xiao Rui Wang Wang Zhu Pengchuan Zhang Guan Pang Robin Jia Lawrence Chen 68 6 0 07 Oct 2024
Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models Xin Zou Yizhou Wang Yibo Yan Yuanhuiyi Lyu Kening Zheng ... Junkai Chen Peijie Jiang Jiaheng Liu Chang Tang Xuming Hu 91 7 0 04 Oct 2024
Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations Nick Jiang Anish Kachinthaya Suzie Petryk Yossi Gandelsman VLM 41 17 0 03 Oct 2024
Embodied-RAG: General Non-parametric Embodied Memory for Retrieval and Generation Quanting Xie So Yeon Min Tianyi Zhang Kedi Xu Aarav Bajaj Ruslan Salakhutdinov Matthew Johnson-Roberson Yonatan Bisk Matthew Johnson-Roberson Yonatan Bisk LM&Ro 57 8 0 26 Sep 2024
FIHA: Autonomous Hallucination Evaluation in Vision-Language Models with Davidson Scene Graphs Bowen Yan Zhengsong Zhang Liqiang Jing Eftekhar Hossain Xinya Du 71 1 0 20 Sep 2024
Mitigating Hallucination in Visual-Language Models via Re-Balancing Contrastive Decoding Xiaoyu Liang Jiayuan Yu Lianrui Mu Jiedong Zhuang Jiaqi Hu Yuchen Yang Jiangnan Ye Lu Lu Jian Chen Haoji Hu VLM 45 2 0 10 Sep 2024
Understanding Multimodal Hallucination with Parameter-Free Representation Alignment Yueqian Wang Jianxin Liang Yuxuan Wang Huishuai Zhang Dongyan Zhao 62 1 0 02 Sep 2024
LLaVA-MoD: Making LLaVA Tiny via MoE Knowledge Distillation Fangxun Shu Yue Liao Le Zhuo Chenning Xu Guanghao Zhang ... Bolin Li Zhelun Yu Si Liu Hongsheng Li Hao Jiang VLM MoE 37 9 0 28 Aug 2024
Leveraging Hallucinations to Reduce Manual Prompt Dependency in Promptable Segmentation Jian Hu Jiayi Lin Junchi Yan Shaogang Gong VLM 46 7 0 27 Aug 2024
Visual Agents as Fast and Slow Thinkers Guangyan Sun Mingyu Jin Zhenting Wang Cheng-Long Wang Siqi Ma Qifan Wang Ying Nian Wu Ying Nian Wu Dongfang Liu Dongfang Liu LLMAG LRM 79 14 0 16 Aug 2024
Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models Fushuo Huo Wenchao Xu Zhong Zhang Yining Qi Zhicheng Chen Peilin Zhao VLM MLLM 70 20 0 04 Aug 2024
Hallu-PI: Evaluating Hallucination in Multi-modal Large Language Models within Perturbed Inputs Peng Ding Jingyu Wu Mark Girolami Dan Ma Xuezhi Cao Xunliang Cai Shi Chen T. J. Sullivan Shujian Huang AAML VLM MLLM 39 0 0 02 Aug 2024
Alleviating Hallucination in Large Vision-Language Models with Active Retrieval Augmentation Cephas Mpungu Qiyuan Chen Xiaoye Qu Jiashuo Sun G. Mapp VLM RALM LRM 46 16 0 01 Aug 2024
Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs Shiping Liu Kecheng Zheng Wei Chen MLLM 52 37 0 31 Jul 2024
Prompting Medical Large Vision-Language Models to Diagnose Pathologies by Visual Question Answering Danfeng Guo Sumitaka Honji LRM 87 0 0 31 Jul 2024
HaloQuest: A Visual Hallucination Dataset for Advancing Multimodal Reasoning Zhecan Wang Garrett Bingham Adams Wei Yu Quoc V. Le Thang Luong Golnaz Ghiasi MLLM LRM 56 9 0 22 Jul 2024
DOPRA: Decoding Over-accumulation Penalization and Re-allocation in Specific Weighting Layer Jinfeng Wei Xiaofeng Zhang 28 13 0 21 Jul 2024
BEAF: Observing BEfore-AFter Changes to Evaluate Hallucination in Vision-language Models Moon Ye-Bin Nam Hyeon-Woo Wonseok Choi Tae-Hyun Oh MLLM 51 6 0 18 Jul 2024
MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation? Zhaorun Chen Yichao Du Zichen Wen Yiyang Zhou Chenhang Cui ... Jiawei Zhou Zhuokai Zhao Rafael Rafailov Chelsea Finn Huaxiu Yao EGVM MLLM 70 29 0 05 Jul 2024
Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness Khyathi Raghavi Chandu Linjie Li Anas Awadalla Ximing Lu Jae Sung Park Jack Hessel Lijuan Wang Yejin Choi 58 2 0 02 Jul 2024
Evaluating and Analyzing Relationship Hallucinations in Large Vision-Language Models Mingrui Wu Jiayi Ji Oucheng Huang Jiale Li Yuhang Wu Xiaoshuai Sun Rongrong Ji 53 8 0 24 Jun 2024
Detecting and Evaluating Medical Hallucinations in Large Vision Language Models Jiawei Chen Dingkang Yang Tong Wu Yue Jiang Xiaolu Hou Mingcheng Li Shunli Wang Dongling Xiao Ke Li Lihua Zhang LM&MA VLM 44 20 0 14 Jun 2024
ImageNet3D: Towards General-Purpose Object-Level 3D Understanding Wufei Ma Guanning Zeng Guofeng Zhang Qihao Liu Letian Zhang Adam Kortylewski Yaoyao Liu Alan Yuille VLM 3DV 49 7 0 13 Jun 2024
VLind-Bench: Measuring Language Priors in Large Vision-Language Models Kang-il Lee Minbeom Kim Seunghyun Yoon Minsung Kim Dongryeol Lee Hyukhun Koh Kyomin Jung CoGe VLM 92 5 0 13 Jun 2024
Understanding Sounds, Missing the Questions: The Challenge of Object Hallucination in Large Audio-Language Models Chun-Yi Kuan Wei-Ping Huang Hung-yi Lee AuLLM 31 7 0 12 Jun 2024
CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Models Peng Xia Ze Chen Juanxi Tian Yangrui Gong Ruibo Hou ... Jimeng Sun Zongyuan Ge Gang Li James Zou Huaxiu Yao MU VLM 69 31 0 10 Jun 2024
3D-GRAND: A Million-Scale Dataset for 3D-LLMs with Better Grounding and Less Hallucination Jianing Yang Xuweiyi Chen Nikhil Madaan Madhavan Iyengar Shengyi Qian David Fouhey Joyce Chai 3DV 82 11 0 07 Jun 2024
A Survey of Language-Based Communication in Robotics William Hunt Sarvapali D. Ramchurn Mohammad D. Soorati LM&Ro 78 12 0 06 Jun 2024
CODE: Contrasting Self-generated Description to Combat Hallucination in Large Multi-modal Models Junho Kim Hyunjun Kim Yeonju Kim Yong Man Ro MLLM 60 10 0 04 Jun 2024
When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs Ryo Kamoi Yusen Zhang Nan Zhang Jiawei Han Rui Zhang LRM 50 62 0 03 Jun 2024
Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaboration Junyang Wang Haiyang Xu Haitao Jia Xi Zhang Ming Yan Weizhou Shen Ji Zhang Fei Huang Jitao Sang LM&Ro LLMAG 44 52 0 03 Jun 2024
MetaToken: Detecting Hallucination in Image Descriptions by Meta Classification Laura Fieback Jakob Spiegelberg Hanno Gottschalk MLLM 67 5 0 29 May 2024
RITUAL: Random Image Transformations as a Universal Anti-hallucination Lever in LVLMs Sangmin Woo Jaehyuk Jang Donguk Kim Yubin Choi Changick Kim 42 1 0 28 May 2024
Don't Miss the Forest for the Trees: Attentional Vision Calibration for Large Vision Language Models Sangmin Woo Donguk Kim Jaehyuk Jang Yubin Choi Changick Kim 47 12 0 28 May 2024
Think Before You Act: A Two-Stage Framework for Mitigating Gender Bias Towards Vision-Language Tasks Yunqi Zhang Songda Li Chunyuan Deng Luyi Wang Hui Zhao 48 0 0 27 May 2024
Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputs Mustafa Shukor Matthieu Cord 71 5 0 26 May 2024
Enhancing Visual-Language Modality Alignment in Large Vision Language Models via Self-Improvement Xiyao Wang Jiuhai Chen Zhaoyang Wang Yuhang Zhou Yiyang Zhou ... Dinesh Manocha Tom Goldstein Parminder Bhatia Furong Huang Cao Xiao 74 35 0 24 May 2024
Alleviating Hallucinations in Large Vision-Language Models through Hallucination-Induced Optimization Beitao Chen Xinyu Lyu Lianli Gao Jingkuan Song Hengtao Shen MLLM 58 10 0 24 May 2024
Calibrated Self-Rewarding Vision Language Models Yiyang Zhou Zhiyuan Fan Dongjie Cheng Sihan Yang Zhaorun Chen Chenhang Cui Xiyao Wang Yun Li Linjun Zhang Huaxiu Yao VLM 79 27 0 23 May 2024
CrossCheckGPT: Universal Hallucination Ranking for Multimodal Foundation Models Guangzhi Sun Potsawee Manakul Adian Liusie Kunat Pipatanakul Chao Zhang P. Woodland Mark Gales HILM MLLM 27 8 0 22 May 2024
THRONE: An Object-based Hallucination Benchmark for the Free-form Generations of Large Vision-Language Models Prannay Kaul Zhizhong Li Hao Yang Yonatan Dukler Ashwin Swaminathan C. Taylor Stefano Soatto HILM 65 16 0 08 May 2024
An Empty Room is All We Want: Automatic Defurnishing of Indoor Panoramas Mira Slavcheva Dave Gausebeck Kevin Chen David Buchhofer Azwad Sabik Chen Ma Sachal Dhillon Olaf Brandt Alan Dolhasz 34 5 0 06 May 2024
Overcoming LLM Challenges using RAG-Driven Precision in Coffee Leaf Disease Remediation Dr. Selva Kumar Mohammed Ajmal Khan Imadh Ajaz Banday Manikantha Gada Vibha Venkatesh 45 3 0 02 May 2024
CodeHalu: Investigating Code Hallucinations in LLMs via Execution-based Verification Yuchen Tian Weixiang Yan Qian Yang Xuandong Zhao Qian Chen Ziyang Luo Lei Ma Lei Ma Dawn Song 48 8 0 30 Apr 2024
Hallucination of Multimodal Large Language Models: A Survey Zechen Bai Pichao Wang Tianjun Xiao Tong He Zongbo Han Zheng Zhang Mike Zheng Shou VLM LRM 97 146 0 29 Apr 2024
Fake Artificial Intelligence Generated Contents (FAIGC): A Survey of Theories, Detection Methods, and Opportunities Xiaomin Yu Yezhaohui Wang Yanfang Chen Zhen Tao Dinghao Xi Shichao Song Pengnian Qi Zhiyu Li 77 9 0 25 Apr 2024