Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training

14 October 2022

Papers citing "Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training"

50 / 58 papers shown

Title
Exploring Hallucination of Large Multimodal Models in Video Understanding: Benchmark, Analysis and Mitigation Hongcheng Gao Jiashu Qu Jingyi Tang Baolong Bi Y. Liu Hongyu Chen Li Liang Li Su Qingming Huang MLLM VLM LRM 85 3 0 25 Mar 2025
HalCECE: A Framework for Explainable Hallucination Detection through Conceptual Counterfactuals in Image Captioning Maria Lymperaiou Giorgos Filandrianos Angeliki Dimitriou Athanasios Voulodimos Giorgos Stamou MLLM 40 0 0 01 Mar 2025
Evaluating Hallucination in Large Vision-Language Models based on Context-Aware Object Similarities Shounak Datta Dhanasekar Sundararaman 39 1 0 28 Jan 2025
Visual Large Language Models for Generalized and Specialized Applications Yifan Li Zhixin Lai Wentao Bao Zhen Tan Anh Dao Kewei Sui Jiayi Shen Dong Liu Huan Liu Yu Kong VLM 88 11 0 06 Jan 2025
Can Large Audio-Language Models Truly Hear? Tackling Hallucinations with Multi-Task Assessment and Stepwise Audio Reasoning Chun-Yi Kuan Hung-yi Lee AuLLM LRM 68 1 0 03 Jan 2025
Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey Yunkai Dang Kaichen Huang Jiahao Huo Yibo Yan S. Huang ... Kun Wang Yong Liu Jing Shao Hui Xiong Xuming Hu LRM 98 14 0 03 Dec 2024
VLN-Game: Vision-Language Equilibrium Search for Zero-Shot Semantic Navigation Bangguo Yu Yuzhen Liu Lei Han H. Kasaei Tingguang Li M. Cao LM&Ro 67 2 0 18 Nov 2024
See or Guess: Counterfactually Regularized Image Captioning Qian Cao Xu Chen Ruihua Song Xiting Wang Xinting Huang Yuchen Ren CML 29 1 0 29 Aug 2024
Piculet: Specialized Models-Guided Hallucination Decrease for MultiModal Large Language Models Afia Anjum Xiang Liu Zhaoxiang Liu Kai Wang Shiguo Lian VLM MLLM 41 0 0 02 Aug 2024
HaloQuest: A Visual Hallucination Dataset for Advancing Multimodal Reasoning Zhecan Wang Garrett Bingham Adams Wei Yu Quoc V. Le Thang Luong Golnaz Ghiasi MLLM LRM 37 9 0 22 Jul 2024
Multi-Object Hallucination in Vision-Language Models Xuweiyi Chen Ziqiao Ma Xuejun Zhang Sihan Xu Shengyi Qian Jianing Yang David Fouhey Joyce Chai 47 15 0 08 Jul 2024
Does Object Grounding Really Reduce Hallucination of Large Vision-Language Models? Gregor Geigle Radu Timofte Goran Glavas 35 0 0 20 Jun 2024
VLind-Bench: Measuring Language Priors in Large Vision-Language Models Kang-il Lee Minbeom Kim Seunghyun Yoon Minsung Kim Dongryeol Lee Hyukhun Koh Kyomin Jung CoGe VLM 86 5 0 13 Jun 2024
Understanding Sounds, Missing the Questions: The Challenge of Object Hallucination in Large Audio-Language Models Chun-Yi Kuan Wei-Ping Huang Hung-yi Lee AuLLM 29 5 0 12 Jun 2024
3D-GRAND: A Million-Scale Dataset for 3D-LLMs with Better Grounding and Less Hallucination Jianing Yang Xuweiyi Chen Nikhil Madaan Madhavan Iyengar Shengyi Qian David Fouhey Joyce Chai 3DV 70 11 0 07 Jun 2024
MetaToken: Detecting Hallucination in Image Descriptions by Meta Classification Laura Fieback Jakob Spiegelberg Hanno Gottschalk MLLM 57 5 0 29 May 2024
Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputs Mustafa Shukor Matthieu Cord 66 5 0 26 May 2024
ALOHa: A New Measure for Hallucination in Captioning Models Suzanne Petryk David M. Chan Anish Kachinthaya Haodi Zou John F. Canny Joseph E. Gonzalez Trevor Darrell HILM 31 11 0 03 Apr 2024
Visual Hallucination: Definition, Quantification, and Prescriptive Remediations Anku Rani Vipula Rawte Harshad Sharma Neeraj Anand Krishnav Rajbangshi Amit P. Sheth Amitava Das MLLM 56 6 0 26 Mar 2024
ChatGPT Incorrectness Detection in Software Reviews M. Tanzil Junaed Younus Khan Gias Uddin 19 4 0 25 Mar 2024
Hallucination Detection in Foundation Models for Decision-Making: A Flexible Definition and Review of the State of the Art Neeloy Chakraborty Melkior Ornik Katherine Driggs-Campbell LRM 57 9 0 25 Mar 2024
HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding Zhaorun Chen Zhuokai Zhao Hongyin Luo Huaxiu Yao Bo Li Jiawei Zhou MLLM 46 57 0 01 Mar 2024
ShapeLLM: Universal 3D Object Understanding for Embodied Interaction Zekun Qi Runpei Dong Shaochen Zhang Haoran Geng Chunrui Han Zheng Ge Li Yi Kaisheng Ma 41 51 0 27 Feb 2024
Probing Multimodal Large Language Models for Global and Local Semantic Representations Mingxu Tao Quzhe Huang Kun Xu Liwei Chen Yansong Feng Dongyan Zhao 19 5 0 27 Feb 2024
GROUNDHOG: Grounding Large Language Models to Holistic Segmentation Yichi Zhang Ziqiao Ma Xiaofeng Gao Suhaila Shakiah Qiaozi Gao Joyce Chai MLLM VLM 40 39 0 26 Feb 2024
Mitigating Object Hallucination in Large Vision-Language Models via Classifier-Free Guidance Linxi Zhao Yihe Deng Weitong Zhang Quanquan Gu MLLM 22 30 0 13 Feb 2024
A Survey on Hallucination in Large Vision-Language Models Hanchao Liu Wenyuan Xue Yifei Chen Dapeng Chen Xiutian Zhao Ke Wang Liping Hou Rong-Zhi Li Wei Peng LRM MLLM 26 112 0 01 Feb 2024
RePLan: Robotic Replanning with Perception and Language Models Marta Skreta Zihan Zhou Jia Lin Yuan Kourosh Darvish Alán Aspuru-Guzik Animesh Garg LM&Ro LRM 37 26 0 08 Jan 2024
Mitigating Open-Vocabulary Caption Hallucinations Assaf Ben-Kish Moran Yanuka Morris Alper Raja Giryes Hadar Averbuch-Elor MLLM VLM 18 6 0 06 Dec 2023
Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewrites Lei Wang Jiabang He Shenshen Li Ning Liu Ee-Peng Lim MLLM 27 38 0 04 Dec 2023
DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedback Yangyi Chen Karan Sikka Michael Cogswell Heng Ji Ajay Divakaran 24 58 0 16 Nov 2023
Trustworthy Large Models in Vision: A Survey Ziyan Guo Li Xu Jun Liu MU 58 0 0 16 Nov 2023
ROME: Evaluating Pre-trained Vision-Language Models on Reasoning beyond Visual Common Sense Kankan Zhou Eason Lai Wei Bin Au Yeong K. Mouratidis Jing Jiang ReLM LRM VLM 25 19 0 30 Oct 2023
Open Visual Knowledge Extraction via Relation-Oriented Multimodality Model Prompting Hejie Cui Xinyu Fang Zihan Zhang Ran Xu Xuan Kan Xin Liu Yue Yu Manling Li Yangqiu Song Carl Yang VLM 15 4 0 28 Oct 2023
Hallucination Detection for Grounded Instruction Generation Lingjun Zhao Khanh Nguyen Hal Daumé HILM 33 7 0 23 Oct 2023
Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learning Mustafa Shukor Alexandre Ramé Corentin Dancette Matthieu Cord LRM MLLM 38 20 0 01 Oct 2023
DreamLLM: Synergistic Multimodal Comprehension and Creation Runpei Dong Chunrui Han Yuang Peng Zekun Qi Zheng Ge ... Hao-Ran Wei Xiangwen Kong Xiangyu Zhang Kaisheng Ma Li Yi MLLM 34 170 0 20 Sep 2023
A Survey of Hallucination in Large Foundation Models Vipula Rawte A. Sheth Amitava Das HILM LRM 23 344 0 12 Sep 2023
Parameter Efficient Audio Captioning With Faithful Guidance Using Audio-text Shared Latent Representation A. Sridhar Yinyi Guo Erik M. Visser Rehana Mahfuz 29 5 0 06 Sep 2023
Pro-Cap: Leveraging a Frozen Vision-Language Model for Hateful Meme Detection Rui Cao Ming Shan Hee Adriel Kuek Wen-Haw Chong Roy Ka-Wei Lee Jing Jiang VLM MLLM 16 35 0 16 Aug 2023
Food-500 Cap: A Fine-Grained Food Caption Benchmark for Evaluating Vision-Language Models Zheng Ma Mianzhi Pan Wenhan Wu Ka Leong Cheng Jianbing Zhang Shujian Huang Jiajun Chen VLM CoGe 23 3 0 06 Aug 2023
Beyond Generic: Enhancing Image Captioning with Real-World Knowledge using Vision-Language Pre-Training Model Ka Leong Cheng Wenpo Song Zheng Ma Wenhao Zhu Zi-Yue Zhu Jianbing Zhang CLIP VLM 22 10 0 02 Aug 2023
UnIVAL: Unified Model for Image, Video, Audio and Language Tasks Mustafa Shukor Corentin Dancette Alexandre Ramé Matthieu Cord MoMe MLLM 58 42 0 30 Jul 2023
Visual Instruction Tuning with Polite Flamingo Delong Chen Jianfeng Liu Wenliang Dai Baoyuan Wang MLLM 26 42 0 03 Jul 2023
Let's Think Frame by Frame with VIP: A Video Infilling and Prediction Dataset for Evaluating Video Chain-of-Thought Vaishnavi Himakunthala Andy Ouyang Daniel Philip Rose Ryan He Alex Mei Yujie Lu Chinmay Sonar Michael Stephen Saxon William Yang Wang MLLM LRM 29 2 0 23 May 2023
Evaluating Object Hallucination in Large Vision-Language Models Yifan Li Yifan Du Kun Zhou Jinpeng Wang Wayne Xin Zhao Ji-Rong Wen MLLM LRM 90 691 0 17 May 2023
Simple Token-Level Confidence Improves Caption Correctness Suzanne Petryk Spencer Whitehead Joseph E. Gonzalez Trevor Darrell Anna Rohrbach Marcus Rohrbach 25 7 0 11 May 2023
A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity Yejin Bang Samuel Cahyawijaya Nayeon Lee Wenliang Dai Dan Su ... Tiezheng Yu Willy Chung Quyet V. Do Yan Xu Pascale Fung ReLM LRM 15 1,333 0 08 Feb 2023
One does not fit all! On the Complementarity of Vision Encoders for Vision and Language Tasks Gregor Geigle Chen Cecilia Liu Jonas Pfeiffer Iryna Gurevych VLM 26 1 0 12 Oct 2022
Survey of Hallucination in Natural Language Generation Ziwei Ji Nayeon Lee Rita Frieske Tiezheng Yu D. Su ... Delong Chen Wenliang Dai Ho Shu Chan Andrea Madotto Pascale Fung HILM LRM 40 2,232 0 08 Feb 2022