Evaluating Object Hallucination in Large Vision-Language Models

17 May 2023

Papers citing "Evaluating Object Hallucination in Large Vision-Language Models"

50 / 585 papers shown

Title
RoboMamba: Multimodal State Space Model for Efficient Robot Reasoning and Manipulation Jiaming Liu Mengzhen Liu Zhenyu Wang Lily Lee Kaichen Zhou Pengju An Senqiao Yang Renrui Zhang Yandong Guo Shanghang Zhang LM&Ro LRM Mamba 32 6 0 06 Jun 2024
DeepStack: Deeply Stacking Visual Tokens is Surprisingly Simple and Effective for LMMs Lingchen Meng Jianwei Yang Rui Tian Xiyang Dai Zuxuan Wu Jianfeng Gao Yu-Gang Jiang VLM 30 9 0 06 Jun 2024
CODE: Contrasting Self-generated Description to Combat Hallucination in Large Multi-modal Models Junho Kim Hyunjun Kim Yeonju Kim Yong Man Ro MLLM 55 10 0 04 Jun 2024
Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaboration Junyang Wang Haiyang Xu Haitao Jia Xi Zhang Ming Yan Weizhou Shen Ji Zhang Fei Huang Jitao Sang LM&Ro LLMAG 34 48 0 03 Jun 2024
Dragonfly: Multi-Resolution Zoom Supercharges Large Visual-Language Model Kezhen Chen Rahul Thapa Rahul Chalamala Ben Athiwaratkun Shuaiwen Leon Song James Zou VLM 58 0 0 03 Jun 2024
FuRL: Visual-Language Models as Fuzzy Rewards for Reinforcement Learning Yuwei Fu Haichao Zhang Di Wu Wei-ping Xu Benoit Boulet VLM 24 12 0 02 Jun 2024
DeCo: Decoupling Token Compression from Semantic Abstraction in Multimodal Large Language Models Linli Yao Lei Li Shuhuai Ren Lean Wang Yuanxin Liu Xu Sun Lu Hou 35 29 0 31 May 2024
Shotluck Holmes: A Family of Efficient Small-Scale Large Language Vision Models For Video Captioning and Summarization Richard Luo Austin Peng Adithya Vasudev Rishabh Jain 34 2 0 31 May 2024
Visual Perception by Large Language Model's Weights Feipeng Ma Hongwei Xue Guangting Wang Yizhou Zhou Fengyun Rao Shilin Yan Yueyi Zhang Siying Wu Mike Zheng Shou Xiaoyan Sun VLM 25 5 0 30 May 2024
NoiseBoost: Alleviating Hallucination with Noise Perturbation for Multimodal Large Language Models Kai Wu Boyuan Jiang Zhengkai Jiang Qingdong He Donghao Luo Shengzhi Wang Qingwen Liu Chengjie Wang VLM MLLM 32 3 0 30 May 2024
Instruction-Guided Visual Masking Jinliang Zheng Jianxiong Li Si Cheng Yinan Zheng Jiaming Li Jihao Liu Yu Liu Jingjing Liu Xianyuan Zhan 53 5 0 30 May 2024
Dr-LLaVA: Visual Instruction Tuning with Symbolic Clinical Grounding Shenghuan Sun Gregory M. Goldgof Alexander Schubert Zhiqing Sun Thomas Hartvigsen A. Butte Ahmed Alaa LM&MA 42 4 0 29 May 2024
Benchmarking and Improving Detail Image Caption Hongyuan Dong Jiawen Li Bohong Wu Jiacong Wang Yuan Zhang Haoyuan Guo VLM MLLM 35 16 0 29 May 2024
MetaToken: Detecting Hallucination in Image Descriptions by Meta Classification Laura Fieback Jakob Spiegelberg Hanno Gottschalk MLLM 65 5 0 29 May 2024
Why are Visually-Grounded Language Models Bad at Image Classification? Yuhui Zhang Alyssa Unell Xiaohan Wang Dhruba Ghosh Yuchang Su Ludwig Schmidt Serena Yeung-Levy VLM 35 27 0 28 May 2024
Seeing the Image: Prioritizing Visual Correlation by Contrastive Alignment Xin Xiao Bohong Wu Jiacong Wang Chunyuan Li Xun Zhou Haoyuan Guo VLM 39 7 0 28 May 2024
RITUAL: Random Image Transformations as a Universal Anti-hallucination Lever in LVLMs Sangmin Woo Jaehyuk Jang Donguk Kim Yubin Choi Changick Kim 42 1 0 28 May 2024
Don't Miss the Forest for the Trees: Attentional Vision Calibration for Large Vision Language Models Sangmin Woo Donguk Kim Jaehyuk Jang Yubin Choi Changick Kim 42 12 0 28 May 2024
Visual Anchors Are Strong Information Aggregators For Multimodal Large Language Model Haogeng Liu Quanzeng You Xiaotian Han Yongfei Liu Huaibo Huang Ran He Hongxia Yang 33 2 0 28 May 2024
Matryoshka Multimodal Models Mu Cai Jianwei Yang Jianfeng Gao Yong Jae Lee VLM 53 25 0 27 May 2024
Privacy-Aware Visual Language Models Laurens Samson Nimrod Barazani S. Ghebreab Yukiyasu Asano PILM VLM 50 1 0 27 May 2024
VoCoT: Unleashing Visually Grounded Multi-Step Reasoning in Large Multi-Modal Models Zejun Li Ruipu Luo Jiwen Zhang Minghui Qiu Zhongyu Wei Zhongyu Wei LRM MLLM 62 8 0 27 May 2024
Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputs Mustafa Shukor Matthieu Cord 71 5 0 26 May 2024
ConvLLaVA: Hierarchical Backbones as Visual Encoder for Large Multimodal Models Chunjiang Ge Sijie Cheng Ziming Wang Jiale Yuan Yuan Gao Jun Song Shiji Song Gao Huang Bo Zheng MLLM VLM 36 17 0 24 May 2024
Meteor: Mamba-based Traversal of Rationale for Large Language and Vision Models Byung-Kwan Lee Chae Won Kim Beomchan Park Yonghyun Ro MLLM LRM 41 18 0 24 May 2024
Alleviating Hallucinations in Large Vision-Language Models through Hallucination-Induced Optimization Beitao Chen Xinyu Lyu Lianli Gao Jingkuan Song Hengtao Shen MLLM 58 10 0 24 May 2024
DEEM: Diffusion Models Serve as the Eyes of Large Language Models for Image Perception Run Luo Yunshui Li Longze Chen Wanwei He Ting-En Lin ... Zikai Song Xiaobo Xia Tongliang Liu Min Yang Binyuan Hui VLM DiffM 75 15 0 24 May 2024
How Culturally Aware are Vision-Language Models? Olena Burda-Lassen Aman Chadha Shashank Goswami Vinija Jain VLM 39 0 0 24 May 2024
M4U: Evaluating Multilingual Understanding and Reasoning for Large Multimodal Models Hongyu Wang Jiayu Xu Senwei Xie Ruiping Wang Jialin Li Zhaojie Xie Bin Zhang Chuyan Xiong Xilin Chen ELM VLM LRM 94 7 0 24 May 2024
Calibrated Self-Rewarding Vision Language Models Yiyang Zhou Zhiyuan Fan Dongjie Cheng Sihan Yang Zhaorun Chen Chenhang Cui Xiyao Wang Yun-Qing Li Linjun Zhang Huaxiu Yao VLM 79 26 0 23 May 2024
Explaining Multi-modal Large Language Models by Analyzing their Vision Perception Loris Giulivi Giacomo Boracchi 41 2 0 23 May 2024
AlignGPT: Multi-modal Large Language Models with Adaptive Alignment Capability Fei Zhao Taotian Pang Chunhui Li Zhen Wu Junjie Guo Shangyu Xing Xinyu Dai 55 7 0 23 May 2024
Maintaining Structural Integrity in Parameter Spaces for Parameter Efficient Fine-tuning Chongjie Si Xuehui Wang Xue Yang Zhengqin Xu Qingyun Li Jifeng Dai Yu Qiao Xiaokang Yang Wei Shen 31 8 0 23 May 2024
Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models Yongxin Guo Zhenglin Cheng Xiaoying Tang Tao R. Lin Tao Lin MoE 66 7 0 23 May 2024
Dense Connector for MLLMs Huanjin Yao Wenhao Wu Taojiannan Yang Yuxin Song Mengxi Zhang Haocheng Feng Yifan Sun Zhiheng Li Wanli Ouyang Jingdong Wang MLLM VLM 42 16 0 22 May 2024
CrossCheckGPT: Universal Hallucination Ranking for Multimodal Foundation Models Guangzhi Sun Potsawee Manakul Adian Liusie Kunat Pipatanakul Chao Zhang P. Woodland Mark J. F. Gales HILM MLLM 22 7 0 22 May 2024
C3L: Content Correlated Vision-Language Instruction Tuning Data Generation via Contrastive Learning Ji Ma Wei Suo Peng Wang Yanning Zhang VLM 48 0 0 21 May 2024
Single Image Unlearning: Efficient Machine Unlearning in Multimodal Large Language Models Jiaqi Li Qianshan Wei Chuanyi Zhang Guilin Qi Miaozeng Du Yongrui Chen Sheng Bi Fan Liu VLM MU 77 12 0 21 May 2024
Imp: Highly Capable Large Multimodal Models for Mobile Devices Zhenwei Shao Zhou Yu Jun Yu Xuecheng Ouyang Lihao Zheng Zhenbiao Gai Mingyang Wang Jiajun Ding 23 10 0 20 May 2024
TinyLLaVA Factory: A Modularized Codebase for Small-scale Large Multimodal Models Junlong Jia Ying Hu Xi Weng Yiming Shi Miao Li ... Baichuan Zhou Ziyu Liu Jie Luo Lei Huang Ji Wu 34 9 0 20 May 2024
Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts Yunxin Li Shenyuan Jiang Baotian Hu Longyue Wang Wanqi Zhong Wenhan Luo Lin Ma Min-Ling Zhang MoE 46 30 0 18 May 2024
Automated Multi-level Preference for MLLMs Mengxi Zhang Wenhao Wu Yu Lu Yuxin Song Kang Rong ... Jianbo Zhao Fanglong Liu Yifan Sun Haocheng Feng Jingdong Wang MLLM 72 10 0 18 May 2024
Efficient Multimodal Large Language Models: A Survey Yizhang Jin Jian Li Yexin Liu Tianjun Gu Kai Wu ... Xin Tan Zhenye Gan Yabiao Wang Chengjie Wang Lizhuang Ma LRM 47 45 0 17 May 2024
Libra: Building Decoupled Vision System on Large Language Models Yifan Xu Xiaoshan Yang Y. Song Changsheng Xu MLLM VLM 43 6 0 16 May 2024
Xmodel-VLM: A Simple Baseline for Multimodal Vision Language Model Wanting Xu Yang Liu Langping He Xucheng Huang Ling Jiang VLM MLLM 43 2 0 15 May 2024
Navigating LLM Ethics: Advancements, Challenges, and Future Directions Junfeng Jiao S. Afroogh Yiming Xu Connor Phillips AILaw 65 20 0 14 May 2024
CuMo: Scaling Multimodal LLM with Co-Upcycled Mixture-of-Experts Jiachen Li Xinyao Wang Sijie Zhu Chia-Wen Kuo Lu Xu Fan Chen Jitesh Jain Humphrey Shi Longyin Wen MLLM MoE 46 28 0 09 May 2024
THRONE: An Object-based Hallucination Benchmark for the Free-form Generations of Large Vision-Language Models Prannay Kaul Zhizhong Li Hao Yang Yonatan Dukler Ashwin Swaminathan C. Taylor Stefano Soatto HILM 63 16 0 08 May 2024
Language-Image Models with 3D Understanding Jang Hyun Cho Boris Ivanovic Yulong Cao Edward Schmerling Yue Wang ... Boyi Li Yurong You Philipp Krahenbuhl Yan Wang Marco Pavone LRM 42 17 0 06 May 2024
Pose Priors from Language Models Sanjay Subramanian Evonne Ng Lea Müller Dan Klein Shiry Ginosar Trevor Darrell 49 4 0 06 May 2024