Title
GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training Renqiu Xia Mingxing Li Hancheng Ye Wenjie Wu Hongbin Zhou ... Zeang Sheng Botian Shi Tao Chen Junchi Yan Bo Zhang 201 10 0 16 Dec 2024
Olympus: A Universal Task Router for Computer Vision Tasks Yuanze Lin Yunsheng Li Dongdong Chen Weijian Xu Ronald Clark Philip Torr VLM ObjD 548 1 0 12 Dec 2024
Chimera: Improving Generalist Model with Domain-Specific Experts Tianshuo Peng Mingxing Li Hongbin Zhou Renqiu Xia Renrui Zhang ... Aojun Zhou Botian Shi Tao Chen Bo Zhang Xiangyu Yue 200 5 0 08 Dec 2024
EgoPlan-Bench2: A Benchmark for Multimodal Large Language Model Planning in Real-World Scenarios Lu Qiu Yuying Ge Yi Chen Yixiao Ge Ying Shan Xihui Liu LLMAG LRM 214 8 0 05 Dec 2024
Orthus: Autoregressive Interleaved Image-Text Generation with Modality-Specific Heads Siqi Kou Jiachun Jin Chang Liu Ye Ma Jian Jia Quan Chen Peng Jiang Zhijie Deng Zhijie Deng DiffM VGen VLM 255 12 0 28 Nov 2024
Is 'Right' Right? Enhancing Object Orientation Understanding in Multimodal Large Language Models through Egocentric Instruction Tuning Ji Hyeok Jung Eun Tae Kim S. Kim Joo Ho Lee Bumsoo Kim Buru Chang VLM 519 2 0 24 Nov 2024
MLAN: Language-Based Instruction Tuning Preserves and Transfers Knowledge in Multimodal Language Models Jianhong Tu Zhuohao Ni Nicholas Crispino Zihao Yu Michael Bendersky ... Ruoxi Jia Xin Liu Lingjuan Lyu Dawn Song Chenguang Wang VLM MLLM 103 0 0 15 Nov 2024
Infinity-MM: Scaling Multimodal Performance with Large-Scale and High-Quality Instruction Data Shuhao Gu Jialing Zhang Siyuan Zhou Kevin Yu Zhaohu Xing ... Yufeng Cui Xinlong Wang Yaoqi Liu Fangxiang Feng Guang Liu SyDa VLM MLLM 130 29 0 24 Oct 2024
JMMMU: A Japanese Massive Multi-discipline Multimodal Understanding Benchmark for Culture-aware Evaluation Shota Onohara Atsuyuki Miyai Yuki Imajuku Kazuki Egashira Jeonghun Baek Xiang Yue Graham Neubig Kiyoharu Aizawa OSLM 262 6 0 22 Oct 2024
LLaVA-KD: A Framework of Distilling Multimodal Large Language Models Y. Cai Jiangning Zhang Haoyang He Xinwei He Ao Tong Zhenye Gan Chengjie Wang Zhucun Xue Yong-Jin Liu X. Bai VLM 98 6 0 21 Oct 2024
NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samples Baiqi Li Zhiqiu Lin Wenxuan Peng Jean de Dieu Nyandwi Daniel Jiang Zixian Ma Simran Khanuja Ranjay Krishna Graham Neubig Deva Ramanan AAML CoGe VLM 241 31 0 18 Oct 2024
Can We Predict Performance of Large Models across Vision-Language Tasks? Qinyu Zhao Ming Xu Kartik Gupta Akshay Asthana Liang Zheng Stephen Gould 135 0 0 14 Oct 2024
ActiView: Evaluating Active Perception Ability for Multimodal Large Language Models Ziyue Wang Chi Chen Ziyue Wang Yurui Dong Yuanchi Zhang Yuzhuang Xu Xiaolong Wang Ziwei Sun Yang Liu LRM 116 3 0 07 Oct 2024
MM-R $^3$ : On (In-)Consistency of Vision-Language Models (VLMs) Shih-Han Chou Shivam Chandhok James J. Little Leonid Sigal 88 0 0 07 Oct 2024
EMMA: Efficient Visual Alignment in Multi-Modal LLMs Sara Ghazanfari Alexandre Araujo Prashanth Krishnamurthy Siddharth Garg Farshad Khorrami VLM 87 2 0 02 Oct 2024
The Labyrinth of Links: Navigating the Associative Maze of Multi-modal LLMs Hong Li Nanxi Li Yuanjie Chen Jianbin Zhu Qinlu Guo Cewu Lu Yong-Lu Li MLLM 111 1 0 02 Oct 2024
Q-Bench-Video: Benchmarking the Video Quality Understanding of LMMs Zicheng Zhang Ziheng Jia H. Wu Chunyi Li Zijian Chen ... Wei Sun Xiaohong Liu Xiongkuo Min Weisi Lin Guangtao Zhai 109 10 0 30 Sep 2024
Enhancing Long Video Understanding via Hierarchical Event-Based Memory Dingxin Cheng Mingda Li Jingyu Liu Yongxin Guo Bin Jiang Qingbin Liu Xi Chen Bo Zhao 92 4 0 10 Sep 2024
StimuVAR: Spatiotemporal Stimuli-aware Video Affective Reasoning with Multimodal Large Language Models Y. Guo Faizan Siddiqui Yang Zhao Rama Chellappa Shao-Yuan Lo LRM 111 5 0 31 Aug 2024
IAA: Inner-Adaptor Architecture Empowers Frozen Large Language Model with Multimodal Capabilities Bin Wang Chunyu Xie Dawei Leng Yuhui Yin MLLM 181 1 0 23 Aug 2024
ParGo: Bridging Vision-Language with Partial and Global Views An-Lan Wang Bin Shan Wei Shi Kun-Yu Lin Xiang Fei Guozhi Tang Lei Liao Jingqun Tang Can Huang Wei-Shi Zheng MLLM VLM 188 17 0 23 Aug 2024
Sycophancy in Vision-Language Models: A Systematic Analysis and an Inference-Time Mitigation Framework Yunpu Zhao Rui Zhang Junbin Xiao Changxin Ke Ruibo Hou Yifan Hao Qi Guo 70 4 0 21 Aug 2024
Attribution Analysis Meets Model Editing: Advancing Knowledge Correction in Vision Language Models with VisEdit Qizhou Chen Taolin Zhang Chengyu Wang Xiaofeng He Dakan Wang Tingting Liu KELM 180 4 0 19 Aug 2024
xGen-MM (BLIP-3): A Family of Open Large Multimodal Models Le Xue Manli Shu Anas Awadalla Jun Wang An Yan ... Zeyuan Chen Silvio Savarese Juan Carlos Niebles Caiming Xiong Ran Xu VLM 110 96 0 16 Aug 2024
Mitigating Multilingual Hallucination in Large Vision-Language Models Xiaoye Qu Mingyang Song Xiaoye Qu Jianfeng Dong Yu Cheng VLM LRM 95 2 0 01 Aug 2024
ControlMLLM: Training-Free Visual Prompt Learning for Multimodal Large Language Models Ming-Kuan Wu Xinyue Cai Jiayi Ji Jiale Li Oucheng Huang Gen Luo Hao Fei Xiaoshuai Sun Rongrong Ji MLLM 162 13 0 31 Jul 2024
LiteGPT: Large Vision-Language Model for Joint Chest X-ray Localization and Classification Task Khai-Nguyen Nguyen Ryan Zhang Ngoc Son Nguyen Tan-Hanh Pham Anh Dao Ba Hung Ngo Anh Totti Nguyen Truong-Son Hy MedIm LM&MA 97 4 0 16 Jul 2024
VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models Haodong Duan Junming Yang Junming Yang Xinyu Fang Lin Chen ... Yuhang Zang Pan Zhang Jiaqi Wang Dahua Lin Kai Chen LM&MA VLM 253 180 0 16 Jul 2024
Rethinking Visual Prompting for Multimodal Large Language Models with External Knowledge Yuanze Lin Yunsheng Li Dongdong Chen Weijian Xu Ronald Clark Philip Torr Lu Yuan LRM VLM 81 8 0 05 Jul 2024
MindBench: A Comprehensive Benchmark for Mind Map Structure Recognition and Analysis Lei Chen Feng Yan Yujie Zhong Shaoxiang Chen Zequn Jie Lin Ma 128 4 0 03 Jul 2024
MIA-Bench: Towards Better Instruction Following Evaluation of Multimodal LLMs Yusu Qian Hanrong Ye J. Fauconnier Peter Grasch Yinfei Yang Zhe Gan 256 18 0 01 Jul 2024
$GenderBias-\emph{VL}: Benchmarking Gender Bias in Vision Language Models via Counterfactual Probing$ GenderBias-\emph{VL}: Benchmarking Gender Bias in Vision Language Models via Counterfactual Probing Yisong Xiao Aishan Liu QianJia Cheng Zhenfei Yin Siyuan Liang Jiapeng Li Jing Shao Xianglong Liu Dacheng Tao 129 8 0 30 Jun 2024
Curriculum Learning with Quality-Driven Data Selection Biao Wu Fang Meng 124 2 0 27 Jun 2024
S3: A Simple Strong Sample-effective Multimodal Dialog System Elisei Rykov Egor Malkershin Alexander Panchenko 89 0 0 26 Jun 2024
Evaluating Visual and Cultural Interpretation: The K-Viscuit Benchmark with Human-VLM Collaboration Yujin Baek Yujin Baek J. Kim Yu-Jung Heo Du-Seong Chang Jaegul Choo 126 6 0 24 Jun 2024
Multimodal Needle in a Haystack: Benchmarking Long-Context Capability of Multimodal Large Language Models Hengyi Wang Haizhou Shi Shiwei Tan Weiyi Qin Wenyuan Wang Tunyu Zhang A. Nambi T. Ganu Hao Wang 141 22 0 17 Jun 2024
MMFakeBench: A Mixed-Source Multimodal Misinformation Detection Benchmark for LVLMs Xuannan Liu Zekun Li Peipei Li Shuhan Xia Xing Cui Linzhi Huang Huaibo Huang Weihong Deng Zhaofeng He 143 23 0 13 Jun 2024
A-Bench: Are LMMs Masters at Evaluating AI-generated Images? Zicheng Zhang H. Wu Chunyi Li Yingjie Zhou Wei Sun Xiongkuo Min Zijian Chen Xiaohong Liu Weisi Lin Guangtao Zhai EGVM 150 18 0 05 Jun 2024
VoCoT: Unleashing Visually Grounded Multi-Step Reasoning in Large Multi-Modal Models Zejun Li Ruipu Luo Jiwen Zhang Minghui Qiu Zhongyu Wei Zhongyu Wei LRM MLLM 185 17 0 27 May 2024
Enhancing Visual-Language Modality Alignment in Large Vision Language Models via Self-Improvement Xiyao Wang Jiuhai Chen Zhaoyang Wang Yuhang Zhou Yiyang Zhou ... Dinesh Manocha Tom Goldstein Parminder Bhatia Furong Huang Cao Xiao 205 38 0 24 May 2024
A Survey on Vision-Language-Action Models for Embodied AI Yueen Ma Zixing Song Yuzheng Zhuang Jianye Hao Irwin King LM&Ro 337 54 0 23 May 2024
Efficient Multimodal Large Language Models: A Survey Yizhang Jin Jian Li Yexin Liu Tianjun Gu Kai Wu ... Xin Tan Zhenye Gan Yabiao Wang Chengjie Wang Lizhuang Ma LRM 122 58 0 17 May 2024
TableVQA-Bench: A Visual Question Answering Benchmark on Multiple Table Domains Yoonsik Kim Moonbin Yim Ka Yeon Song LMTD 113 23 0 30 Apr 2024
Hallucination of Multimodal Large Language Models: A Survey Zechen Bai Pichao Wang Tianjun Xiao Tong He Zongbo Han Zheng Zhang Mike Zheng Shou VLM LRM 289 197 0 29 Apr 2024
How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites Zhe Chen Weiyun Wang Hao Tian Shenglong Ye Zhangwei Gao ... Tong Lu Dahua Lin Yu Qiao Jifeng Dai Wenhai Wang MLLM VLM 205 644 0 25 Apr 2024
Joint Visual and Text Prompting for Improved Object-Centric Perception with Multimodal Large Language Models Songtao Jiang Yan Zhang Chenyi Zhou Yeying Jin Yang Feng Jian Wu Zuozhu Liu LRM VLM 102 6 0 06 Apr 2024
Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation Yunhao Gou Kai Chen Zhili Liu Lanqing Hong Hang Xu Zhenguo Li Dit-Yan Yeung James T. Kwok Yu Zhang MLLM 125 52 0 14 Mar 2024
A Cognitive Evaluation Benchmark of Image Reasoning and Description for Large Vision-Language Models Xiujie Song Mengyue Wu Ke Zhu Chunhao Zhang Yanyi Chen LRM ELM 134 3 0 28 Feb 2024
Visual Hallucinations of Multi-modal Large Language Models Wen Huang Hongbin Liu Minxin Guo Neil Zhenqiang Gong MLLM VLM 118 35 0 22 Feb 2024
RJUA-MedDQA: A Multimodal Benchmark for Medical Document Question Answering and Clinical Reasoning Congyun Jin Ming Zhang Xiaowei Ma Yujiao Li Yingbo Wang ... Chenfei Chi Xiangguo Lv Fangzhou Li Wei Xue Yiran Huang LM&MA 79 3 0 19 Feb 2024