Title
OmniBench: Towards The Future of Universal Omni-Language Models Yizhi Li Ge Zhang Yinghao Ma Ruibin Yuan Kang Zhu ... Zhaoxiang Zhang Zachary Liu Emmanouil Benetos Wenhao Huang Chenghua Lin LRM 184 19 0 23 Sep 2024
Can-Do! A Dataset and Neuro-Symbolic Grounded Framework for Embodied Planning with Large Multimodal Models Yew Ken Chia Qi Sun Lidong Bing Soujanya Poria LM&Ro 79 1 0 22 Sep 2024
A Survey on Multimodal Benchmarks: In the Era of Large AI Models Lin Li Guikun Chen Hanrong Shi Jun Xiao Long Chen 121 11 0 21 Sep 2024
JourneyBench: A Challenging One-Stop Vision-Language Understanding Benchmark of Generated Images Zhecan Wang Junzhang Liu Chia-Wei Tang Hani Alomari Anushka Sivakumar ... Haoxuan You A. Ishmam Kai-Wei Chang Shih-Fu Chang Chris Thomas CoGe VLM 173 2 0 19 Sep 2024
Oryx MLLM: On-Demand Spatial-Temporal Understanding at Arbitrary Resolution Zuyan Liu Yuhao Dong Ziwei Liu Winston Hu Jiwen Lu Yongming Rao ObjD 221 72 0 19 Sep 2024
NVLM: Open Frontier-Class Multimodal LLMs Wenliang Dai Nayeon Lee Wei Ping Zhuoling Yang Zihan Liu Jon Barker Tuomas Rintamaki Mohammad Shoeybi Bryan Catanzaro Ming-Yu Liu MLLM VLM LRM 123 73 0 17 Sep 2024
CAST: Cross-modal Alignment Similarity Test for Vision Language Models Gautier Dagan Olga Loginova Anil Batra CoGe 138 1 0 17 Sep 2024
TG-LLaVA: Text Guided LLaVA via Learnable Latent Embeddings Dawei Yan Pengcheng Li Yang Li Hao Chen Qingguo Chen Weihua Luo Wei Dong Kelu Yao Haokui Zhang Chunhua Shen 3DV VLM 90 5 0 15 Sep 2024
One missing piece in Vision and Language: A Survey on Comics Understanding Emanuele Vivoli Andrey Barsky Mohamed Ali Souibgui Artemis LLabres Marco Bertini Dimosthenis Karatzas 124 5 0 14 Sep 2024
SimulBench: Evaluating Language Models with Creative Simulation Tasks Qi Jia Xiang Yue Tianyu Zheng Jie Huang Bill Yuchen Lin LM&MA 66 5 0 11 Sep 2024
MathGLM-Vision: Solving Mathematical Problems with Multi-Modal Large Language Model Zhen Yang Jinhao Chen Zhengxiao Du Wenmeng Yu Weihan Wang Wenyi Hong Zhihuan Jiang Bin Xu Yuxiao Dong Jie Tang VLM LRM 84 11 0 10 Sep 2024
POINTS: Improving Your Vision-language Model with Affordable Strategies Yuan Liu Zhongyin Zhao Ziyuan Zhuang Le Tian Xiao Zhou Jie Zhou VLM 99 9 0 07 Sep 2024
Blocks as Probes: Dissecting Categorization Ability of Large Multimodal Models Bin Fu Qiyang Wan Jialin Li Ruiping Wang Xilin Chen 58 0 0 03 Sep 2024
Training on the Benchmark Is Not All You Need Shiwen Ni Xiangtao Kong Chengming Li Xiping Hu Ruifeng Xu Jia Zhu Min Yang 155 6 0 03 Sep 2024
Seeing Through Their Eyes: Evaluating Visual Perspective Taking in Vision Language Models Gracjan Góral Alicja Ziarko Michal Nauman Maciej Wołczyk LRM 85 2 0 02 Sep 2024
UrBench: A Comprehensive Benchmark for Evaluating Large Multimodal Models in Multi-View Urban Scenarios Baichuan Zhou Haote Yang Dairong Chen Junyan Ye Tianyi Bai Jinhua Yu Songyang Zhang Dahua Lin Conghui He Weijia Li VLM 179 7 0 30 Aug 2024
CogVLM2: Visual Language Models for Image and Video Understanding Wenyi Hong Weihan Wang Ming Ding Wenmeng Yu Qingsong Lv ... Debing Liu Bin Xu Juanzi Li Yuxiao Dong Jie Tang VLM MLLM 116 121 0 29 Aug 2024
Law of Vision Representation in MLLMs Shijia Yang Bohan Zhai Quanzeng You Jianbo Yuan Hongxia Yang Chenfeng Xu 157 12 0 29 Aug 2024
VLM4Bio: A Benchmark Dataset to Evaluate Pretrained Vision-Language Models for Trait Discovery from Biological Images M. Maruf Arka Daw Kazi Sajeed Mehrab Harish Babu Manogaran Abhilash Neog ... Wei-Lun Chao Charles V. Stewart T. Berger-Wolf Wasila Dahdul Anuj Karpatne CoGe 91 4 0 28 Aug 2024
A Survey on Evaluation of Multimodal Large Language Models Jiaxing Huang Jingyi Zhang LM&MA ELM LRM 120 26 0 28 Aug 2024
GlaLSTM: A Concurrent LSTM Stream Framework for Glaucoma Detection via Biomarker Mining Cheng Huang Weizheng Xie Jian Zhou Karanjit S Kooner Karanjit Kooner Yishen Liu 85 1 0 28 Aug 2024
Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders Min Shi Fuxiao Liu Shihao Wang Shijia Liao Subhashree Radhakrishnan ... Andrew Tao Andrew Tao Zhiding Yu Guilin Liu Guilin Liu MLLM 155 68 0 28 Aug 2024
Grounded Multi-Hop VideoQA in Long-Form Egocentric Videos Qirui Chen Shangzhe Di Weidi Xie 81 13 0 26 Aug 2024
Multimodal Contrastive In-Context Learning Yosuke Miyanishi Minh Le Nguyen 76 2 0 23 Aug 2024
IAA: Inner-Adaptor Architecture Empowers Frozen Large Language Model with Multimodal Capabilities Bin Wang Chunyu Xie Dawei Leng Yuhui Yin MLLM 179 1 0 23 Aug 2024
Building and better understanding vision-language models: insights and future directions Hugo Laurençon Andrés Marafioti Victor Sanh Léo Tronchon VLM 138 78 0 22 Aug 2024
EE-MLLM: A Data-Efficient and Compute-Efficient Multimodal Large Language Model Feipeng Ma Yizhou Zhou Hebei Li Zilong He Siying Wu Fengyun Rao Siying Wu Fengyun Rao Yueyi Zhang Xiaoyan Sun 191 8 0 21 Aug 2024
Open-FinLLMs: Open Multimodal Large Language Models for Financial Applications J. Huang Dong Li Mengxi Xiao Zihao Jiang Yuzhe Yang ... Benyou Wang Alejandro Lopez-Lira Qianqian Xie Sophia Ananiadou Junichi Tsujii AIFin AI4TS 79 25 0 20 Aug 2024
Harnessing Multimodal Large Language Models for Multimodal Sequential Recommendation Yuyang Ye Zhi Zheng Yishan Shen Tianshu Wang Hengruo Zhang Peijun Zhu Runlong Yu Kai Zhang Hui Xiong 102 12 0 19 Aug 2024
Quality Assessment in the Era of Large Models: A Survey Zicheng Zhang Yingjie Zhou Chunyi Li Baixuan Zhao Xiaohong Liu Guangtao Zhai 103 12 0 17 Aug 2024
xGen-MM (BLIP-3): A Family of Open Large Multimodal Models Le Xue Manli Shu Anas Awadalla Jun Wang An Yan ... Zeyuan Chen Silvio Savarese Juan Carlos Niebles Caiming Xiong Ran Xu VLM 108 96 0 16 Aug 2024
Level Up Your Tutorials: VLMs for Game Tutorials Quality Assessment Daniele Rege Cambrin Gabriele Scaffidi Militone Luca Colomba Giovanni Malnati D. Apiletti Paolo Garza 94 1 0 15 Aug 2024
CROME: Cross-Modal Adapters for Efficient Multimodal LLM Sayna Ebrahimi Sercan O. Arik Tejas Nama Tomas Pfister 79 1 0 13 Aug 2024
VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agents Xiao-Yang Liu Tianjie Zhang Yu Gu Iat Long Iong Yifan Xu ... Zhengxiao Du Chan Hee Song Yu Su Yuxiao Dong Jie Tang VLM LLMAG 126 38 0 12 Aug 2024
VITA: Towards Open-Source Interactive Omni Multimodal LLM Chaoyou Fu Haojia Lin Zuwei Long Yunhang Shen Meng Zhao ... Rongrong Ji Xing Sun Ran He Caifeng Shan Xing Sun MLLM 140 96 0 09 Aug 2024
Openstory++: A Large-scale Dataset and Benchmark for Instance-aware Open-domain Visual Storytelling Zilyu Ye Yu Lei Ruotian Peng Jinjin Cao Zhiyang Chen ... Mingyuan Zhou Xiaoqian Shen Mohamed Elhoseiny Nan Zhuang Guo-Jun Qi VGen VLM 74 1 0 07 Aug 2024
LLaVA-OneVision: Easy Visual Task Transfer Bo Li Yuanhan Zhang Dong Guo Renrui Zhang Feng Li Hao Zhang Kaichen Zhang Yanwei Li Ziwei Liu Chunyuan Li MLLM SyDa VLM 171 865 0 06 Aug 2024
MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models Fanqing Meng Jun Wang Chuanhao Li Quanfeng Lu Hao Tian ... Jifeng Dai Ping Luo Ping Luo Kaipeng Zhang Wenqi Shao VLM 100 26 0 05 Aug 2024
Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretraining Dongyang Liu Shitian Zhao Le Zhuo Weifeng Lin Ping Luo Xinyue Li Qi Qin Yu Qiao Hongsheng Li Peng Gao MLLM 168 59 0 05 Aug 2024
Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models Fushuo Huo Wenchao Xu Zhong Zhang Yining Qi Zhicheng Chen Peilin Zhao VLM MLLM 212 31 0 04 Aug 2024
Dissecting Dissonance: Benchmarking Large Multimodal Models Against Self-Contradictory Instructions Jin Gao Lei Gan Yuankai Li Yixin Ye Dequan Wang 73 3 0 02 Aug 2024
MM-Vet v2: A Challenging Benchmark to Evaluate Large Multimodal Models for Integrated Capabilities Weihao Yu Zhengyuan Yang Linfeng Ren Linjie Li Jianfeng Wang Kevin Qinghong Lin Chung-Ching Lin Zicheng Liu Lijuan Wang Xinchao Wang VLM MLLM 129 24 0 01 Aug 2024
Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Survey Atsuyuki Miyai Jingkang Yang Jingyang Zhang Yifei Ming Sisir Dhakal ... Yixuan Li Hai "Helen" Li Ziwei Liu Toshihiko Yamasaki Kiyoharu Aizawa 141 13 0 31 Jul 2024
$VILA^2$ : VILA Augmented VILA Yunhao Fang Ligeng Zhu Yao Lu Yan Wang Pavlo Molchanov Jang Hyun Cho Marco Pavone Song Han Hongxu Yin VLM 90 10 0 24 Jul 2024
Multi-label Cluster Discrimination for Visual Representation Learning Xiang An Kaicheng Yang Xiangzi Dai Ziyong Feng Jiankang Deng VLM 98 7 0 24 Jul 2024
MLLM-CompBench: A Comparative Reasoning Benchmark for Multimodal LLMs Jihyung Kil Zheda Mai Justin Lee Zihe Wang Kerrie Cheng Jingyan Bai Ye Liu A. Chowdhury Wei-Lun Chao CoGe VLM 145 19 0 23 Jul 2024
INF-LLaVA: Dual-perspective Perception for High-Resolution Multimodal Large Language Model Yiwei Ma Zhibin Wang Xiaoshuai Sun Weihuang Lin Qiang-feng Zhou Jiayi Ji Rongrong Ji MLLM VLM 110 2 0 23 Jul 2024
MMInstruct: A High-Quality Multi-Modal Instruction Tuning Dataset with Extensive Diversity Yangzhou Liu Yue Cao Zhangwei Gao Weiyun Wang Zhe Chen ... Lewei Lu Xizhou Zhu Tong Lu Yu Qiao Jifeng Dai VLM MLLM 116 29 0 22 Jul 2024
Accelerating Pre-training of Multimodal LLMs via Chain-of-Sight Ziyuan Huang Kaixiang Ji Biao Gong Zhiwu Qing Qinglong Zhang Kecheng Zheng Jian Wang Jingdong Chen Ming Yang LRM 75 2 0 22 Jul 2024
VideoGameBunny: Towards vision assistants for video games Mohammad Reza Taesiri Cor-Paul Bezemer VLM MLLM 81 2 0 21 Jul 2024