Title
Law of Vision Representation in MLLMs Shijia Yang Bohan Zhai Quanzeng You Jianbo Yuan Hongxia Yang Chenfeng Xu 42 9 0 29 Aug 2024
VLM4Bio: A Benchmark Dataset to Evaluate Pretrained Vision-Language Models for Trait Discovery from Biological Images M. Maruf Arka Daw Kazi Sajeed Mehrab Harish Babu Manogaran Abhilash Neog ... Wei-Lun Chao Charles V. Stewart T. Berger-Wolf Wasila Dahdul Anuj Karpatne CoGe 45 3 0 28 Aug 2024
A Survey on Evaluation of Multimodal Large Language Models Jiaxing Huang Jingyi Zhang LM&MA ELM LRM 52 20 0 28 Aug 2024
Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders Min Shi Fuxiao Liu Shihao Wang Shijia Liao Subhashree Radhakrishnan ... Andrew Tao Andrew Tao Zhiding Yu Guilin Liu Guilin Liu MLLM 38 54 0 28 Aug 2024
GlaLSTM: A Concurrent LSTM Stream Framework for Glaucoma Detection via Biomarker Mining Cheng Huang Weizheng Xie Jian Zhou Karanjit S Kooner Karanjit Kooner Yishen Liu 41 1 0 28 Aug 2024
Grounded Multi-Hop VideoQA in Long-Form Egocentric Videos Qirui Chen Shangzhe Di Weidi Xie 32 12 0 26 Aug 2024
Multimodal Contrastive In-Context Learning Yosuke Miyanishi Minh Le Nguyen 32 2 0 23 Aug 2024
IAA: Inner-Adaptor Architecture Empowers Frozen Large Language Model with Multimodal Capabilities Bin Wang Chunyu Xie Dawei Leng Yuhui Yin MLLM 54 1 0 23 Aug 2024
Building and better understanding vision-language models: insights and future directions Hugo Laurençon Andrés Marafioti Victor Sanh Léo Tronchon VLM 44 61 0 22 Aug 2024
EE-MLLM: A Data-Efficient and Compute-Efficient Multimodal Large Language Model Feipeng Ma Yizhou Zhou Hebei Li Zilong He Siying Wu Fengyun Rao Siying Wu Fengyun Rao Yueyi Zhang Xiaoyan Sun 41 3 0 21 Aug 2024
Open-FinLLMs: Open Multimodal Large Language Models for Financial Applications Qianqian Xie Dong Li Mengxi Xiao Zihao Jiang Ruoyu Xiang ... Benyou Wang Alejandro Lopez-Lira Qianqian Xie Sophia Ananiadou Junichi Tsujii AIFin AI4TS 38 15 0 20 Aug 2024
Harnessing Multimodal Large Language Models for Multimodal Sequential Recommendation Yuyang Ye Zhi Zheng Yishan Shen Tianshu Wang Hengruo Zhang Peijun Zhu Runlong Yu Kai Zhang Hui Xiong 42 8 0 19 Aug 2024
Quality Assessment in the Era of Large Models: A Survey Zicheng Zhang Yingjie Zhou Chunyi Li Baixuan Zhao Xiaohong Liu Guangtao Zhai 53 10 0 17 Aug 2024
xGen-MM (BLIP-3): A Family of Open Large Multimodal Models Le Xue Manli Shu Anas Awadalla Jun Wang An Yan ... Zeyuan Chen Silvio Savarese Juan Carlos Niebles Caiming Xiong Ran Xu VLM 44 91 0 16 Aug 2024
Level Up Your Tutorials: VLMs for Game Tutorials Quality Assessment Daniele Rege Cambrin Gabriele Scaffidi Militone Luca Colomba Giovanni Malnati D. Apiletti Paolo Garza 36 0 0 15 Aug 2024
CROME: Cross-Modal Adapters for Efficient Multimodal LLM Sayna Ebrahimi Sercan Ö. Arik Tejas Nama Tomas Pfister 49 1 0 13 Aug 2024
VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agents Xiao-Yang Liu Tianjie Zhang Yu Gu Iat Long Iong Yifan Xu ... Zhengxiao Du Chan Hee Song Yu Su Yuxiao Dong Jie Tang VLM LLMAG 55 22 0 12 Aug 2024
VITA: Towards Open-Source Interactive Omni Multimodal LLM Chaoyou Fu Haojia Lin Zuwei Long Yunhang Shen Meng Zhao ... Ran He Rongrong Ji Yunsheng Wu Caifeng Shan Xing Sun MLLM 47 81 0 09 Aug 2024
Openstory++: A Large-scale Dataset and Benchmark for Instance-aware Open-domain Visual Storytelling Zilyu Ye Yu Lei Ruotian Peng Jinjin Cao Zhiyang Chen ... Mingyuan Zhou Xiaoqian Shen Mohamed Elhoseiny Nan Zhuang Guo-Jun Qi VGen VLM 40 1 0 07 Aug 2024
LLaVA-OneVision: Easy Visual Task Transfer Bo Li Yuanhan Zhang Dong Guo Renrui Zhang Feng Li Hao Zhang Kaichen Zhang Yanwei Li Ziwei Liu Chunyuan Li MLLM SyDa VLM 58 578 0 06 Aug 2024
MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models Fanqing Meng Jun Wang Chuanhao Li Quanfeng Lu Hao Tian ... Jifeng Dai Ping Luo Ping Luo Kaipeng Zhang Wenqi Shao VLM 60 18 0 05 Aug 2024
Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretraining Dongyang Liu Shitian Zhao Le Zhuo Weifeng Lin Ping Luo Xinyue Li Qi Qin Yu Qiao Hongsheng Li Peng Gao MLLM 76 48 0 05 Aug 2024
Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models Fushuo Huo Wenchao Xu Zhong Zhang Yining Qi Zhicheng Chen Peilin Zhao VLM MLLM 68 19 0 04 Aug 2024
Dissecting Dissonance: Benchmarking Large Multimodal Models Against Self-Contradictory Instructions Jin Gao Lei Gan Yuankai Li Yixin Ye Dequan Wang 29 3 0 02 Aug 2024
MM-Vet v2: A Challenging Benchmark to Evaluate Large Multimodal Models for Integrated Capabilities Weihao Yu Zhengyuan Yang Linfeng Ren Linjie Li Jianfeng Wang K. Lin Chung-Ching Lin Zicheng Liu Lijuan Wang Xinchao Wang VLM MLLM 42 17 0 01 Aug 2024
Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Survey Atsuyuki Miyai Jingkang Yang Jingyang Zhang Yifei Ming Sisir Dhakal ... Yixuan Li Hai "Helen" Li Ziwei Liu Toshihiko Yamasaki Kiyoharu Aizawa 46 10 0 31 Jul 2024
$VILA^2$ : VILA Augmented VILA Yunhao Fang Ligeng Zhu Yao Lu Yan Wang Pavlo Molchanov Jang Hyun Cho Marco Pavone Song Han Hongxu Yin VLM 47 7 0 24 Jul 2024
Multi-label Cluster Discrimination for Visual Representation Learning Xiang An Kaicheng Yang Xiangzi Dai Ziyong Feng Jiankang Deng VLM 45 6 0 24 Jul 2024
MLLM-CompBench: A Comparative Reasoning Benchmark for Multimodal LLMs Jihyung Kil Zheda Mai Justin Lee Zihe Wang Kerrie Cheng Lemeng Wang Ye Liu A. Chowdhury Wei-Lun Chao CoGe VLM 38 14 0 23 Jul 2024
INF-LLaVA: Dual-perspective Perception for High-Resolution Multimodal Large Language Model Yiwei Ma Zhibin Wang Xiaoshuai Sun Weihuang Lin Qiang-feng Zhou Jiayi Ji Rongrong Ji MLLM VLM 57 1 0 23 Jul 2024
MMInstruct: A High-Quality Multi-Modal Instruction Tuning Dataset with Extensive Diversity Yangzhou Liu Yue Cao Zhangwei Gao Weiyun Wang Zhe Chen ... Lewei Lu Xizhou Zhu Tong Lu Yu Qiao Jifeng Dai VLM MLLM 57 24 0 22 Jul 2024
Accelerating Pre-training of Multimodal LLMs via Chain-of-Sight Ziyuan Huang Kaixiang Ji Biao Gong Zhiwu Qing Qinglong Zhang Kecheng Zheng Jian Wang Jingdong Chen Ming Yang LRM 42 1 0 22 Jul 2024
VideoGameBunny: Towards vision assistants for video games Mohammad Reza Taesiri C. Bezemer VLM MLLM 33 2 0 21 Jul 2024
MMAU: A Holistic Benchmark of Agent Capabilities Across Diverse Domains Guoli Yin Haoping Bai Shuang Ma Feng Nan Yanchao Sun ... Xiaoming Wang Jiulong Shan Meng Cao Ruoming Pang Zirui Wang LLMAG ELM 52 3 0 18 Jul 2024
LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Models Kaichen Zhang Bo Li Peiyuan Zhang Fanyi Pu Joshua Adrian Cahyono ... Shuai Liu Yuanhan Zhang Jingkang Yang Chunyuan Li Ziwei Liu 97 76 0 17 Jul 2024
FIRE: A Dataset for Feedback Integration and Refinement Evaluation of Multimodal Models Pengxiang Li Zhi Gao Bofei Zhang Tao Yuan Yuwei Wu Mehrtash Harandi Yunde Jia Song-Chun Zhu Qing Li VLM MLLM 48 5 0 16 Jul 2024
VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models Haodong Duan Junming Yang Junming Yang Xinyu Fang Lin Chen ... Yuhang Zang Pan Zhang Jiaqi Wang Dahua Lin Kai Chen LM&MA VLM 39 119 0 16 Jul 2024
NTSEBENCH: Cognitive Reasoning Benchmark for Vision Language Models Pranshu Pandya Agney S Talwarr Vatsal Gupta Tushar Kataria Dan Roth Vivek Gupta LRM 67 2 0 15 Jul 2024
SoupLM: Model Integration in Large Language and Multi-Modal Models Yue Bai Zichen Zhang Jiasen Lu Yun Fu MoMe 35 1 0 11 Jul 2024
LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models Feng Li Renrui Zhang Hao Zhang Yuanhan Zhang Bo Li Wei Li Zejun Ma Chunyuan Li MLLM VLM 52 196 0 10 Jul 2024
Decompose and Compare Consistency: Measuring VLMs' Answer Reliability via Task-Decomposition Consistency Comparison Qian Yang Weixiang Yan Aishwarya Agrawal CoGe 31 4 0 10 Jul 2024
Multimodal Self-Instruct: Synthetic Abstract Image and Visual Reasoning Instruction Using Language Model Wenqi Zhang Zhenglin Cheng Yuanyu He Mengna Wang Yongliang Shen ... Guiyang Hou Mingqian He Yanna Ma Weiming Lu Yueting Zhuang SyDa 74 9 0 09 Jul 2024
A Single Transformer for Scalable Vision-Language Modeling Yangyi Chen Xingyao Wang Hao Peng Heng Ji LRM 46 15 0 08 Jul 2024
OmChat: A Recipe to Train Multimodal Language Models with Strong Long Context and Video Understanding Tiancheng Zhao Qianqian Zhang Kyusong Lee Peng Liu Lu Zhang Chunxin Fang Jiajia Liao Kelei Jiang Yibo Ma Ruochen Xu MLLM VLM 54 5 0 06 Jul 2024
MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation? Zhaorun Chen Yichao Du Zichen Wen Yiyang Zhou Chenhang Cui ... Jiawei Zhou Zhuokai Zhao Rafael Rafailov Chelsea Finn Huaxiu Yao EGVM MLLM 63 29 0 05 Jul 2024
On scalable oversight with weak LLMs judging strong LLMs Zachary Kenton Noah Y. Siegel János Kramár Jonah Brown-Cohen Samuel Albanie ... Rishabh Agarwal David Lindner Yunhao Tang Noah D. Goodman Rohin Shah ELM 43 31 0 05 Jul 2024
Unified Interpretation of Smoothing Methods for Negative Sampling Loss Functions in Knowledge Graph Embedding Xincan Feng Hidetaka Kamigaito Katsuhiko Hayashi Taro Watanabe 44 1 0 05 Jul 2024
Smart Vision-Language Reasoners Denisa Roberts Lucas Roberts VLM ReLM LRM 51 4 0 05 Jul 2024
A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations Md Tahmid Rahman Laskar Sawsan Alqahtani M Saiful Bari Mizanur Rahman Mohammad Abdullah Matin Khan ... Chee Wei Tan Md. Rizwan Parvez Enamul Hoque Chenyu You Jimmy Huang ELM ALM 31 28 0 04 Jul 2024
M5 -- A Diverse Benchmark to Assess the Performance of Large Multimodal Models Across Multilingual and Multicultural Vision-Language Tasks Florian Schneider Sunayana Sitaram VLM 45 7 0 04 Jul 2024