Title
OneLLM: One Framework to Align All Modalities with Language Jiaming Han Kaixiong Gong Yiyuan Zhang Jiaqi Wang Kaipeng Zhang Dahua Lin Yu Qiao Peng Gao Xiangyu Yue MLLM 140 121 0 10 Jan 2025
Sa2VA: Marrying SAM2 with LLaVA for Dense Grounded Understanding of Images and Videos Haobo Yuan Xianrui Li Tao Zhang Zilong Huang Shilin Xu S. Ji Yunhai Tong Lu Qi Jiashi Feng Ming-Hsuan Yang VLM 118 16 0 07 Jan 2025
FOLDER: Accelerating Multi-modal Large Language Models with Enhanced Performance Haicheng Wang Zhemeng Yu Gabriele Spadaro Chen Ju Victor Quétu Enzo Tartaglione Enzo Tartaglione VLM 321 5 0 05 Jan 2025
VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks Jiannan Wu Muyan Zhong Sen Xing Zeqiang Lai Zhaoyang Liu ... Lewei Lu Tong Lu Ping Luo Yu Qiao Jifeng Dai MLLM VLM LRM 204 51 0 03 Jan 2025
GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Models Zhangyang Qi Zhixiong Zhang Ye Fang Jiaqi Wang Hengshuang Zhao 135 9 0 02 Jan 2025
HoVLE: Unleashing the Power of Monolithic Vision-Language Models with Holistic Vision-Language Embedding Chenxin Tao Shiqian Su X. Zhu Chenyu Zhang Zhe Chen ... Wenhai Wang Lewei Lu Gao Huang Yu Qiao Jifeng Dai MLLM VLM 159 2 0 20 Dec 2024
FCMR: Robust Evaluation of Financial Cross-Modal Multi-Hop Reasoning Seunghee Kim Changhyeon Kim Taeuk Kim LRM 123 1 0 17 Dec 2024
Olympus: A Universal Task Router for Computer Vision Tasks Yuanze Lin Yunsheng Li Dongdong Chen Weijian Xu Ronald Clark Philip Torr VLM ObjD 412 0 0 12 Dec 2024
EgoPlan-Bench2: A Benchmark for Multimodal Large Language Model Planning in Real-World Scenarios Lu Qiu Yuying Ge Yi Chen Yixiao Ge Ying Shan Xihui Liu LLMAG LRM 138 5 0 05 Dec 2024
AdvDreamer Unveils: Are Vision-Language Models Truly Ready for Real-World 3D Variations? Shouwei Ruan Hanqin Liu Yao Huang Xiaoqi Wang Caixin Kang Hang Su Yinpeng Dong Xingxing Wei VGen 131 0 0 04 Dec 2024
OBI-Bench: Can LMMs Aid in Study of Ancient Script on Oracle Bones? Zhongfu Chen Tingzhu Chen Wenjun Zhang Guangtao Zhai 113 4 0 02 Dec 2024
ChatRex: Taming Multimodal LLM for Joint Perception and Understanding Qing Jiang Gen Luo Yuqin Yang Yuda Xiong Yihao Chen Zhaoyang Zeng Tianhe Ren Lei Zhang VLM LRM 134 8 0 27 Nov 2024
Evaluating Vision-Language Models as Evaluators in Path Planning Mohamed Aghzal Xiang Yue Erion Plaku Ziyu Yao LRM 127 1 0 27 Nov 2024
Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning Di Zhang Jingdi Lei Junxian Li Xunzhi Wang Yong Liu ... Steve Yang Jianbo Wu Peng Ye Wanli Ouyang Dongzhan Zhou OffRL LRM 131 7 0 27 Nov 2024
RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics Chan Hee Song Valts Blukis Jonathan Tremblay Stephen Tyree Yu-Chuan Su Stan Birchfield 125 12 0 25 Nov 2024
All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages Ashmal Vayani Dinura Dissanayake Hasindri Watawana Noor Ahsan Nevasini Sasikumar ... Monojit Choudhury Ivan Laptev Mubarak Shah Salman Khan Fahad A Khan 162 11 0 25 Nov 2024
MC-LLaVA: Multi-Concept Personalized Vision-Language Model Ruichuan An Sihan Yang Ming Lu Kai Zeng Yulin Luo ... Hao Liang Qi She Shanghang Zhang Wentao Zhang Wentao Zhang 132 8 0 18 Nov 2024
Both Text and Images Leaked! A Systematic Analysis of Multimodal LLM Data Contamination D. Song Sicheng Lai Shunian Chen Lichao Sun Benyou Wang 360 0 0 06 Nov 2024
Bongard in Wonderland: Visual Puzzles that Still Make AI Go Mad? Antonia Wüst Tim Nelson Tobiasch Lukas Helff Inga Ibs Wolfgang Stammer Devendra Singh Dhami Constantin Rothkopf Kristian Kersting CoGe ReLM VLM LRM 115 1 0 25 Oct 2024
Revealing and Reducing Gender Biases in Vision and Language Assistants (VLAs) Leander Girrbach Yiran Huang Stephan Alaniz Trevor Darrell Zeynep Akata VLM 70 2 0 25 Oct 2024
Infinity-MM: Scaling Multimodal Performance with Large-Scale and High-Quality Instruction Data Shuhao Gu Jialing Zhang Siyuan Zhou Kevin Yu Zhaohu Xing ... Yufeng Cui Xinlong Wang Yaoqi Liu Fangxiang Feng Guang Liu SyDa VLM MLLM 62 25 0 24 Oct 2024
Do Vision-Language Models Represent Space and How? Evaluating Spatial Frame of Reference Under Ambiguities Zheyuan Zhang Fengyuan Hu Jayjun Lee Freda Shi Parisa Kordjamshidi Joyce Chai Ziqiao Ma 97 12 0 22 Oct 2024
PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction Long Xing Qidong Huang Xiaoyi Dong Jiajie Lu Pan Zhang ... Yuhang Cao Zeang Sheng Jiaqi Wang Feng Wu Dahua Lin VLM 66 35 0 22 Oct 2024
NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samples Baiqi Li Zhiqiu Lin Wenxuan Peng Jean de Dieu Nyandwi Daniel Jiang Zixian Ma Simran Khanuja Ranjay Krishna Graham Neubig Deva Ramanan AAML CoGe VLM 123 27 0 18 Oct 2024
MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models Peng Xia Siwei Han Shi Qiu Yiyang Zhou Zhaoyang Wang ... Chenhang Cui Mingyu Ding Linjie Li Lijuan Wang Huaxiu Yao 80 11 0 14 Oct 2024
Can We Predict Performance of Large Models across Vision-Language Tasks? Qinyu Zhao Ming Xu Kartik Gupta Akshay Asthana Liang Zheng Stephen Gould 73 0 0 14 Oct 2024
3DArticCyclists: Generating Synthetic Articulated 8D Pose-Controllable Cyclist Data for Computer Vision Applications Eduardo R. Corral-Soto Yang Liu Tongtong Cao Y. Ren Liu Bingbing 79 5 0 14 Oct 2024
Dynamic Multimodal Evaluation with Flexible Complexity by Vision-Language Bootstrapping Yue Yang Shanghang Zhang Wenqi Shao Kaipeng Zhang Yi Bin Yu Wang Ping Luo 70 2 0 11 Oct 2024
MRAG-Bench: Vision-Centric Evaluation for Retrieval-Augmented Multimodal Models Wenbo Hu Jia-Chen Gu Zi-Yi Dou Mohsen Fayyaz Pan Lu Kai-Wei Chang Nanyun Peng VLM 85 6 0 10 Oct 2024
Mono-InternVL: Pushing the Boundaries of Monolithic Multimodal Large Language Models with Endogenous Visual Pre-training Gen Luo Xue Yang Wenhan Dou Zhaokai Wang Jifeng Dai Jifeng Dai Yu Qiao Xizhou Zhu VLM MLLM 91 26 0 10 Oct 2024
ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time Yi Ding Bolian Li Ruqi Zhang MLLM 84 11 0 09 Oct 2024
ActiView: Evaluating Active Perception Ability for Multimodal Large Language Models Ziyue Wang Chi Chen Ziyue Wang Yurui Dong Yuanchi Zhang Yuzhuang Xu Xiaolong Wang Ziwei Sun Yang Liu LRM 60 3 0 07 Oct 2024
Geometric Analysis of Reasoning Trajectories: A Phase Space Approach to Understanding Valid and Invalid Multi-Hop Reasoning in LLMs Javier Marin LRM 98 45 0 06 Oct 2024
Self-Correction is More than Refinement: A Learning Framework for Visual and Language Reasoning Tasks Jiayi He Hehai Lin Q. Wang Yi R. Fung Chenhui Xu ReLM LRM 133 7 0 05 Oct 2024
Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models Xin Zou Yizhou Wang Yibo Yan Yuanhuiyi Lyu Kening Zheng ... Junkai Chen Peijie Jiang Qingbin Liu Chang Tang Xuming Hu 104 8 0 04 Oct 2024
From Pixels to Tokens: Byte-Pair Encoding on Quantized Visual Modalities Wanpeng Zhang Zilong Xie Yicheng Feng Yijiang Li Xingrun Xing Sipeng Zheng Zongqing Lu MLLM 52 1 0 03 Oct 2024
EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotions Kai Chen Yunhao Gou Runhui Huang Zhili Liu Daxin Tan ... Qun Liu Jun Yao Lu Hou Hang Xu Hang Xu AuLLM MLLM VLM 100 25 0 26 Sep 2024
EventHallusion: Diagnosing Event Hallucinations in Video LLMs Jiacheng Zhang Yang Jiao Shaoxiang Chen Jingjing Chen Zhiyu Tan Hao Li Jingjing Chen MLLM 83 18 0 25 Sep 2024
OmniBench: Towards The Future of Universal Omni-Language Models Yizhi Li Ge Zhang Yinghao Ma Ruibin Yuan Kang Zhu ... Zhaoxiang Zhang Zachary Liu Emmanouil Benetos Wenhao Huang Chenghua Lin LRM 88 15 0 23 Sep 2024
JourneyBench: A Challenging One-Stop Vision-Language Understanding Benchmark of Generated Images Zhecan Wang Junzhang Liu Chia-Wei Tang Hani Alomari Anushka Sivakumar ... Haoxuan You A. Ishmam Kai-Wei Chang Shih-Fu Chang Chris Thomas CoGe VLM 96 2 0 19 Sep 2024
Oryx MLLM: On-Demand Spatial-Temporal Understanding at Arbitrary Resolution Zuyan Liu Yuhao Dong Ziwei Liu Winston Hu Jiwen Lu Yongming Rao ObjD 108 59 0 19 Sep 2024
UrBench: A Comprehensive Benchmark for Evaluating Large Multimodal Models in Multi-View Urban Scenarios Baichuan Zhou Haote Yang Dairong Chen Junyan Ye Tianyi Bai Jinhua Yu Songyang Zhang Dahua Lin Conghui He Weijia Li VLM 94 4 0 30 Aug 2024
Law of Vision Representation in MLLMs Shijia Yang Bohan Zhai Quanzeng You Jianbo Yuan Hongxia Yang Chenfeng Xu 68 10 0 29 Aug 2024
IAA: Inner-Adaptor Architecture Empowers Frozen Large Language Model with Multimodal Capabilities Bin Wang Chunyu Xie Dawei Leng Yuhui Yin MLLM 93 1 0 23 Aug 2024
MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans? Yi-Fan Zhang Huanyu Zhang Haochen Tian Chaoyou Fu Shuangqing Zhang ... Qingsong Wen Zhang Zhang Liwen Wang Rong Jin Tieniu Tan OffRL 90 44 0 23 Aug 2024
EE-MLLM: A Data-Efficient and Compute-Efficient Multimodal Large Language Model Feipeng Ma Yizhou Zhou Hebei Li Zilong He Siying Wu Fengyun Rao Siying Wu Fengyun Rao Yueyi Zhang Xiaoyan Sun 113 8 0 21 Aug 2024
BaThe: Defense against the Jailbreak Attack in Multimodal Large Language Models by Treating Harmful Instruction as Backdoor Trigger Yulin Chen Haoran Li Zihao Zheng Zihao Zheng Yangqiu Song Bryan Hooi 69 6 0 17 Aug 2024
Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretraining Dongyang Liu Shitian Zhao Le Zhuo Weifeng Lin Ping Luo Xinyue Li Qi Qin Yu Qiao Hongsheng Li Peng Gao MLLM 112 54 0 05 Aug 2024
Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models Fushuo Huo Wenchao Xu Zhong Zhang Yining Qi Zhicheng Chen Peilin Zhao VLM MLLM 115 25 0 04 Aug 2024
VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models Haodong Duan Junming Yang Junming Yang Xinyu Fang Lin Chen ... Yuhang Zang Pan Zhang Jiaqi Wang Dahua Lin Kai Chen LM&MA VLM 97 142 0 16 Jul 2024