Title
Infi-MMR: Curriculum-based Unlocking Multimodal Reasoning via Phased Reinforcement Learning in Multimodal Small Language Models Zeyu Liu Y. Liu Guanghao Zhu C. Xie Zhen Li ... Qing Li Shing-Chi Cheung Shengyu Zhang Fei Wu Hongxia Yang ReLM LRM 69 0 0 29 May 2025
Qwen2.5-VL Technical Report S. Bai Keqin Chen Xuejing Liu Jialin Wang Wenbin Ge ... Zesen Cheng Hang Zhang Zhibo Yang Haiyang Xu Junyang Lin VLM 354 699 0 20 Feb 2025
MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency Dongzhi Jiang Renrui Zhang Ziyu Guo Yanwei Li Yu Qi ... Shen Yan Bo Zhang Chaoyou Fu Peng Gao Hongsheng Li MLLM LRM 121 38 0 13 Feb 2025
COCONut-PanCap: Joint Panoptic Segmentation and Grounded Captions for Fine-Grained Understanding and Generation XueQing Deng Qihang Yu Ali Athar Chenglin Yang Linjie Yang Xiaojie Jin Xiaohui Shen Liang-Chieh Chen VLM 105 5 0 04 Feb 2025
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning DeepSeek-AI Daya Guo Dejian Yang Haowei Zhang Junxiao Song ... Shiyu Wang S. Yu Shunfeng Zhou Shuting Pan S.S. Li ReLM VLM OffRL AI4TS LRM 380 2,000 0 22 Jan 2025
Open Eyes, Then Reason: Fine-grained Visual Mathematical Understanding in MLLMs Shan Zhang Aotian Chen Yanpeng Sun Jindong Gu Yi-Yu Zheng Piotr Koniusz Kai Zou Anton van den Hengel Yuan Xue LRM 49 6 0 11 Jan 2025
URSA: Understanding and Verifying Chain-of-thought Reasoning in Multimodal Mathematics Ruilin Luo Zhuofan Zheng Yifan Wang Xinzhe Ni Zicheng Lin ... Yiyao Yu C. Shi Ruihang Chu Jin Zeng Yujiu Yang LRM 171 25 0 08 Jan 2025
Progressive Multimodal Reasoning via Active Retrieval Guanting Dong Chenghao Zhang Mengjie Deng Yinlin Zhu Zhicheng Dou Ji-Rong Wen LRM 116 11 0 19 Dec 2024
GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training Renqiu Xia Mingxing Li Hancheng Ye Wenjie Wu Hongbin Zhou ... Zeang Sheng Botian Shi Tao Chen Junchi Yan Bo Zhang 163 10 0 16 Dec 2024
Chimera: Improving Generalist Model with Domain-Specific Experts Tianshuo Peng Mingxing Li Hongbin Zhou Renqiu Xia Renrui Zhang ... Aojun Zhou Botian Shi Tao Chen Bo Zhang Xiangyu Yue 173 5 0 08 Dec 2024
CompCap: Improving Multimodal Large Language Models with Composite Captions Xiaohui Chen Satya Narayan Shukla Mahmoud Azab Aashu Singh Qifan Wang ... ShengYun Peng Hanchao Yu Shen Yan Xinming Zhang Baosheng He VLM 71 4 0 06 Dec 2024
Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization Weiyun Wang Zhe Chen Wenhai Wang Yue Cao Yangzhou Liu ... Jinguo Zhu X. Zhu Lewei Lu Yu Qiao Jifeng Dai LRM 129 92 1 15 Nov 2024
Harnessing Webpage UIs for Text-Rich Visual Understanding Junpeng Liu Tianyue Ou Yifan Song Yuxiao Qu Wai Lam Chenyan Xiong Wenhu Chen Graham Neubig Xiang Yue 121 10 0 17 Oct 2024
Mono-InternVL: Pushing the Boundaries of Monolithic Multimodal Large Language Models with Endogenous Visual Pre-training Gen Luo Xue Yang Wenhan Dou Zhaokai Wang Jifeng Dai Jifeng Dai Yu Qiao Xizhou Zhu VLM MLLM 155 34 0 10 Oct 2024
MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuning Haotian Zhang Mingfei Gao Zhe Gan Philipp Dufter Nina Wenzel ... Haoxuan You Zirui Wang Afshin Dehghan Peter Grasch Yinfei Yang VLM MLLM 119 41 1 30 Sep 2024
AutoGeo: Automating Geometric Image Dataset Creation for Enhanced Geometry Understanding Zihan Huang Tao Wu Wang Lin Shengyu Zhang Jingyuan Chen Fei Wu 54 12 0 28 Aug 2024
LLaVA-OneVision: Easy Visual Task Transfer Bo Li Yuanhan Zhang Dong Guo Renrui Zhang Feng Li Hao Zhang Kaichen Zhang Yanwei Li Ziwei Liu Chunyuan Li MLLM SyDa VLM 126 867 0 06 Aug 2024
MiniCPM-V: A GPT-4V Level MLLM on Your Phone Yuan Yao Tianyu Yu Ao Zhang Chongyi Wang Junbo Cui ... Xu Han Guoyang Zeng Dahai Li Zhiyuan Liu Maosong Sun VLM MLLM 120 477 0 03 Aug 2024
MoMa: Efficient Early-Fusion Pre-training with Mixture of Modality-Aware Experts Xi Lin Akshat Shrivastava Liang Luo Srinivasan Iyer Mike Lewis Gargi Gosh Luke Zettlemoyer Armen Aghajanyan MoE 94 23 0 31 Jul 2024
DenseFusion-1M: Merging Vision Experts for Comprehensive Multimodal Perception Xiaotong Li Fan Zhang Haiwen Diao Yueze Wang Xinlong Wang Ling-yu Duan VLM 112 32 0 11 Jul 2024
OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation Kepan Nan Rui Xie Penghao Zhou Tiehan Fan Zhenheng Yang Zhijie Chen Xiang Li Jian Yang Ying Tai 140 93 0 02 Jul 2024
Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs Shengbang Tong Ellis L Brown Penghao Wu Sanghyun Woo Manoj Middepogu ... Xichen Pan Austin Wang Rob Fergus Yann LeCun Saining Xie 3DV MLLM 148 378 0 24 Jun 2024
Unveiling Encoder-Free Vision-Language Models Haiwen Diao Yufeng Cui Xiaotong Li Yueze Wang Huchuan Lu Xinlong Wang VLM 103 36 0 17 Jun 2024
Multimodal Table Understanding Mingyu Zheng Xinwei Feng Q. Si Qiaoqiao She Zheng Lin Wenbin Jiang Weiping Wang LMTD VLM 127 20 0 12 Jun 2024
Benchmarking and Improving Detail Image Caption Hongyuan Dong Jiawen Li Bohong Wu Jiacong Wang Yuan Zhang Haoyuan Guo VLM MLLM 75 31 0 29 May 2024
Chameleon: Mixed-Modal Early-Fusion Foundation Models Chameleon Team MLLM 196 339 0 16 May 2024
Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformers Peng Gao Le Zhuo Ziyi Lin Ruoyi Du Xu Luo ... Weicai Ye He Tong Jingwen He Yu Qiao Hongsheng Li VGen 84 91 0 09 May 2024
How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites Zhe Chen Weiyun Wang Hao Tian Shenglong Ye Zhangwei Gao ... Tong Lu Dahua Lin Yu Qiao Jifeng Dai Wenhai Wang MLLM VLM 133 642 0 25 Apr 2024
LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understanding Chuwei Luo Yufan Shen Zhaoqing Zhu Qi Zheng Zhi Yu Cong Yao 108 49 0 08 Apr 2024
MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems? Renrui Zhang Dongzhi Jiang Yichi Zhang Haokun Lin Ziyu Guo ... Aojun Zhou Pan Lu Kai-Wei Chang Peng Gao Hongsheng Li 80 254 0 21 Mar 2024
mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understanding Anwen Hu Haiyang Xu Jiabo Ye Mingshi Yan Liang Zhang ... Chen Li Ji Zhang Qin Jin Fei Huang Jingren Zhou VLM 112 124 0 19 Mar 2024
Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers Tsai-Shien Chen Aliaksandr Siarohin Willi Menapace Ekaterina Deyneka Hsiang-wei Chao ... Yuwei Fang Hsin-Ying Lee Jian Ren Ming-Hsuan Yang Sergey Tulyakov VGen 155 210 0 29 Feb 2024
Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset Ke Wang Junting Pan Weikang Shi Zimu Lu Mingjie Zhan Hongsheng Li 86 188 0 22 Feb 2024
OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems Chaoqun He Renjie Luo Yuzhuo Bai Shengding Hu Zhen Leng Thai ... Yuxiang Zhang Jie Liu Lei Qi Zhiyuan Liu Maosong Sun ELM AIMat 129 282 0 21 Feb 2024
ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoning Renqiu Xia Bo Zhang Hancheng Ye Xiangchao Yan Qi Liu ... Min Dou Botian Shi Junchi Yan Junchi Yan Yu Qiao LRM 134 68 0 19 Feb 2024
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models Chris Liu Renrui Zhang Longtian Qiu Siyuan Huang Weifeng Lin ... Hao Shao Pan Lu Hongsheng Li Yu Qiao Peng Gao MLLM 210 116 0 08 Feb 2024
MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI Xiang Yue Yuansheng Ni Kai Zhang Tianyu Zheng Ruoqi Liu ... Yibo Liu Wenhao Huang Huan Sun Yu-Chuan Su Wenhu Chen OSLM ELM VLM 268 960 0 27 Nov 2023
ShareGPT4V: Improving Large Multi-Modal Models with Better Captions Lin Chen Jinsong Li Xiao-wen Dong Pan Zhang Conghui He Jiaqi Wang Feng Zhao Dahua Lin MLLM VLM 200 682 0 21 Nov 2023
SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Models Ziyi Lin Chris Liu Renrui Zhang Peng Gao Longtian Qiu ... Siyuan Huang Yichi Zhang Xuming He Hongsheng Li Yu Qiao MLLM VLM 89 230 0 13 Nov 2023
GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment Dhruba Ghosh Hanna Hajishirzi Ludwig Schmidt 93 203 0 17 Oct 2023
MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts Pan Lu Hritik Bansal Tony Xia Jiacheng Liu Chun-yue Li Hannaneh Hajishirzi Hao Cheng Kai-Wei Chang Michel Galley Jianfeng Gao LRM MLLM 128 665 0 03 Oct 2023
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 413 12,076 0 18 Jul 2023
MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models Chaoyou Fu Peixian Chen Yunhang Shen Yulei Qin Mengdan Zhang ... Xiawu Zheng Ke Li Xing Sun Zhenyu Qiu Rongrong Ji ELM MLLM 128 859 0 23 Jun 2023
Visual Instruction Tuning Haotian Liu Chunyuan Li Qingyang Wu Yong Jae Lee SyDa VLM MLLM 571 4,925 0 17 Apr 2023
ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning Ahmed Masry Do Xuan Long J. Tan Shafiq Joty Enamul Hoque AIMat 134 685 0 19 Mar 2022
CLIPScore: A Reference-free Evaluation Metric for Image Captioning Jack Hessel Ari Holtzman Maxwell Forbes Ronan Le Bras Yejin Choi CLIP 169 1,588 0 18 Apr 2021
DocVQA: A Dataset for VQA on Document Images Minesh Mathew Dimosthenis Karatzas C. V. Jawahar 151 747 0 01 Jul 2020