Title
VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models Haodong Duan Junming Yang Junming Yang Xinyu Fang Lin Chen ... Yuhang Zang Pan Zhang Jiaqi Wang Dahua Lin Kai Chen LM&MA VLM 97 142 0 16 Jul 2024
What matters when building vision-language models? Hugo Laurençon Léo Tronchon Matthieu Cord Victor Sanh VLM 74 166 0 03 May 2024
MANTIS: Interleaved Multi-Image Instruction Tuning Dongfu Jiang Xuan He Huaye Zeng Cong Wei Max Ku Qian Liu Wenhu Chen VLM MLLM 43 111 0 02 May 2024
Yi: Open Foundation Models by 01.AI 01. AI Alex Young 01.AI Alex Young Bei Chen Chao Li ... Yue Wang Yuxuan Cai Zhenyu Gu Zhiyuan Liu Zonghong Dai OSLM LRM 202 538 0 07 Mar 2024
World Model on Million-Length Video And Language With Blockwise RingAttention Hao Liu Wilson Yan Matei A. Zaharia Pieter Abbeel VGen 83 74 0 13 Feb 2024
MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI Xiang Yue Yuansheng Ni Kai Zhang Tianyu Zheng Ruoqi Liu ... Yibo Liu Wenhao Huang Huan Sun Yu-Chuan Su Wenhu Chen OSLM ELM VLM 148 833 0 27 Nov 2023
Qwen Technical Report Jinze Bai Shuai Bai Yunfei Chu Zeyu Cui Kai Dang ... Zhenru Zhang Chang Zhou Jingren Zhou Xiaohuan Zhou Tianhang Zhu OSLM 157 1,756 0 28 Sep 2023
MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning Xiang Yue Xingwei Qu Ge Zhang Yao Fu Wenhao Huang Huan Sun Yu-Chuan Su Wenhu Chen AIMat LRM 97 391 0 11 Sep 2023
CMMLU: Measuring massive multitask language understanding in Chinese Haonan Li Yixuan Zhang Fajri Koto Yifei Yang Hai Zhao Yeyun Gong Nan Duan Tim Baldwin ALM ELM 64 253 0 15 Jun 2023
CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society Ge Li Hasan Hammoud Hani Itani Dmitrii Khizbullin Guohao Li SyDa ALM 82 475 0 31 Mar 2023
Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning Pan Lu Liang Qiu Kai-Wei Chang Ying Nian Wu Song-Chun Zhu Tanmay Rajpurohit Peter Clark Ashwin Kalyan ReLM LRM 108 285 0 29 Sep 2022
A-OKVQA: A Benchmark for Visual Question Answering using World Knowledge Dustin Schwenk Apoorv Khandelwal Christopher Clark Kenneth Marino Roozbeh Mottaghi 43 530 0 03 Jun 2022
LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs Christoph Schuhmann Richard Vencu Romain Beaumont R. Kaczmarczyk Clayton Mullis Aarush Katta Theo Coombes J. Jitsev Aran Komatsuzaki VLM MLLM CLIP 180 1,398 0 03 Nov 2021
IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning Pan Lu Liang Qiu Jiaqi Chen Tony Xia Yizhou Zhao Wei Zhang Zhou Yu Xiaodan Liang Song-Chun Zhu AIMat 66 197 0 25 Oct 2021
Screen2Words: Automatic Mobile UI Summarization with Multimodal Learning Bryan Wang Gang Li Xin Zhou Zhourong Chen Tovi Grossman Yang Li 188 156 0 07 Aug 2021
Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning Pan Lu Ran Gong Shibiao Jiang Liang Qiu Siyuan Huang Xiaodan Liang Song-Chun Zhu AIMat LRM 35 223 0 10 May 2021
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 386 1,103 0 17 Feb 2021
DocVQA: A Dataset for VQA on Document Images Minesh Mathew Dimosthenis Karatzas C. V. Jawahar 94 700 0 01 Jul 2020
The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes Douwe Kiela Hamed Firooz Aravind Mohan Vedanuj Goswami Amanpreet Singh Pratik Ringshia Davide Testuggine 68 601 0 10 May 2020
TextCaps: a Dataset for Image Captioning with Reading Comprehension Oleksii Sidorov Ronghang Hu Marcus Rohrbach Amanpreet Singh 56 406 0 24 Mar 2020
Captioning Images Taken by People Who Are Blind Danna Gurari Yinan Zhao Meng Zhang Nilavra Bhattacharya 54 182 0 20 Feb 2020
Chinese Street View Text: Large-scale Chinese Text Reading with Partially Supervised Learning Yipeng Sun Jiaming Liu Wei Liu Junyu Han Errui Ding Jingtuo Liu 63 51 0 17 Sep 2019
ICDAR2019 Robust Reading Challenge on Arbitrary-Shaped Text (RRC-ArT) Chee-Kheng Chng Yuliang Liu Yipeng Sun Chun Chet Ng Canjie Luo ... Errui Ding Jingtuo Liu Dimosthenis Karatzas Chee Seng Chan Lianwen Jin 3DV 64 211 0 16 Sep 2019
Scene Text Visual Question Answering Ali Furkan Biten Rubèn Pérez Tito Andrés Mafla Lluís Gómez Marçal Rusiñol Ernest Valveny C. V. Jawahar Dimosthenis Karatzas 68 348 0 31 May 2019
Towards VQA Models That Can Read Amanpreet Singh Vivek Natarajan Meet Shah Yu Jiang Xinlei Chen Dhruv Batra Devi Parikh Marcus Rohrbach EgoV 50 1,174 0 18 Apr 2019
A Corpus for Reasoning About Natural Language Grounded in Photographs Alane Suhr Stephanie Zhou Ally Zhang Iris Zhang Huajun Bai Yoav Artzi LRM 78 596 0 01 Nov 2018
Learning to Describe Differences Between Pairs of Similar Images Harsh Jhamtani Taylor Berg-Kirkpatrick 49 151 0 31 Aug 2018
VizWiz Grand Challenge: Answering Visual Questions from Blind People Danna Gurari Qing Li Abigale Stangl Anhong Guo Chi Lin Kristen Grauman Jiebo Luo Jeffrey P. Bigham CoGe 66 831 0 22 Feb 2018
DVQA: Understanding Data Visualizations via Question Answering Kushal Kafle Brian L. Price Scott D. Cohen Christopher Kanan AIMat 49 379 0 24 Jan 2018
CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning Justin Johnson B. Hariharan Laurens van der Maaten Li Fei-Fei C. L. Zitnick Ross B. Girshick CoGe 275 2,346 0 20 Dec 2016
Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering Yash Goyal Tejas Khot D. Summers-Stay Dhruv Batra Devi Parikh CoGe 291 3,187 0 02 Dec 2016
Exploring Models and Data for Image Question Answering Mengye Ren Ryan Kiros R. Zemel 75 713 0 08 May 2015
Microsoft COCO Captions: Data Collection and Evaluation Server Xinlei Chen Hao Fang Nayeon Lee Ramakrishna Vedantam Saurabh Gupta Piotr Dollar C. L. Zitnick 153 2,461 0 01 Apr 2015