Can We Predict Performance of Large Models across Vision-Language Tasks?

14 October 2024

Papers citing "Can We Predict Performance of Large Models across Vision-Language Tasks?"

50 / 59 papers shown

Title
Qwen2.5-VL Technical Report S. Bai Keqin Chen Xuejing Liu Jialin Wang Wenbin Ge ... Zesen Cheng Hang Zhang Zhibo Yang Haiyang Xu Junyang Lin VLM 196 504 0 20 Feb 2025
Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark Yunzhuo Hao Jiawei Gu Huichen Will Wang Linjie Li Zhiyong Yang Lijuan Wang Yu Cheng LRM 80 32 0 10 Jan 2025
Apollo: An Exploration of Video Understanding in Large Multimodal Models Orr Zohar Xiaohan Wang Yann Dubois Nikhil Mehta Tong Xiao ... Xiaofang Wang F. Xu Ning Zhang Serena Yeung-Levy Xide Xia VLM 145 22 0 13 Dec 2024
LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding Haoning Wu Dongxu Li Bei Chen Junnan Li 74 143 0 22 Jul 2024
LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Models Kaichen Zhang Bo Li Peiyuan Zhang Fanyi Pu Joshua Adrian Cahyono ... Shuai Liu Yuanhan Zhang Jingkang Yang Chunyuan Li Ziwei Liu 120 88 0 17 Jul 2024
VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models Haodong Duan Junming Yang Junming Yang Xinyu Fang Lin Chen ... Yuhang Zang Pan Zhang Jiaqi Wang Dahua Lin Kai Chen LM&MA VLM 113 158 0 16 Jul 2024
HEMM: Holistic Evaluation of Multimodal Foundation Models Paul Pu Liang Akshay Goindani Talha Chafekar Leena Mathur Haofei Yu Ruslan Salakhutdinov Louis-Philippe Morency 64 15 0 03 Jul 2024
Collaborative Performance Prediction for Large Language Models Qiyuan Zhang Fuyuan Lyu Xue Liu Chen Ma 47 4 0 01 Jul 2024
Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs Shengbang Tong Ellis L Brown Penghao Wu Sanghyun Woo Manoj Middepogu ... Xichen Pan Austin Wang Rob Fergus Yann LeCun Saining Xie 3DV MLLM 87 337 0 24 Jun 2024
Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis Chaoyou Fu Yuhan Dai Yondong Luo Lei Li Shuhuai Ren ... Xiawu Zheng Enhong Chen Caifeng Shan Xing Sun Xing Sun VLM MLLM 109 372 0 31 May 2024
How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites Zhe Chen Weiyun Wang Hao Tian Shenglong Ye Zhangwei Gao ... Tong Lu Dahua Lin Yu Qiao Jifeng Dai Wenhai Wang MLLM VLM 93 601 0 25 Apr 2024
Bounding Box Stability against Feature Dropout Reflects Detector Generalization across Environments Yang Yang Wenhai Wang Zhe Chen Jifeng Dai Liang Zheng 61 3 0 20 Mar 2024
DeepSeek-VL: Towards Real-World Vision-Language Understanding Haoyu Lu Wen Liu Bo Zhang Bing-Li Wang Kai Dong ... Yaofeng Sun Chengqi Deng Hanwei Xu Zhenda Xie Chong Ruan VLM 73 348 0 08 Mar 2024
Lifelong Benchmarks: Efficient Model Evaluation in an Era of Rapid Progress Ameya Prabhu Vishaal Udandarao Philip Torr Matthias Bethge Adel Bibi Samuel Albanie 59 3 0 29 Feb 2024
tinyBenchmarks: evaluating LLMs with fewer examples Felipe Maia Polo Lucas Weber Leshem Choshen Yuekai Sun Gongjun Xu Mikhail Yurochkin ELM 72 91 0 22 Feb 2024
Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset Ke Wang Junting Pan Weikang Shi Zimu Lu Mingjie Zhan Hongsheng Li 54 159 0 22 Feb 2024
Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models Siddharth Karamcheti Suraj Nair Ashwin Balakrishna Percy Liang Thomas Kollar Dorsa Sadigh MLLM VLM 89 119 0 12 Feb 2024
Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs Shengbang Tong Zhuang Liu Yuexiang Zhai Yi-An Ma Yann LeCun Saining Xie VLM MLLM 79 320 0 11 Jan 2024
InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks Zhe Chen Jiannan Wu Wenhai Wang Weijie Su Guo Chen ... Bin Li Ping Luo Tong Lu Yu Qiao Jifeng Dai VLM MLLM 224 1,102 0 21 Dec 2023
SEED-Bench-2: Benchmarking Multimodal Large Language Models Bohao Li Yuying Ge Yixiao Ge Guangzhi Wang Rui Wang Ruimao Zhang Ying Shan MLLM VLM 51 68 0 28 Nov 2023
MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI Xiang Yue Yuansheng Ni Kai Zhang Tianyu Zheng Ruoqi Liu ... Yibo Liu Wenhao Huang Huan Sun Yu-Chuan Su Wenhu Chen OSLM ELM VLM 188 891 0 27 Nov 2023
mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration Qinghao Ye Haiyang Xu Jiabo Ye Mingshi Yan Anwen Hu Haowei Liu Qi Qian Ji Zhang Fei Huang Jingren Zhou MLLM VLM 172 408 0 07 Nov 2023
Improved Baselines with Visual Instruction Tuning Haotian Liu Chunyuan Li Yuheng Li Yong Jae Lee VLM MLLM 108 2,672 0 05 Oct 2023
Efficient Benchmarking of Language Models Yotam Perlitz Elron Bandel Ariel Gera Ofir Arviv L. Ein-Dor Eyal Shnarch Noam Slonim Michal Shmueli-Scheuer Leshem Choshen ALM 44 26 0 22 Aug 2023
MMBench: Is Your Multi-modal Model an All-around Player? Yuanzhan Liu Haodong Duan Yuanhan Zhang Yue Liu Songyang Zhang ... Jiaqi Wang Conghui He Ziwei Liu Kai-xiang Chen Dahua Lin 82 1,006 0 12 Jul 2023
MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models Chaoyou Fu Peixian Chen Yunhang Shen Yulei Qin Mengdan Zhang ... Xiawu Zheng Ke Li Xing Sun Zhenyu Qiu Rongrong Ji ELM MLLM 77 821 0 23 Jun 2023
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 312 4,253 0 09 Jun 2023
MemeCap: A Dataset for Captioning and Interpreting Memes EunJeong Hwang Vered Shwartz VLM 53 37 0 23 May 2023
Evaluating Object Hallucination in Large Vision-Language Models Yifan Li Yifan Du Kun Zhou Jinpeng Wang Wayne Xin Zhao Ji-Rong Wen MLLM LRM 255 768 0 17 May 2023
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning Wenliang Dai Junnan Li Dongxu Li A. M. H. Tiong Junqi Zhao Weisheng Wang Boyang Albert Li Pascale Fung Steven C. H. Hoi MLLM VLM 91 1,977 0 11 May 2023
MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models Deyao Zhu Jun Chen Xiaoqian Shen Xiang Li Mohamed Elhoseiny VLM MLLM 111 2,007 0 20 Apr 2023
Visual Instruction Tuning Haotian Liu Chunyuan Li Qingyang Wu Yong Jae Lee SyDa VLM MLLM 398 4,678 0 17 Apr 2023
IRFL: Image Recognition of Figurative Language Ron Yosef Yonatan Bitton Dafna Shahaf 63 19 0 27 Mar 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 1.0K 14,179 0 15 Mar 2023
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Pan Lu Swaroop Mishra Tony Xia Liang Qiu Kai-Wei Chang Song-Chun Zhu Oyvind Tafjord Peter Clark Ashwin Kalyan ELM ReLM LRM 250 1,230 0 20 Sep 2022
Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contest Jack Hessel Ana Marasović Jena D. Hwang Lillian Lee Jeff Da Rowan Zellers Robert Mankoff Yejin Choi VLM 68 89 0 13 Sep 2022
Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality Tristan Thrush Ryan Jiang Max Bartolo Amanpreet Singh Adina Williams Douwe Kiela Candace Ross CoGe 87 421 0 07 Apr 2022
Screen2Words: Automatic Mobile UI Summarization with Multimodal Learning Bryan Wang Gang Li Xin Zhou Zhourong Chen Tovi Grossman Yang Li 192 156 0 07 Aug 2021
Predicting with Confidence on Unseen Distributions Devin Guillory Vaishaal Shankar Sayna Ebrahimi Trevor Darrell Ludwig Schmidt UQCV OOD 48 120 0 07 Jul 2021
BARTScore: Evaluating Generated Text as Text Generation Weizhe Yuan Graham Neubig Pengfei Liu 95 838 0 22 Jun 2021
SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answering Bo Liu Li-Ming Zhan Li Xu Lin Ma Y. Yang Xiao-Ming Wu 70 259 0 18 Feb 2021
Are Labels Always Necessary for Classifier Accuracy Evaluation? Weijian Deng Liang Zheng 53 116 0 06 Jul 2020
The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes Douwe Kiela Hamed Firooz Aravind Mohan Vedanuj Goswami Amanpreet Singh Pratik Ringshia Davide Testuggine 87 604 0 10 May 2020
MPNet: Masked and Permuted Pre-training for Language Understanding Kaitao Song Xu Tan Tao Qin Jianfeng Lu Tie-Yan Liu 99 1,114 0 20 Apr 2020
PathVQA: 30000+ Questions for Medical Visual Question Answering Xuehai He Yichen Zhang Luntian Mou Eric Xing P. Xie LM&MA 50 234 0 07 Mar 2020
OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge Kenneth Marino Mohammad Rastegari Ali Farhadi Roozbeh Mottaghi 74 1,064 0 31 May 2019
Implicit Regularization in Deep Matrix Factorization Sanjeev Arora Nadav Cohen Wei Hu Yuping Luo AI4CE 74 503 0 31 May 2019
BERTScore: Evaluating Text Generation with BERT Tianyi Zhang Varsha Kishore Felix Wu Kilian Q. Weinberger Yoav Artzi 271 5,764 0 21 Apr 2019
nocaps: novel object captioning at scale Harsh Agrawal Karan Desai Yufei Wang Xinlei Chen Rishabh Jain Mark Johnson Dhruv Batra Devi Parikh Stefan Lee Peter Anderson VLM 102 476 0 20 Dec 2018
From Recognition to Cognition: Visual Commonsense Reasoning Rowan Zellers Yonatan Bisk Ali Farhadi Yejin Choi LRM BDL OCL ReLM 151 877 0 27 Nov 2018