Title
VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Tasks Lawrence Jang Yinheng Li Charles Ding Justin Lin Paul Pu Liang Dan Zhao Rogerio Bonatti K. Koishida 139 12 0 24 Oct 2024
VoiceTextBlender: Augmenting Large Language Models with Speech Capabilities via Single-Stage Joint Speech-Text Supervised Fine-Tuning Yifan Peng Krishna Puvvada Zhehuai Chen Piotr .Zelasko He Huang Kunal Dhawan Ke Hu Shinji Watanabe Jagadeesh Balam Boris Ginsburg 166 5 0 23 Oct 2024
MM-Ego: Towards Building Egocentric Multimodal LLMs for Video QA Hanrong Ye Haotian Zhang Erik Daxberger Lin Chen Zongyu Lin ... Haoxuan You Dan Xu Zhe Gan Jiasen Lu Yinfei Yang EgoV MLLM 182 14 0 09 Oct 2024
AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark Wenhao Chai Enxin Song Y. Du Chenlin Meng Vashisht Madhavan Omer Bar-Tal Jeng-Neng Hwang Saining Xie Christopher D. Manning 3DV 219 37 0 04 Oct 2024
Frame-Voyager: Learning to Query Frames for Video Large Language Models Sicheng Yu Chengkai Jin Huanyu Wang Zhenghao Chen Sheng Jin ... Zhenbang Sun Bingni Zhang Jiawei Wu Hao Zhang Qianru Sun 171 9 0 04 Oct 2024
Video Instruction Tuning With Synthetic Data Yuanhan Zhang Jinming Wu Wei Li Bo Li Zejun Ma Ziwei Liu Chunyuan Li SyDa VGen 120 215 0 03 Oct 2024
LLaVA-Critic: Learning to Evaluate Multimodal Models Tianyi Xiong Xinze Wang Dong Guo Qinghao Ye Haoqi Fan Quanquan Gu Heng Huang Chunyuan Li MLLM VLM LRM 137 53 0 03 Oct 2024
Leopard: A Vision Language Model For Text-Rich Multi-Image Tasks Mengzhao Jia Wenhao Yu Kaixin Ma Tianqing Fang Z. Zhang Siru Ouyang Hongming Zhang Meng Jiang Dong Yu VLM 109 7 0 02 Oct 2024
LMOD: A Large Multimodal Ophthalmology Dataset and Benchmark for Large Vision-Language Models Zhenyue Qin Yu Yin Dylan Campbell Xuansheng Wu Ke Zou Yih-Chung Tham Ninghao Liu Xiuzhen Zhang Qingyu Chen 123 1 0 02 Oct 2024
LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness Chenming Zhu Tai Wang Wenwei Zhang Jiangmiao Pang Xihui Liu 248 52 0 26 Sep 2024
Can Vision Language Models Learn from Visual Demonstrations of Ambiguous Spatial Reasoning? Bowen Zhao Leo Parker Dirac Paulina Varshavskaya VLM LRM 101 0 0 25 Sep 2024
EventHallusion: Diagnosing Event Hallucinations in Video LLMs Jiacheng Zhang Yang Jiao Shaoxiang Chen Jingjing Chen Zhiyu Tan Hao Li Jingjing Chen MLLM 148 23 0 25 Sep 2024
Multi-modal Generative AI: Multi-modal LLMs, Diffusions and the Unification X. Wang Yuwei Zhou Bin Huang Hong Chen Wenwu Zhu DiffM 151 9 0 23 Sep 2024
JourneyBench: A Challenging One-Stop Vision-Language Understanding Benchmark of Generated Images Zhecan Wang Junzhang Liu Chia-Wei Tang Hani Alomari Anushka Sivakumar ... Haoxuan You A. Ishmam Kai-Wei Chang Shih-Fu Chang Chris Thomas CoGe VLM 173 2 0 19 Sep 2024
Large Language Models are Strong Audio-Visual Speech Recognition Learners Umberto Cappellazzo Minsu Kim Honglie Chen Pingchuan Ma Stavros Petridis Daniele Falavigna Alessio Brutti Maja Pantic 114 12 0 18 Sep 2024
UrBench: A Comprehensive Benchmark for Evaluating Large Multimodal Models in Multi-View Urban Scenarios Baichuan Zhou Haote Yang Dairong Chen Junyan Ye Tianyi Bai Jinhua Yu Songyang Zhang Dahua Lin Conghui He Weijia Li VLM 179 7 0 30 Aug 2024
Law of Vision Representation in MLLMs Shijia Yang Bohan Zhai Quanzeng You Jianbo Yuan Hongxia Yang Chenfeng Xu 157 12 0 29 Aug 2024
Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders Min Shi Fuxiao Liu Shihao Wang Shijia Liao Subhashree Radhakrishnan ... Andrew Tao Andrew Tao Zhiding Yu Guilin Liu Guilin Liu MLLM 155 68 0 28 Aug 2024
RoVRM: A Robust Visual Reward Model Optimized via Auxiliary Textual Preference Data Chenglong Wang Yang Gan Yifu Huo Yongyu Mu Murun Yang ... Chunliang Zhang Tongran Liu Quan Du Di Yang Jingbo Zhu VLM 173 6 0 22 Aug 2024
xGen-MM (BLIP-3): A Family of Open Large Multimodal Models Le Xue Manli Shu Anas Awadalla Jun Wang An Yan ... Zeyuan Chen Silvio Savarese Juan Carlos Niebles Caiming Xiong Ran Xu VLM 108 96 0 16 Aug 2024
VidGen-1M: A Large-Scale Dataset for Text-to-video Generation Zhiyu Tan Xiaomeng Yang Luozheng Qin Hao Li VGen 105 22 0 05 Aug 2024
Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Survey Atsuyuki Miyai Jingkang Yang Jingyang Zhang Yifei Ming Sisir Dhakal ... Yixuan Li Hai "Helen" Li Ziwei Liu Toshihiko Yamasaki Kiyoharu Aizawa 139 13 0 31 Jul 2024
Effectively Leveraging CLIP for Generating Situational Summaries of Images and Videos Dhruv Verma Debaditya Roy Basura Fernando 86 1 0 30 Jul 2024
VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models Haodong Duan Junming Yang Junming Yang Xinyu Fang Lin Chen ... Yuhang Zang Pan Zhang Jiaqi Wang Dahua Lin Kai Chen LM&MA VLM 242 180 0 16 Jul 2024
LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models Feng Li Renrui Zhang Hao Zhang Yuanhan Zhang Bo Li Wei Li Zejun Ma Chunyuan Li MLLM VLM 135 233 0 10 Jul 2024
MIA-Bench: Towards Better Instruction Following Evaluation of Multimodal LLMs Yusu Qian Hanrong Ye J. Fauconnier Peter Grasch Yinfei Yang Zhe Gan 249 18 0 01 Jul 2024
MMFakeBench: A Mixed-Source Multimodal Misinformation Detection Benchmark for LVLMs Xuannan Liu Zekun Li Peipei Li Shuhan Xia Xing Cui Linzhi Huang Huaibo Huang Weihong Deng Zhaofeng He 141 23 0 13 Jun 2024
F-LMM: Grounding Frozen Large Multimodal Models Size Wu Sheng Jin Wenwei Zhang Lumin Xu Wentao Liu Wei Li Chen Change Loy MLLM 190 15 0 09 Jun 2024
Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis Chaoyou Fu Yuhan Dai Yondong Luo Lei Li Shuhuai Ren ... Xiawu Zheng Enhong Chen Caifeng Shan Xing Sun Xing Sun VLM MLLM 177 421 0 31 May 2024
Enhancing Visual-Language Modality Alignment in Large Vision Language Models via Self-Improvement Xiyao Wang Jiuhai Chen Zhaoyang Wang Yuhang Zhou Yiyang Zhou ... Dinesh Manocha Tom Goldstein Parminder Bhatia Furong Huang Cao Xiao 203 38 0 24 May 2024
Efficient Multimodal Large Language Models: A Survey Yizhang Jin Jian Li Yexin Liu Tianjun Gu Kai Wu ... Xin Tan Zhenye Gan Yabiao Wang Chengjie Wang Lizhuang Ma LRM 119 58 0 17 May 2024
An empirical study of LLaMA3 quantization: from LLMs to MLLMs Wei Huang Xingyu Zheng Xudong Ma Haotong Qin Chengtao Lv Hong Chen Jie Luo Xiaojuan Qi Xianglong Liu Michele Magno MQ 152 42 0 22 Apr 2024
CausalChaos! Dataset for Comprehensive Causal Action Question Answering Over Longer Causal Chains Grounded in Dynamic Visual Scenes Paritosh Parmar Eric Peh Ruirui Chen Ting En Lam Yuhan Chen Elston Tan Basura Fernando CML 93 7 0 01 Apr 2024
The Revolution of Multimodal Large Language Models: A Survey Davide Caffagni Federico Cocchi Luca Barsellotti Nicholas Moratelli Sara Sarto Lorenzo Baraldi Lorenzo Baraldi Marcella Cornia Rita Cucchiara LRM VLM 135 64 0 19 Feb 2024
Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perception Junyang Wang Haiyang Xu Jiabo Ye Mingshi Yan Weizhou Shen Ji Zhang Fei Huang Jitao Sang 140 129 0 29 Jan 2024
Video Understanding with Large Language Models: A Survey Yunlong Tang Jing Bi Siting Xu Luchuan Song Susan Liang ... Feng Zheng Jianguo Zhang Chenliang Xu Jiebo Luo Chenliang Xu VLM 222 100 0 29 Dec 2023
MM-GEF: Multi-modal representation meet collaborative filtering Hao Wu Alejandro Ariza-Casabona Bartlomiej Twardowski Tri Kurniawan Wijaya 49 2 0 14 Aug 2023
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration Ji Lin Jiaming Tang Haotian Tang Shang Yang Wei-Ming Chen Wei-Chen Wang Guangxuan Xiao Xingyu Dang Chuang Gan Song Han EDL MQ 226 588 0 01 Jun 2023
Structural Pruning via Latency-Saliency Knapsack Maying Shen Hongxu Yin Pavlo Molchanov Lei Mao Jianna Liu J. Álvarez 100 50 0 13 Oct 2022