v1v2v3v4 (latest)

Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding

Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023

5 June 2023

Hang Zhang

Xin Li

Lidong Bing

MLLM

ArXiv (abs)PDF HTML HuggingFace (19 upvotes)

Papers citing "Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding"

50 / 669 papers shown

Title
Seeing Beyond the Scene: Enhancing Vision-Language Models with Interactional Reasoning Dayong Liang Changmeng Zheng Zhiyuan Wen Yi Cai Xiao Wei Qing Li LRM 160 1 0 14 May 2025
STORYANCHORS: Generating Consistent Multi-Scene Story Frames for Long-Form Narratives Bo Wang Haoyang Huang Zhiying Lu Fengyuan Liu Guoqing Ma Jianlong Yuan Y. Zhang Nan Duan Daxin Jiang VGen 353 3 0 13 May 2025
SpatialPrompting: Keyframe-driven Zero-Shot Spatial Reasoning with Off-the-Shelf Multimodal Large Language Models Shun Taguchi Hideki Deguchi Takumi Hamazaki Hiroyuki Sakai ReLM LRM 291 0 0 08 May 2025
StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistant Haibo Wang Bo Feng Zhengfeng Lai Mingze Xu Shiyu Li Weifeng Ge Afshin Dehghan Meng Cao Ping Huang OffRL 464 4 0 08 May 2025
R^3-VQA: "Read the Room" by Video Social Reasoning Lixing Niu Jiapeng Li Xingping Yu Shu Wang Ruining Feng Bo Wu Ping Wei Longji Xu Lifeng Fan 230 1 0 07 May 2025
EchoInk-R1: Exploring Audio-Visual Reasoning in Multimodal LLMs via Reinforcement Learning Zhenghao Xing Xiaowei Hu Chi-Wing Fu Wei Wang Jifeng Dai Pheng-Ann Heng MLLM OffRL VLM LRM 290 12 0 07 May 2025
"I Can See Forever!": Evaluating Real-time VideoLLMs for Assisting Individuals with Visual Impairments Zheng Zhang Zhen Sun Zhenru Zhang Zifan Peng Yuemeng Zhao Liang Luo Zeren Luo Ruiting Zuo Xinlei He 159 2 0 07 May 2025
TxP: Reciprocal Generation of Ground Pressure Dynamics and Activity Descriptions for Improving Human Activity RecognitionProceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies (IMWUT), 2025 L. Ray Lars Krupp Vitor Fortes Rey Bo Zhou Sungho Suh Paul Lukowicz AI4CE 595 1 0 04 May 2025
Enhancing the Learning Experience: Using Vision-Language Models to Generate Questions for Educational VideosInternational Conference on Artificial Intelligence in Education (AIED), 2025 Markos Stamatakis Joshua Berger Christian Wartena Ralph Ewerth Anett Hoppe AI4Ed 273 1 0 03 May 2025
TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Action Jen-Hao Cheng Vivian Wang Huayu Wang Huapeng Zhou Yi-Hao Peng ... Wenhao Chai Yi-Ling Chen Vibhav Vineet Qin Cai Lei Li AI4TS 672 7 0 02 May 2025
AdCare-VLM: Towards a Unified and Pre-aligned Latent Representation for Healthcare Video Understanding Md Asaduzzaman Jabin Hanqi Jiang Yuchen Ren Patrick Kaggwa Eugene Douglass Juliet N. Sekandi Tianming Liu LM&MA 348 0 0 01 May 2025
FSBench: A Figure Skating Benchmark for Advancing Artistic Sports UnderstandingComputer Vision and Pattern Recognition (CVPR), 2025 Rong Gao Xin Liu Zhuozhao Hu Bohao Xing Baiqiang Xia Zitong Yu Heikki Kälviäinen 237 2 0 28 Apr 2025
VideoMultiAgents: A Multi-Agent Framework for Video Question Answering Noriyuki Kugo Xiang Li Zhiyu Li Ashish Gupta Arpandeep Khatua ... Yuta Kyuragi Yasunori Ishii Masamoto Tanabiki Kazuki Kozuka Ehsan Adeli 352 9 0 25 Apr 2025
ActionArt: Advancing Multimodal Large Models for Fine-Grained Human-Centric Video Understanding Yi-Xing Peng Q. Yang Yu-Ming Tang Shenghao Fu Kun-Yu Lin Xihan Wei Wei-Shi Zheng 179 5 0 25 Apr 2025
VEU-Bench: Towards Comprehensive Understanding of Video EditingComputer Vision and Pattern Recognition (CVPR), 2025 Bozheng Li Y. Wu Yi Lu Jiashuo Yu Licheng Tang Jiawang Cao Wenqing Zhu Yuyang Sun Jay Wu Wenbo Zhu 220 1 0 24 Apr 2025
TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videos Linli Yao You Li Y. X. Wei Lei Li Shuhuai Ren ... Sida Li Dianbo Sui Qi Liu Yanzhe Zhang Xu Sun 203 11 0 24 Apr 2025
TimeSoccer: An End-to-End Multimodal Large Language Model for Soccer Commentary Generation Ling You Hao Wu Xinni Xie Xiangyi Wei Bangyan Li Shaohui Lin Yang Li Changbo Wang VGen 935 3 0 24 Apr 2025
A Survey of Foundation Model-Powered Recommender Systems: From Feature-Based, Generative to Agentic Paradigms Chengkai Huang Hongtao Huang Tong Yu Kaige Xie Junda Wu Shuai Zhang Julian McAuley Dietmar Jannach Lina Yao LRM AI4CE 226 7 0 23 Apr 2025
ViSMaP: Unsupervised Hour-long Video Summarisation by Meta-Prompting Jian Hu Dimitrios Korkinof S. Gong Mariano Beguerisse-Díaz VLM 178 0 0 22 Apr 2025
IV-Bench: A Benchmark for Image-Grounded Video Perception and Reasoning in Multimodal LLMs David Ma Yanzhe Zhang J. Ren Jarvis Guo Yifan Yao ... Shiwen Ni Jing Liu Wenhao Huang Ge Zhang Xiaojie Jin VLM 258 3 0 21 Apr 2025
Grounding-MD: Grounded Video-language Pre-training for Open-World Moment Detection Weijun Zhuang Qizhang Li Xin Li Ming-Yu Liu Xiaopeng Hong Feng Gao Fan Yang W. Zuo 218 1 0 20 Apr 2025
ResNetVLLM -- Multi-modal Vision LLM for the Video Understanding Task Ahmad Khalil Mahmoud Khalil A. Ngom VLM 199 1 0 20 Apr 2025
ResNetVLLM-2: Addressing ResNetVLLM's Multi-Modal Hallucinations Ahmad Khalil Mahmoud Khalil A. Ngom MLLM VLM 215 1 0 20 Apr 2025
Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmark Enxin Song Wenhao Chai Weili Xu Jianwen Xie Yuxuan Liu Gaoang Wang 297 19 0 20 Apr 2025
VideoPASTA: 7K Preference Pairs That Matter for Video-LLM Alignment Yogesh Kulkarni Pooyan Fazli 364 4 0 18 Apr 2025
VistaDPO: Video Hierarchical Spatial-Temporal Direct Preference Optimization for Large Video Models Haojian Huang Haodong Chen Shengqiong Wu Meng Luo Jinlan Fu Xinya Du Hao Zhang Hao Fei AI4TS 830 8 0 17 Apr 2025
HAVT-IVD: Heterogeneity-Aware Cross-Modal Network for Audio-Visual Surveillance: Idling Vehicles Detection With Multichannel Audio and Multiscale Visual Cues Xiwen Li Ross T. Whitaker Tolga Tasdizen 166 0 0 15 Apr 2025
Video Summarization with Large Language ModelsComputer Vision and Pattern Recognition (CVPR), 2025 Min Jung Lee Dayoung Gong Minsu Cho 189 6 0 15 Apr 2025
Multimodal Long Video Modeling Based on Temporal Dynamic Context Haoran Hao Jiaming Han Yiyuan Zhang Xiangyu Yue 352 0 0 14 Apr 2025
PACT: Pruning and Clustering-Based Token Reduction for Faster Visual Language ModelsComputer Vision and Pattern Recognition (CVPR), 2025 M. Dhouib Davide Buscaldi Sonia Vanier A. Shabou VLM 229 14 0 11 Apr 2025
VideoExpert: Augmented LLM for Temporal-Sensitive Video Understanding Henghao Zhao Ge-Peng Ji Rui Yan Huan Xiong Zechao Li 155 7 0 10 Apr 2025
VCR-Bench: A Comprehensive Evaluation Framework for Video Chain-of-Thought Reasoning Yukun Qi Yiming Zhao Y. Zeng Xikun Bao Wenjie Huang Lin Yen-Chen Zehui Chen Jie Zhao Zhongang Qi Feng Zhao LRM 255 16 0 10 Apr 2025
Memory-efficient Streaming VideoLLMs for Real-time Procedural Video Understanding Dibyadip Chatterjee Edoardo Remelli Yale Song Bugra Tekin Abhay Mittal ... Shreyas Hampali Eric Sauser Shugao Ma Angela Yao Fadime Sener VLM 202 2 0 10 Apr 2025
How Can Objects Help Video-Language Understanding? Zitian Tang Shijie Wang Junho Cho Jaewook Yoo Chen Sun 242 1 0 10 Apr 2025
SF2T: Self-supervised Fragment Finetuning of Video-LLMs for Fine-Grained UnderstandingComputer Vision and Pattern Recognition (CVPR), 2025 Yangliu Hu Zikai Song Na Feng Yawei Luo Junqing Yu Yi-Ping Phoebe Chen Wei Yang 137 10 0 10 Apr 2025
Gaze-Guided Learning: Avoiding Shortcut Bias in Visual Classification Jiahang Li Shibo Xue Yong Su 178 2 0 08 Apr 2025
REEF: Relevance-Aware and Efficient LLM Adapter for Video Understanding Sakib Reza Xiyun Song Heather Yu Zongfang Lin Mohsen Moghaddam Mario Sznaier 178 0 0 07 Apr 2025
REVEAL: Relation-based Video Representation Learning for Video-Question-Answering Sofian Chaybouti Walid Bousselham Moritz Wolter Hilde Kuehne 772 0 0 07 Apr 2025
Advancing Egocentric Video Question Answering with Multimodal Large Language Models Alkesh Patel Vibhav Chitalia Yinfei Yang 149 4 0 06 Apr 2025
Window Token Concatenation for Efficient Visual Large Language Models Jiayi Zhang Wentao Bao Botao Ye Zhen Tan Tianlong Chen Huan Liu Yu Kong VLM 200 1 0 05 Apr 2025
Scaling Video-Language Models to 10K Frames via Hierarchical Differential Distillation Chuanqi Cheng Jian Guan Wei Wu Rui Yan VLM 481 16 0 03 Apr 2025
Multifaceted Evaluation of Audio-Visual Capability for MLLMs: Effectiveness, Efficiency, Generalizability and Robustness Yusheng Zhao Junyu Luo Zhiyuan Ning Weizhi Zhang Zhiping Xiao Wei Ju Philip S. Yu Ming Zhang AuLLM 241 0 0 03 Apr 2025
VEGAS: Towards Visually Explainable and Grounded Artificial Social IntelligenceAAAI Conference on Artificial Intelligence (AAAI), 2025 Hao Li Hao Fei Zechao Hu Zhengwei Yang Zheng Wang 138 4 0 03 Apr 2025
Aligned Better, Listen Better for Audio-Visual Large Language ModelsInternational Conference on Learning Representations (ICLR), 2025 Yuxin Guo Shuailei Ma Shijie Ma Xiaoyi Bao Chen-Wei Xie Kecheng Zheng Tingyu Weng Siyang Sun Yun Zheng Wei Zou MLLM AuLLM 243 6 0 02 Apr 2025
Leveraging Modality Tags for Enhanced Cross-Modal Video Retrieval A. Fragomeni Dima Damen Michael Wray 366 1 0 02 Apr 2025
Slow-Fast Architecture for Video Multi-Modal Large Language Models Min Shi Shihao Wang Chieh-Yun Chen Jitesh Jain Kai Wang Junjun Xiong Guilin Liu Zhiding Yu Humphrey Shi 205 7 0 02 Apr 2025
Shot-by-Shot: Film-Grammar-Aware Training-Free Audio Description Generation Junyu Xie Tengda Han Max Bain Arsha Nagrani Eshika Khandelwal Gül Varol Weidi Xie Andrew Zisserman DiffM VGen 331 3 0 01 Apr 2025
FortisAVQA and MAVEN: a Benchmark Dataset and Debiasing Framework for Robust Multimodal Reasoning Jie Ma Zhitao Gao Qi Chai Jing Liu Peijie Wang Jing Tao Zhou Su 307 4 0 01 Apr 2025
Fair Dynamic Spectrum Access via Fully Decentralized Multi-Agent Reinforcement LearningInternational Symposium on Modeling and Optimization in Mobile, Ad-Hoc and Wireless Networks (WiOpt), 2025 Yubo Zhang Pedro Botelho Trevor Gordon Gil Zussman I. Kadota 199 1 0 31 Mar 2025
Chapter-Llama: Efficient Chaptering in Hour-Long Videos with LLMsComputer Vision and Pattern Recognition (CVPR), 2025 Lucas Ventura Antoine Yang Cordelia Schmid Gül Varol 210 1 0 31 Mar 2025