VideoChat: Chat-Centric Video Understanding

10 May 2023

Yi Wang

Ping Luo

Yu Qiao

Papers citing "VideoChat: Chat-Centric Video Understanding"

50 / 425 papers shown

Title
AWT: Transferring Vision-Language Models via Augmentation, Weighting, and Transportation Yuhan Zhu Yuyang Ji Zhiyu Zhao Gangshan Wu Limin Wang VLM 41 7 0 05 Jul 2024
InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output Pan Zhang Xiaoyi Dong Yuhang Zang Yuhang Cao Rui Qian ... Kai Chen Jifeng Dai Yu Qiao Dahua Lin Jiaqi Wang 45 100 0 03 Jul 2024
Video Watermarking: Safeguarding Your Video from (Unauthorized) Annotations by Video-based LLMs Jinmin Li Kuofeng Gao Yang Bai Jingyun Zhang Shu-Tao Xia 48 4 0 02 Jul 2024
Meerkat: Audio-Visual Large Language Model for Grounding in Space and Time Sanjoy Chowdhury Sayan Nag Subhrajyoti Dasgupta Jun Chen Mohamed Elhoseiny Ruohan Gao Dinesh Manocha VLM MLLM 41 9 0 01 Jul 2024
Tarsier: Recipes for Training and Evaluating Large Video Description Models Jiawei Wang Liping Yuan Yuchen Zhang 44 52 0 30 Jun 2024
ReXTime: A Benchmark Suite for Reasoning-Across-Time in Videos Jr-Jen Chen Yu-Chien Liao Hsi-Che Lin Yu-Chu Yu Yen-Chun Chen Yu-Chiang Frank Wang 37 10 0 27 Jun 2024
GUIDE: A Guideline-Guided Dataset for Instructional Video Comprehension Jiafeng Liang Shixin Jiang Zekun Wang Haojie Pan Zerui Chen Zheng Chu Ming Liu Ruiji Fu Zhongyuan Wang Bing Qin 29 2 0 26 Jun 2024
MLLM as Video Narrator: Mitigating Modality Imbalance in Video Moment Retrieval Weitong Cai Jiabo Huang Shaogang Gong Hailin Jin Yang Liu 44 0 0 25 Jun 2024
MG-LLaVA: Towards Multi-Granularity Visual Instruction Tuning Xiangyu Zhao Xiangtai Li Haodong Duan Haian Huang Yining Li Kai Chen Hua Yang VLM MLLM 42 10 0 25 Jun 2024
Zero-Shot Long-Form Video Understanding through Screenplay Yongliang Wu Bozheng Li Jiawang Cao Wenbo Zhu Yi Lu ... Chuyun Xie Haolin Zheng Ziyue Su Jay Wu Xu Yang 40 4 0 25 Jun 2024
EVALALIGN: Supervised Fine-Tuning Multimodal LLMs with Human-Aligned Data for Evaluating Text-to-Image Models Zhiyu Tan Xiaomeng Yang Luozheng Qin Mengping Yang Cheng Zhang Hao Li 44 7 0 24 Jun 2024
Directed Domain Fine-Tuning: Tailoring Separate Modalities for Specific Training Tasks Daniel Wen Nafisa Hussain 102 0 0 24 Jun 2024
video-SALMONN: Speech-Enhanced Audio-Visual Large Language Models Guangzhi Sun Wenyi Yu Changli Tang Xianzhao Chen Tian Tan Wei Li Lu Lu Zejun Ma Yuxuan Wang Chao Zhang 45 22 0 22 Jun 2024
MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding Xinyu Fang Kangrui Mao Haodong Duan Xiangyu Zhao Yining Li Dahua Lin Kai Chen VLM 57 61 0 20 Jun 2024
Through the Theory of Mind's Eye: Reading Minds with Multimodal Video Large Language Models Zhawnen Chen Tianchun Wang Yizhou Wang Michal Kosinski Xiang Zhang Yun Fu Sheng Li LRM 29 2 0 19 Jun 2024
Holmes-VAD: Towards Unbiased and Explainable Video Anomaly Detection via Multi-modal LLM Huaxin Zhang Xiaohao Xu Xiang Wang Jialong Zuo Chuchu Han Xiaonan Huang Changxin Gao Yuehuan Wang Nong Sang 60 16 0 18 Jun 2024
VoCo-LLaMA: Towards Vision Compression with Large Language Models Xubing Ye Yukang Gan Xiaoke Huang Yixiao Ge Yansong Tang MLLM VLM 40 22 0 18 Jun 2024
VideoLLM-online: Online Video Large Language Model for Streaming Video Joya Chen Zhaoyang Lv Shiwei Wu Kevin Qinghong Lin Chenan Song Difei Gao Jia-Wei Liu Ziteng Gao Dongxing Mao Mike Zheng Shou MLLM MoMe 45 49 0 17 Jun 2024
VideoVista: A Versatile Benchmark for Video Understanding and Reasoning Yunxin Li Xinyu Chen Baotian Hu Longyue Wang Haoyuan Shi Min-Ling Zhang MLLM LRM 53 25 0 17 Jun 2024
Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning Zebang Cheng Zhi-Qi Cheng Jun-Yan He Jingdong Sun Kai Wang Yuxiang Lin Zheng Lian Xiaojiang Peng Alexander G. Hauptmann MLLM 31 29 0 17 Jun 2024
Beyond Raw Videos: Understanding Edited Videos with Large Multimodal Model Lu Xu Sijie Zhu Chunyuan Li Chia-Wen Kuo Fan Chen Xinyao Wang Guang Chen Dawei Du Ye Yuan Longyin Wen 44 4 0 15 Jun 2024
GPT-4o: Visual perception performance of multimodal large language models in piglet activity understanding Yiqi Wu Xiaodan Hu Ziming Fu Siling Zhou Jiangong Li MLLM 32 10 0 14 Jun 2024
VANE-Bench: Video Anomaly Evaluation Benchmark for Conversational LMMs Rohit K Bharadwaj Hanan Gani Muzammal Naseer F. Khan Salman Khan 67 3 0 14 Jun 2024
VideoGPT+: Integrating Image and Video Encoders for Enhanced Video Understanding Muhammad Maaz H. Rasheed Salman Khan Fahad A Khan VLM MLLM 34 49 0 13 Jun 2024
Explore the Limits of Omni-modal Pretraining at Scale Yiyuan Zhang Handong Li Jing Liu Xiangyu Yue VLM LRM 49 1 0 13 Jun 2024
Needle In A Video Haystack: A Scalable Synthetic Evaluator for Video MLLMs Zijia Zhao Haoyu Lu Yuqi Huo Yifan Du Tongtian Yue Longteng Guo Bingning Wang Weipeng Chen Jing Liu 44 2 0 13 Jun 2024
OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Text Qingyun Li Zhe Chen Weiyun Wang Wenhai Wang Shenglong Ye ... Dahua Lin Yu Qiao Botian Shi Conghui He Jifeng Dai VLM OffRL 56 20 0 12 Jun 2024
MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videos Xuehai He Weixi Feng Kaizhi Zheng Yujie Lu Wanrong Zhu ... Zhengyuan Yang Kevin Lin William Yang Wang Lijuan Wang Xin Eric Wang VGen LRM 43 12 0 12 Jun 2024
Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streams Haoji Zhang Yiqin Wang Yansong Tang Yong-Jin Liu Jiashi Feng Jifeng Dai Xiaojie Jin 45 38 0 12 Jun 2024
LVBench: An Extreme Long Video Understanding Benchmark Weihan Wang Zehai He Wenyi Hong Yean Cheng Xiaohan Zhang ... Shiyu Huang Bin Xu Yuxiao Dong Ming Ding Jie Tang ELM VLM 47 65 0 12 Jun 2024
Fewer Tokens and Fewer Videos: Extending Video Understanding Abilities in Large Vision-Language Models Shimin Chen Yitian Yuan Shaoxiang Chen Zequn Jie Lin Ma VLM 29 3 0 12 Jun 2024
Needle In A Multimodal Haystack Weiyun Wang Shuibo Zhang Yiming Ren Yuchen Duan Tiantong Li ... Ping Luo Yu Qiao Jifeng Dai Wenqi Shao Wenhai Wang VLM 59 17 0 11 Jun 2024
iMotion-LLM: Motion Prediction Instruction Tuning Abdulwahab Felemban Eslam Mohamed Bakr Xiaoqian Shen Jian Ding Abduallah A. Mohamed Mohamed Elhoseiny 60 1 0 10 Jun 2024
Vript: A Video Is Worth Thousands of Words Dongjie Yang Suyuan Huang Chengqiang Lu Xiaodong Han Haoxin Zhang Yan Gao Yao Hu Hai Zhao VGen 77 22 0 10 Jun 2024
VP-LLM: Text-Driven 3D Volume Completion with Large Language Models through Patchification Jianmeng Liu Yichen Liu Yuyao Zhang Zeyuan Meng Yu-Wing Tai Chi-Keung Tang 49 0 0 08 Jun 2024
ShareGPT4Video: Improving Video Understanding and Generation with Better Captions Lin Chen Xilin Wei Jinsong Li Xiaoyi Dong Pan Zhang ... Li Yuan Yu Qiao Dahua Lin Feng Zhao Jiaqi Wang 72 142 0 06 Jun 2024
AD-H: Autonomous Driving with Hierarchical Agents Zaibin Zhang Shiyu Tang Yuanhang Zhang Talas Fu Yifan Wang Yang Liu Dong Wang Jing Shao Lijun Wang H. Lu 52 3 0 05 Jun 2024
Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis Chaoyou Fu Yuhan Dai Yondong Luo Lei Li Shuhuai Ren ... Tong Xu Xiawu Zheng Enhong Chen Rongrong Ji Xing Sun VLM MLLM 50 300 0 31 May 2024
MotionLLM: Understanding Human Behaviors from Human Motions and Videos Ling-Hao Chen Shunlin Lu Ailing Zeng Hao Zhang Benyou Wang Ruimao Zhang Lei Zhang 55 34 0 30 May 2024
SAM-E: Leveraging Visual Foundation Model with Sequence Imitation for Embodied Manipulation Junjie Zhang Chenjia Bai Haoran He Wenke Xia Zhigang Wang Bin Zhao Xiu Li Xuelong Li 40 12 0 30 May 2024
X-VILA: Cross-Modality Alignment for Large Language Model Hanrong Ye De-An Huang Yao Lu Zhiding Yu Ming-Yu Liu ... Jan Kautz Song Han Dan Xu Pavlo Molchanov Hongxu Yin MLLM VLM 45 29 0 29 May 2024
Cross-Modal Safety Alignment: Is textual unlearning all you need? Trishna Chakraborty Erfan Shayegani Zikui Cai Nael B. Abu-Ghazaleh M. Salman Asif Yue Dong A. Roy-Chowdhury Chengyu Song 41 16 0 27 May 2024
Hawk: Learning to Understand Open-World Video Anomalies Jiaqi Tang Hao Lu Ruizheng Wu Xiaogang Xu Ke Ma Cheng Fang Bin Guo Jiangbo Lu Qifeng Chen Ying-Cong Chen VLM 40 9 0 27 May 2024
Streaming Long Video Understanding with Large Language Models Rui Qian Xiao-wen Dong Pan Zhang Yuhang Zang Shuangrui Ding Dahua Lin Jiaqi Wang VLM 39 40 0 25 May 2024
Prompt-Aware Adapter: Towards Learning Adaptive Visual Tokens for Multimodal Large Language Models Yue Zhang Hehe Fan Yi Yang 53 3 0 24 May 2024
Continuously Learning, Adapting, and Improving: A Dual-Process Approach to Autonomous Driving Jianbiao Mei Yukai Ma Xuemeng Yang Licheng Wen Xinyu Cai ... Min Dou Botian Shi Liang He Yong-Jin Liu Yu Qiao 48 9 0 24 May 2024
A Survey on Vision-Language-Action Models for Embodied AI Yueen Ma Zixing Song Yuzheng Zhuang Jianye Hao Irwin King LM&Ro 82 42 0 23 May 2024
Dense Connector for MLLMs Huanjin Yao Wenhao Wu Taojiannan Yang Yuxin Song Mengxi Zhang Haocheng Feng Yifan Sun Zhiheng Li Wanli Ouyang Jingdong Wang MLLM VLM 42 16 0 22 May 2024
An Empirical Study and Analysis of Text-to-Image Generation Using Large Language Model-Powered Textual Representation Zhiyu Tan Mengping Yang Luozheng Qin Hao Yang Ye Qian Qiang-feng Zhou Cheng Zhang Hao Li 67 3 0 21 May 2024
Context-Enhanced Video Moment Retrieval with Large Language Models Weijia Liu Bo Miao Jiuxin Cao Xueling Zhu Bo Liu Mehwish Nasim Ajmal Saeed Mian 37 2 0 21 May 2024