VideoChat: Chat-Centric Video Understanding

10 May 2023

Yi Wang

Ping Luo

Yu Qiao

Papers citing "VideoChat: Chat-Centric Video Understanding"

50 / 425 papers shown

Title
Imp: Highly Capable Large Multimodal Models for Mobile Devices Zhenwei Shao Zhou Yu Jun Yu Xuecheng Ouyang Lihao Zheng Zhenbiao Gai Mingyang Wang Jiajun Ding 21 10 0 20 May 2024
Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts Yunxin Li Shenyuan Jiang Baotian Hu Longyue Wang Wanqi Zhong Wenhan Luo Lin Ma Min-Ling Zhang MoE 46 28 0 18 May 2024
Efficient Multimodal Large Language Models: A Survey Yizhang Jin Jian Li Yexin Liu Tianjun Gu Kai Wu ... Xin Tan Zhenye Gan Yabiao Wang Chengjie Wang Lizhuang Ma LRM 47 45 0 17 May 2024
SOK-Bench: A Situated Video Reasoning Benchmark with Aligned Open-World Knowledge Andong Wang Bo Wu Sunli Chen Zhenfang Chen Haotian Guan Wei-Ning Lee Li Erran Li Chuang Gan LRM RALM 29 16 0 15 May 2024
FreeVA: Offline MLLM as Training-Free Video Assistant Wenhao Wu VLM OffRL 40 19 0 13 May 2024
DoLLM: How Large Language Models Understanding Network Flow Data to Detect Carpet Bombing DDoS Qingyang Li Yihang Zhang Zhidong Jia Yannan Hu Lei Zhang Jianrong Zhang Yongming Xu Yong Cui Zongming Guo Xinggong Zhang AI4CE 42 7 0 13 May 2024
How Good is my Video LMM? Complex Video Reasoning and Robustness Evaluation Suite for Video-LMMs Muhammad Uzair Khattak Muhammad Ferjad Naeem Jameel Hassan Muzammal Naseer Federico Tombari Fahad Shahbaz Khan Salman Khan LRM ELM 42 10 0 06 May 2024
WorldQA: Multimodal World Knowledge in Videos through Long-Chain Reasoning Yuanhan Zhang Kaichen Zhang Bo-wen Li Fanyi Pu Christopher Arif Setiadharma Jingkang Yang Ziwei Liu VGen 50 7 0 06 May 2024
Octopi: Object Property Reasoning with Large Tactile-Language Models Samson Yu Kelvin Lin Anxing Xiao Jiafei Duan Harold Soh LRM 36 26 0 05 May 2024
Uncovering What, Why and How: A Comprehensive Benchmark for Causation Understanding of Video Anomaly Hang Du Sicheng Zhang Binzhu Xie Gu Nan Jiayang Zhang ... Xuhuan Li Hao Zhang Jianhang Chen Qimei Cui Xiao Tao CML 29 15 0 30 Apr 2024
MovieChat+: Question-aware Sparse Memory for Long Video Question Answering Enxin Song Wenhao Chai Tianbo Ye Jenq-Neng Hwang Xi Li Gaoang Wang VLM MLLM 37 30 0 26 Apr 2024
MER 2024: Semi-Supervised Learning, Noise Robustness, and Open-Vocabulary Multimodal Emotion Recognition Zheng Lian Haiyang Sun Guoying Zhao Zhuofan Wen Siyuan Zhang ... Bin Liu Erik Cambria Guoying Zhao Björn W. Schuller Jianhua Tao VLM 33 11 0 26 Apr 2024
PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioning Lin Xu Yilin Zhao Daquan Zhou Zhijie Lin See Kiong Ng Jiashi Feng MLLM VLM 38 159 0 25 Apr 2024
How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites Zhe Chen Weiyun Wang Hao Tian Shenglong Ye Zhangwei Gao ... Tong Lu Dahua Lin Yu Qiao Jifeng Dai Wenhai Wang MLLM VLM 49 533 0 25 Apr 2024
SEED-Bench-2-Plus: Benchmarking Multimodal Large Language Models with Text-Rich Visual Comprehension Bohao Li Yuying Ge Yi Chen Yixiao Ge Ruimao Zhang Ying Shan VLM 39 42 0 25 Apr 2024
Pegasus-v1 Technical Report Raehyuk Jung Hyojun Go Jaehyuk Yi Jiho Jang Daniel Kim ... Maninder Saini Meredith Sanders Soyoung Lee Sue Kim Travis Couture MLLM VLM 29 5 0 23 Apr 2024
Graphic Design with Large Multimodal Model Yutao Cheng Zhao Zhang Maoke Yang Hui Nie Chunyuan Li Xinglong Wu Jie Shao 52 10 0 22 Apr 2024
From Image to Video, what do we need in multimodal LLMs? Suyuan Huang Haoxin Zhang Yan Gao Honggu Chen Yan Gao Yao Hu Zhanyue Qin VLM 41 8 0 18 Apr 2024
HOI-Ref: Hand-Object Interaction Referral in Egocentric Vision Siddhant Bansal Michael Wray Dima Damen 41 3 0 15 Apr 2024
Learning to Localize Objects Improves Spatial Reasoning in Visual-LLMs Kanchana Ranasinghe Satya Narayan Shukla Omid Poursaeed Michael S. Ryoo Tsung-Yu Lin LRM 49 23 0 11 Apr 2024
MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding Bo He Hengduo Li Young Kyun Jang Menglin Jia Xuefei Cao Ashish Shah Abhinav Shrivastava Ser-Nam Lim MLLM 83 88 0 08 Apr 2024
JRDB-Social: A Multifaceted Robotic Dataset for Understanding of Context and Dynamics of Human Interactions Within Social Groups Simindokht Jahangard Zhixi Cai Shiki Wen Hamid Rezatofighi 36 6 0 06 Apr 2024
Koala: Key frame-conditioned long video-LLM Reuben Tan Ximeng Sun Ping Hu Jui-hsien Wang Hanieh Deilamsalehy Bryan A. Plummer Bryan C. Russell Kate Saenko 38 35 0 05 Apr 2024
SemGrasp: Semantic Grasp Generation via Language Aligned Discretization Kailin Li Jingbo Wang Lixin Yang Cewu Lu Bo Dai 46 16 0 04 Apr 2024
LongVLM: Efficient Long Video Understanding via Large Language Models Yuetian Weng Mingfei Han Haoyu He Xiaojun Chang Bohan Zhuang VLM 68 56 0 04 Apr 2024
Direct Preference Optimization of Video Large Multimodal Models from Language Model Reward Ruohong Zhang Liangke Gui Zhiqing Sun Yihao Feng Keyang Xu ... Di Fu Chunyuan Li Alexander G. Hauptmann Yonatan Bisk Yiming Yang MLLM 53 57 0 01 Apr 2024
ST-LLM: Large Language Models Are Effective Temporal Learners Ruyang Liu Chen Li Haoran Tang Yixiao Ge Ying Shan Ge Li 43 70 0 30 Mar 2024
LITA: Language Instructed Temporal-Localization Assistant De-An Huang Shijia Liao Subhashree Radhakrishnan Hongxu Yin Pavlo Molchanov Zhiding Yu Jan Kautz VLM 45 49 0 27 Mar 2024
An Image Grid Can Be Worth a Video: Zero-shot Video Question Answering Using a VLM Wonkyun Kim Changin Choi Wonseok Lee Wonjong Rhee VLM 47 51 0 27 Mar 2024
Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning Hao Shao Shengju Qian Han Xiao Guanglu Song Zhuofan Zong Letian Wang Yu Liu Hongsheng Li VGen LRM MLLM 63 37 0 25 Mar 2024
Elysium: Exploring Object-level Perception in Videos via MLLM Hang Wang Yanjie Wang Yongjie Ye Yuxiang Nie Can Huang MLLM 42 19 0 25 Mar 2024
AVicuna: Audio-Visual LLM with Interleaver and Context-Boundary Alignment for Temporal Referential Dialogue Yunlong Tang Daiki Shimada Jing Bi Chenliang Xu VGen 39 10 0 24 Mar 2024
InternVideo2: Scaling Video Foundation Models for Multimodal Video Understanding Yi Wang Kunchang Li Xinhao Li Jiashuo Yu Yinan He ... Hongjie Zhang Yifei Huang Yu Qiao Yali Wang Limin Wang 39 47 0 22 Mar 2024
FMM-Attack: A Flow-based Multi-modal Adversarial Attack on Video-based LLMs Jinmin Li Kuofeng Gao Yang Bai Jingyun Zhang Shu-Tao Xia Yisen Wang AAML 30 7 0 20 Mar 2024
RelationVLM: Making Large Vision-Language Models Understand Visual Relations Zhipeng Huang Zhizheng Zhang Zheng-Jun Zha Yan Lu Baining Guo VLM 36 3 0 19 Mar 2024
Contextual AD Narration with Interleaved Multimodal Sequence Hanlin Wang Zhan Tong Kecheng Zheng Yujun Shen Limin Wang VGen 57 4 0 19 Mar 2024
HawkEye: Training Video-Text LLMs for Grounding Text in Videos Yueqian Wang Xiaojun Meng Jianxin Liang Yuxuan Wang Qun Liu Dongyan Zhao 34 30 0 15 Mar 2024
GPT as Psychologist? Preliminary Evaluations for GPT-4V on Visual Affective Computing Hao Lu Xuesong Niu Jiyao Wang Yin Wang Qingyong Hu ... Dengbo He Shuiguang Deng Hao Chen Ying Chen Shiguang Shan MLLM 54 11 0 09 Mar 2024
CAT: Enhancing Multimodal Large Language Model to Answer Questions in Dynamic Audio-Visual Scenarios Qilang Ye Zitong Yu Rui Shao Xinyu Xie Philip H. S. Torr Xiaochun Cao MLLM 47 24 0 07 Mar 2024
Embodied Understanding of Driving Scenarios Yunsong Zhou Linyan Huang Qingwen Bu Jia Zeng Tianyu Li Hang Qiu Hongzi Zhu Minyi Guo Yu Qiao Hongyang Li LM&Ro 62 31 0 07 Mar 2024
GPTSee: Enhancing Moment Retrieval and Highlight Detection via Description-Based Similarity Features Yunzhuo Sun Yifang Xu Zien Xie Yukun Shu Sidan Du 30 6 0 03 Mar 2024
Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers Tsai-Shien Chen Aliaksandr Siarohin Willi Menapace Ekaterina Deyneka Hsiang-wei Chao ... Yuwei Fang Hsin-Ying Lee Jian Ren Ming-Hsuan Yang Sergey Tulyakov VGen 86 178 0 29 Feb 2024
The All-Seeing Project V2: Towards General Relation Comprehension of the Open World Weiyun Wang Yiming Ren Hao Luo Tiantong Li Chenxiang Yan ... Qingyun Li Lewei Lu Xizhou Zhu Yu Qiao Jifeng Dai MLLM 52 47 0 29 Feb 2024
Navigating Hallucinations for Reasoning of Unintentional Activities Shresth Grover Vibhav Vineet Y. S. Rawat LRM 50 1 0 29 Feb 2024
Percept, Chat, and then Adapt: Multimodal Knowledge Transfer of Foundation Models for Open-World Video Recognition Boyu Chen Siran Chen Kunchang Li Qinglin Xu Yu Qiao Yali Wang 34 3 0 29 Feb 2024
OSCaR: Object State Captioning and State Change Representation Nguyen Nguyen Jing Bi A. Vosoughi Yapeng Tian Pooyan Fazli Chenliang Xu 40 8 0 27 Feb 2024
PhyGrasp: Generalizing Robotic Grasping with Physics-informed Large Multimodal Models Dingkun Guo Yuqi Xiang Shuqi Zhao Xinghao Zhu Masayoshi Tomizuka Mingyu Ding Wei Zhan 32 10 0 26 Feb 2024
RoboCodeX: Multimodal Code Generation for Robotic Behavior Synthesis Yao Mu Junting Chen Qinglong Zhang Shoufa Chen Qiaojun Yu ... Wenhai Wang Jifeng Dai Yu Qiao Mingyu Ding Ping Luo 42 21 0 25 Feb 2024
Slot-VLM: SlowFast Slots for Video-Language Modeling Jiaqi Xu Cuiling Lan Wenxuan Xie Xuejin Chen Yan Lu MLLM VLM 35 7 0 20 Feb 2024
Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoning Long Qian Juncheng Billy Li Yu-hao Wu Yaobo Ye Hao Fei Tat-Seng Chua Yueting Zhuang Siliang Tang MLLM LRM 60 47 0 18 Feb 2024