Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models

8 June 2023

Salman Khan

Papers citing "Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models"

50 / 462 papers shown

Title
Neptune: The Long Orbit to Benchmarking Long Video Understanding Arsha Nagrani Ruotong Wang Ramin Mehran Rachel Hornung N. B. Gundavarapu ... Boqing Gong Cordelia Schmid Mikhail Sirotenko Yukun Zhu Tobias Weyand 114 4 0 12 Dec 2024
Dynamic-VLM: Simple Dynamic Visual Token Compression for VideoLLM Haozhao Wang Yuxiang Nie Yongjie Ye Deng GuanYu Yanjie Wang Shuai Li Haiyang Yu Jinghui Lu Can Huang VLM MLLM 84 1 0 12 Dec 2024
Foundation Models and Adaptive Feature Selection: A Synergistic Approach to Video Question Answering Sai Bhargav Rongali M. Cui Ankit Jha Neha Bhargava Saurabh Prasad Biplab Banerjee 86 0 0 12 Dec 2024
TimeRefine: Temporal Grounding with Time Refining Video LLM Xizi Wang Feng Cheng Ziyang Wang Huiyu Wang Md. Mohaiminul Islam Lorenzo Torresani Joey Tianyi Zhou Gedas Bertasius David J. Crandall 109 1 0 12 Dec 2024
EgoPlan-Bench2: A Benchmark for Multimodal Large Language Model Planning in Real-World Scenarios Lu Qiu Yuying Ge Yi Chen Yixiao Ge Ying Shan Xihui Liu LLMAG LRM 106 5 0 05 Dec 2024
AIM: Adaptive Inference of Multi-Modal LLMs via Token Merging and Pruning Yiwu Zhong Zhuoming Liu Yin Li Liwei Wang 90 3 0 04 Dec 2024
Video LLMs for Temporal Reasoning in Long Videos Fawad Javed Fateh Umer Ahmed Hamza Khan M. Zia Quoc-Huy Tran VLM 92 0 0 04 Dec 2024
PhysGame: Uncovering Physical Commonsense Violations in Gameplay Videos Meng Cao Haoran Tang Haoze Zhao Hangyu Guo Jing Liu Ge Zhang Ruyang Liu Qiang Sun Ian Reid Xiaodan Liang 106 2 0 02 Dec 2024
Beyond Text-Visual Attention: Exploiting Visual Cues for Effective Token Pruning in VLMs Qizhe Zhang Aosong Cheng Ming Lu Zhiyong Zhuo Minqi Wang Jiajun Cao Shaobo Guo Qi She Shanghang Zhang VLM 105 11 0 02 Dec 2024
SEAL: Semantic Attention Learning for Long Video Representation Lan Wang Yujia Chen Wen-Sheng Chu Vishnu Boddeti Du Tran VLM 83 0 0 02 Dec 2024
VideoSAVi: Self-Aligned Video Language Models without Human Supervision Yogesh Kulkarni Pooyan Fazli VLM 114 2 0 01 Dec 2024
ATP-LLaVA: Adaptive Token Pruning for Large Vision Language Models Xubing Ye Yukang Gan Yixiao Ge Xiao Zhang Yansong Tang 101 7 0 30 Nov 2024
HyperGLM: HyperGraph for Video Scene Graph Generation and Anticipation Trong-Thuan Nguyen Pha Nguyen J. Cothren Alper Yilmaz Khoa Luu 95 1 0 27 Nov 2024
TechCoach: Towards Technical-Point-Aware Descriptive Action Coaching Yuan-Ming Li An-Lan Wang Kun-Yu Lin Yu-Ming Tang Ling-an Zeng Jian-Fang Hu Wei-Shi Zheng 98 6 0 26 Nov 2024
Seq2Time: Sequential Knowledge Transfer for Video LLM Temporal Grounding Andong Deng Zhongpai Gao Anwesa Choudhuri Benjamin Planche Meng Zheng Bin Wang Terrence Chen Chong Chen Ziyan Wu AI4TS 83 1 0 25 Nov 2024
Video-Text Dataset Construction from Multi-AI Feedback: Promoting Weak-to-Strong Preference Learning for Video Large Language Models Hao Yi Qingyang Li Yihan Hu Fuzheng Zhang Di Zhang Yong Liu VGen 81 0 0 25 Nov 2024
Human-Activity AGV Quality Assessment: A Benchmark Dataset and an Objective Evaluation Metric Zhichao Zhang Wei Sun Xinyue Li Yunhao Li Qihang Ge ... Zhongpeng Ji Fengyu Sun Shangling Jui Xiongkuo Min Guangtao Zhai EGVM 122 1 0 25 Nov 2024
VideoOrion: Tokenizing Object Dynamics in Videos Yicheng Feng Yijiang Li Wanpeng Zhang Sipeng Zheng Zongqing Lu Sipeng Zheng Zongqing Lu 109 1 0 25 Nov 2024
All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages Ashmal Vayani Dinura Dissanayake Hasindri Watawana Noor Ahsan Nevasini Sasikumar ... Monojit Choudhury Ivan Laptev Mubarak Shah Salman Khan Fahad A Khan 124 9 0 25 Nov 2024
ReWind: Understanding Long Videos with Instructed Learnable Memory Anxhelo Diko Tinghuai Wang Wassim Swaileh Shiyan Sun Ioannis Patras KELM VLM 87 0 0 23 Nov 2024
Neuro-Symbolic Evaluation of Text-to-Video Models using Formal Verification S P Sharan Minkyu Choi Sahil Shah Harsh Goel Mohammad Omama Sandeep Chinchali EGVM 108 3 0 22 Nov 2024
Beyond Training: Dynamic Token Merging for Zero-Shot Video Understanding Yiming Zhang Zhuokai Zhao Zhaorun Chen Zenghui Ding Xianjun Yang Yining Sun 293 1 0 21 Nov 2024
Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehension Yongdong Luo Xiawu Zheng Xiao Yang Guilin Li Haojia Lin Jinfa Huang Jiayi Ji Rongrong Ji Jiebo Luo Rongrong Ji VLM 87 18 0 20 Nov 2024
On the Consistency of Video Large Language Models in Temporal Comprehension Minjoon Jung Junbin Xiao Byoung-Tak Zhang Angela Yao 89 2 0 20 Nov 2024
Generative Timelines for Instructed Visual Assembly Alejandro Pardo Jui-hsien Wang Guohao Li Josef Sivic Bryan C. Russell Fabian Caba Heilbron VGen 74 0 0 19 Nov 2024
TS-LLaVA: Constructing Visual Tokens through Thumbnail-and-Sampling for Training-Free Video Large Language Models Tingyu Qu Mingxiao Li Tinne Tuytelaars Marie-Francine Moens VLM 47 2 0 17 Nov 2024
Motion-Grounded Video Reasoning: Understanding and Perceiving Motion at Pixel Level Andong Deng Tongjia Chen Shoubin Yu Taojiannan Yang Lincoln Spencer Yapeng Tian Ajmal Mian Joey Tianyi Zhou Chen Chen LRM 68 1 0 15 Nov 2024
Multimodal Instruction Tuning with Hybrid State Space Models Jianing Zhou Han Li Shuai Zhang Ning Xie Ruijie Wang Xiaohan Nie Sheng Liu Lingyun Wang 46 0 0 13 Nov 2024
Artificial Intelligence for Biomedical Video Generation Linyuan Li Jianing Qiu Anujit Saha Lin Li Poyuan Li Mengxian He Ziyu Guo Wu Yuan VGen 68 1 0 12 Nov 2024
EVQAScore: A Fine-grained Metric for Video Question Answering Data Quality Evaluation Hao Liang Zirong Chen Feiyu Xiong Wentao Zhang 41 1 0 11 Nov 2024
Towards Multi-Modal Mastery: A 4.5B Parameter Truly Multi-Modal Small Language Model Ben Koska Mojmír Horváth MoE 42 1 0 08 Nov 2024
HourVideo: 1-Hour Video-Language Understanding Keshigeyan Chandrasegaran Agrim Gupta Lea M. Hadzic Taran Kota Jimming He Cristobal Eyzaguirre Zane Durante Manling Li Jiajun Wu L. Fei-Fei VLM 56 35 0 07 Nov 2024
VideoGLaMM: A Large Multimodal Model for Pixel-Level Visual Grounding in Videos Shehan Munasinghe Hanan Gani Wenqi Zhu Jiale Cao Eric P. Xing Fahad Shahbaz Khan Salman Khan MLLM VGen VLM 44 6 0 07 Nov 2024
Benchmarking Vision Language Model Unlearning via Fictitious Facial Identity Dataset Yingzi Ma Jiongxiao Wang Fei Wang Siyuan Ma Jiazhao Li ... B. Li Yejin Choi Mengzhao Chen Chaowei Xiao Chaowei Xiao MU 63 6 0 05 Nov 2024
PPLLaVA: Varied Video Sequence Understanding With Prompt Guidance Ruyang Liu Haoran Tang Haibo Liu Yixiao Ge Ying Shan Chen Li Jiankun Yang VLM 53 6 0 04 Nov 2024
LLaMo: Large Language Model-based Molecular Graph Assistant Jinyoung Park Minseong Bae Dohwan Ko Hyunwoo J. Kim 42 1 0 31 Oct 2024
FLAASH: Flow-Attention Adaptive Semantic Hierarchical Fusion for Multi-Modal Tobacco Content Analysis N. V. R. Chappa P. Dobbs Bhiksha Raj Khoa Luu 47 3 0 25 Oct 2024
TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuning Xiangyu Zeng Kunchang Li Chenting Wang Xinhao Li Tianxiang Jiang ... Zhengrong Yue Yi Wang Yali Wang Yu Qiao Limin Wang MLLM VLM AI4TS 71 15 0 25 Oct 2024
ADEM-VL: Adaptive and Embedded Fusion for Efficient Vision-Language Tuning Zhiwei Hao Jianyuan Guo Li Shen Yong Luo Han Hu Yonggang Wen VLM 31 0 0 23 Oct 2024
PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction Long Xing Qidong Huang Xiaoyi Dong Jiajie Lu Pan Zhang ... Yuhang Cao Zeang Sheng Jiaqi Wang Feng Wu Dahua Lin VLM 53 29 0 22 Oct 2024
xGen-MM-Vid (BLIP-3-Video): You Only Need 32 Tokens to Represent a Video Even in VLMs Michael S Ryoo Honglu Zhou Shrikant B. Kendre Can Qin Le Xue Manli Shu Silvio Savarese Ran Xu Caiming Xiong Juan Carlos Niebles VGen 53 13 0 21 Oct 2024
EVA: An Embodied World Model for Future Video Anticipation Xiaowei Chi Hengyuan Zhang Chun-Kai Fan Xingqun Qi Rongyu Zhang ... Chi-Min Chan Wei Xue Wenhan Luo Shanghang Zhang Yike Guo VGen 46 5 0 20 Oct 2024
FIOVA: A Multi-Annotator Benchmark for Human-Aligned Video Captioning Shiyu Hu Xuchen Li Xuzhao Li Jing Zhang Yipei Wang Xin Zhao Kang Hao Cheong VLM 31 1 0 20 Oct 2024
Making Every Frame Matter: Continuous Activity Recognition in Streaming Video via Adaptive Video Context Modeling Hao Wu Donglin Bai Shiqi Jiang Qianxi Zhang Yue Yang Ting Cao Fengyuan Xu Yunxin Liu Fengyuan Xu 190 0 0 19 Oct 2024
Exploring the Design Space of Visual Context Representation in Video MLLMs Yifan Du Yuqi Huo K. Zhou Zijia Zhao Haoyu Lu Han Huang Wayne Xin Zhao Bin Wang Weipeng Chen Zhicheng Dou 38 2 0 17 Oct 2024
OMCAT: Omni Context Aware Transformer Arushi Goel Karan Sapra Matthieu Le Rafael Valle Andrew Tao Bryan Catanzaro MLLM VLM 41 0 0 15 Oct 2024
Character-aware audio-visual subtitling in context Jaesung Huh Andrew Zisserman 43 0 0 14 Oct 2024
Free Video-LLM: Prompt-guided Visual Perception for Efficient Training-free Video LLMs Kai Han Jianyuan Guo Yehui Tang W. He Enhua Wu Yunhe Wang MLLM VLM 21 3 0 14 Oct 2024
Depth Any Video with Scalable Synthetic Data Honghui Yang Di Huang Wei Yin Chunhua Shen Haifeng Liu Xiaofei He Binbin Lin Wanli Ouyang Tong He VGen MDE 31 16 0 14 Oct 2024
Surgical-LLaVA: Toward Surgical Scenario Understanding via Large Language and Vision Models Juseong Jin Chang Wook Jeong 33 3 0 13 Oct 2024