Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding

5 June 2023

Xin Li

Papers citing "Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding"

50 / 703 papers shown

Title
VideoPrism: A Foundational Visual Encoder for Video Understanding Long Zhao N. B. Gundavarapu Liangzhe Yuan Hao Zhou Shen Yan ... Huisheng Wang Hartwig Adam Mikhail Sirotenko Ting Liu Boqing Gong VGen 50 29 0 20 Feb 2024
Slot-VLM: SlowFast Slots for Video-Language Modeling Jiaqi Xu Cuiling Lan Wenxuan Xie Xuejin Chen Yan Lu MLLM VLM 35 7 0 20 Feb 2024
Model Composition for Multimodal Large Language Models Chi Chen Yiyang Du Zheng Fang Ziyue Wang Ziyue Wang ... Ming Yan Ji Zhang Fei Huang Maosong Sun Yang Liu MoMe 28 3 0 20 Feb 2024
The Revolution of Multimodal Large Language Models: A Survey Davide Caffagni Federico Cocchi Luca Barsellotti Nicholas Moratelli Sara Sarto Lorenzo Baraldi Lorenzo Baraldi Marcella Cornia Rita Cucchiara LRM VLM 66 43 0 19 Feb 2024
LVCHAT: Facilitating Long Video Comprehension Yu Wang Zeyuan Zhang Julian McAuley Zexue He VLM 32 4 0 19 Feb 2024
Model Tailor: Mitigating Catastrophic Forgetting in Multi-modal Large Language Models Didi Zhu Zhongyi Sun Zexi Li Tao Shen Ke Yan Shouhong Ding Kun Kuang Chao Wu CLL KELM MoMe 77 25 0 19 Feb 2024
CoCo-Agent: A Comprehensive Cognitive MLLM Agent for Smartphone GUI Automation Xinbei Ma Zhuosheng Zhang Hai Zhao LLMAG 41 23 0 19 Feb 2024
Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoning Long Qian Juncheng Billy Li Yu-hao Wu Yaobo Ye Hao Fei Tat-Seng Chua Yueting Zhuang Siliang Tang MLLM LRM 65 47 0 18 Feb 2024
RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Model Jianhao Yuan Shuyang Sun Daniel Omeiza Bo Zhao Paul Newman Lars Kunze Matthew Gadd LRM 41 51 0 16 Feb 2024
Rec-GPT4V: Multimodal Recommendation with Large Vision-Language Models Yuqing Liu Yu Wang Lichao Sun Philip S. Yu 27 6 0 13 Feb 2024
World Model on Million-Length Video And Language With Blockwise RingAttention Hao Liu Wilson Yan Matei A. Zaharia Pieter Abbeel VGen 44 64 0 13 Feb 2024
Walia-LLM: Enhancing Amharic-LLaMA by Integrating Task-Specific and Generative Datasets Israel Abebe Azime A. Tonja Tadesse Destaw Belay Mitiku Yohannes Fuge A. Wassie Eyasu Shiferaw Jada Yonas Chanie W. Sewunetie Seid Muhie Yimam 27 1 0 12 Feb 2024
Unsupervised Sign Language Translation and Generation Zhengsheng Guo Zhiwei He Wenxiang Jiao Xing Wang Rui Wang Kehai Chen Zhaopeng Tu Yong-mei Xu Min Zhang 76 0 0 12 Feb 2024
It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognition Chen Chen Ruizhe Li Yuchen Hu Sabato Marco Siniscalchi Pin-Yu Chen Ensiong Chng Chao-Han Huck Yang 43 20 0 08 Feb 2024
CREMA: Generalizable and Efficient Video-Language Reasoning via Multimodal Modular Fusion Shoubin Yu Jaehong Yoon Mohit Bansal 85 4 0 08 Feb 2024
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models Chris Liu Renrui Zhang Longtian Qiu Siyuan Huang Weifeng Lin ... Hao Shao Pan Lu Hongsheng Li Yu Qiao Peng Gao MLLM 130 110 0 08 Feb 2024
RA-Rec: An Efficient ID Representation Alignment Framework for LLM-based Recommendation Xiaohan Yu Li Zhang Xin Zhao Yue Wang Zhongrui Ma 58 10 0 07 Feb 2024
Sentiment-enhanced Graph-based Sarcasm Explanation in Dialogue Kun Ouyang Liqiang Jing Xuemeng Song Meng Liu Yupeng Hu Liqiang Nie 99 3 0 06 Feb 2024
Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenization Yang Jin Zhicheng Sun Kun Xu Kun Xu Liwei Chen ... Yuliang Liu Di Zhang Yang Song Kun Gai Yadong Mu VGen 55 42 0 05 Feb 2024
Delving into Multi-modal Multi-task Foundation Models for Road Scene Understanding: From Learning Paradigm Perspectives Sheng Luo Wei Chen Wanxin Tian Rui Liu Luanxuan Hou ... Ling Shao Yi Yang Bojun Gao Qun Li Guobin Wu 58 13 0 05 Feb 2024
A Survey for Foundation Models in Autonomous Driving Haoxiang Gao Yaqian Li Kaiwen Long Ming Yang Yiqing Shen VLM LRM 58 25 0 02 Feb 2024
Image Anything: Towards Reasoning-coherent and Training-free Multi-modal Image Generation Yuanhuiyi Lyu Xueye Zheng Lin Wang DiffM 49 11 0 31 Jan 2024
A Survey on Generative AI and LLM for Video Generation, Understanding, and Streaming Pengyuan Zhou Lin Wang Zhi Liu Yanbin Hao Pan Hui Sasu Tarkoma J. Kangasharju VGen 54 27 0 30 Jan 2024
Towards 3D Molecule-Text Interpretation in Language Models Changhao Nai Zhiyuan Liu Yancheng Luo Xiang Wang Xiangnan He Kenji Kawaguchi Tat-Seng Chua Qi Tian AI4CE 40 44 0 25 Jan 2024
MM-LLMs: Recent Advances in MultiModal Large Language Models Duzhen Zhang Yahan Yu Jiahua Dong Chenxing Li Dan Su Chenhui Chu Dong Yu OffRL LRM 56 183 0 24 Jan 2024
Mementos: A Comprehensive Benchmark for Multimodal Large Language Model Reasoning over Image Sequences Xiyao Wang Yuhang Zhou Xiaoyu Liu Hongjin Lu Yuancheng Xu ... Taixi Lu Gedas Bertasius Mohit Bansal Huaxiu Yao Furong Huang LRM VLM 99 65 0 19 Jan 2024
Large Language Models are Efficient Learners of Noise-Robust Speech Recognition Yuchen Hu Chen Chen Chao-Han Huck Yang Ruizhe Li Chao Zhang Pin-Yu Chen Ensiong Chng 32 21 0 19 Jan 2024
MLLM-Tool: A Multimodal Large Language Model For Tool Agent Learning Chenyu Wang Weixin Luo Qianyu Chen Haonan Mai Jindi Guo Sixun Dong Xiaohua Xuan MLLM LLMAG 52 19 0 19 Jan 2024
Temporal Insight Enhancement: Mitigating Temporal Hallucination in Multimodal Large Language Models Li Sun Liuan Wang Jun Sun Takayuki Okatani MLLM 24 0 0 18 Jan 2024
On the Audio Hallucinations in Large Audio-Video Language Models Taichi Nishimura Shota Nakada Masayoshi Kondo VLM 30 5 0 18 Jan 2024
MMToM-QA: Multimodal Theory of Mind Question Answering Chuanyang Jin Yutong Wu Jing Cao Jiannan Xiang Yen-Ling Kuo Zhiting Hu T. Ullman Antonio Torralba Joshua B. Tenenbaum Tianmin Shu 44 34 0 16 Jan 2024
Towards A Better Metric for Text-to-Video Generation Jay Zhangjie Wu Guian Fang Haoning Wu Xintao Wang Yixiao Ge ... Rui Zhao Weisi Lin Wynne Hsu Ying Shan Mike Zheng Shou VGen 46 34 0 15 Jan 2024
GroundingGPT:Language Enhanced Multi-modal Grounding Model Zhaowei Li Qi Xu Dong Zhang Hang Song Yiqing Cai ... Junting Pan Zefeng Li Van Tu Vu Zhida Huang Tao Wang 43 38 0 11 Jan 2024
Video Anomaly Detection and Explanation via Large Language Models Hui Lv Qianru Sun 35 21 0 11 Jan 2024
SonicVisionLM: Playing Sound with Vision Language Models Zhifeng Xie Shengye Yu Qile He Mengtian Li VLM VGen 28 2 0 09 Jan 2024
STAIR: Spatial-Temporal Reasoning with Auditable Intermediate Results for Video Question Answering Yueqian Wang Yuxuan Wang Kai Chen Dongyan Zhao 38 2 0 08 Jan 2024
LightHouse: A Survey of AGI Hallucination Feng Wang LRM HILM VLM 37 3 0 08 Jan 2024
Towards Truly Zero-shot Compositional Visual Reasoning with LLMs as Programmers Aleksandar Stanić Sergi Caelles Michael Tschannen LRM VLM 27 9 0 03 Jan 2024
Detours for Navigating Instructional Videos Kumar Ashutosh Zihui Xue Tushar Nagarajan Kristen Grauman 34 6 0 03 Jan 2024
Holistic Autonomous Driving Understanding by Bird's-Eye-View Injected Multi-Modal Large Models Xinpeng Ding Jinahua Han Hang Xu Xiaodan Liang Wei Zhang Xiaomeng Li 46 39 0 02 Jan 2024
E-chat: Emotion-sensitive Spoken Dialogue System with Large Language Models Hongfei Xue Yuhao Liang Bingshen Mu Shiliang Zhang Mengzhe Chen Qian Chen Lei Xie AuLLM 29 10 0 31 Dec 2023
Boosting Large Language Model for Speech Synthesis: An Empirical Study Hong-ping Hao Long Zhou Shujie Liu Jinyu Li Shujie Hu Rui Wang Furu Wei 34 18 0 30 Dec 2023
Video Understanding with Large Language Models: A Survey Yunlong Tang Jing Bi Siting Xu Luchuan Song Susan Liang ... Feng Zheng Jianguo Zhang Ping Luo Jiebo Luo Chenliang Xu VLM 70 84 0 29 Dec 2023
Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action Jiasen Lu Christopher Clark Sangho Lee Zichen Zhang Savya Khosla Ryan Marten Derek Hoiem Aniruddha Kembhavi VLM MLLM 40 147 0 28 Dec 2023
Grounding-Prompter: Prompting LLM with Multimodal Information for Temporal Sentence Grounding in Long Videos Houlun Chen Xin Wang Hong Chen Zihan Song Jia Jia Wenwu Zhu LRM 49 10 0 28 Dec 2023
Visual Instruction Tuning towards General-Purpose Multimodal Model: A Survey Jiaxing Huang Jingyi Zhang Kai Jiang Han Qiu Shijian Lu 44 22 0 27 Dec 2023
InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks Zhe Chen Jiannan Wu Wenhai Wang Weijie Su Guo Chen ... Bin Li Ping Luo Tong Lu Yu Qiao Jifeng Dai VLM MLLM 176 972 0 21 Dec 2023
LLM4VG: Large Language Models Evaluation for Video Grounding Wei Feng Xin Wang Hong Chen Zeyang Zhang Zihan Song Yuwei Zhou Wenwu Zhu 44 8 0 21 Dec 2023
Perception Test 2023: A Summary of the First Challenge And Outcome Joseph Heyward João Carreira Dima Damen Andrew Zisserman Viorica Patraucean 18 0 0 20 Dec 2023
Text-Conditioned Resampler For Long Form Video Understanding Bruno Korbar Yongqin Xian A. Tonioni Andrew Zisserman Federico Tombari 43 12 0 19 Dec 2023