Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding

5 June 2023

Xin Li

Papers citing "Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding"

50 / 703 papers shown

Title
KwaiYiiMath: Technical Report Jia-Yi Fu Lei Lin Xiaoyang Gao Pengli Liu Zhengzong Chen ... Zijia Lin Fuzheng Zhang Zhongyuan Wang Di Zhang Kun Gai LRM ReLM RALM 56 2 0 11 Oct 2023
Uncovering Hidden Connections: Iterative Search and Reasoning for Video-grounded Dialog Haoyu Zhang Meng Liu Yaowei Wang Da Cao Weili Guan Liqiang Nie 41 0 0 11 Oct 2023
MuseChat: A Conversational Music Recommendation System for Videos Zhikang Dong Bin Chen Xiulong Liu Paweł Polak Peng Zhang LRM 50 26 0 10 Oct 2023
FireAct: Toward Language Agent Fine-tuning Baian Chen Chang Shu Ehsan Shareghi Nigel Collier Karthik Narasimhan Shunyu Yao ALM LLMAG 107 98 0 09 Oct 2023
Fine-grained Audio-Visual Joint Representations for Multimodal Large Language Models Guangzhi Sun Wenyi Yu Changli Tang Xianzhao Chen Tian Tan Wei Li Lu Lu Zejun Ma Chao Zhang 41 12 0 09 Oct 2023
Video-Teller: Enhancing Cross-Modal Generation with Fusion and Decoupling Haogeng Liu Qihang Fan Tingkai Liu Linjie Yang Yunzhe Tao Huaibo Huang Ran He Hongxia Yang VGen 29 12 0 08 Oct 2023
Expedited Training of Visual Conditioned Language Generation via Redundancy Reduction Yiren Jian Tingkai Liu Yunzhe Tao Chunhui Zhang Soroush Vosoughi HX Yang VLM 25 7 0 05 Oct 2023
LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving Hao Sha Yao Mu Yuxuan Jiang Li Chen Chenfeng Xu Ping Luo Shengbo Eben Li Masayoshi Tomizuka Wei Zhan Mingyu Ding 140 162 0 04 Oct 2023
DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model Zhenhua Xu Yujia Zhang Enze Xie Zhen Zhao Yong Guo Kwan-Yee. K. Wong Zhenguo Li Hengshuang Zhao MLLM 22 257 0 02 Oct 2023
LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examples Jia-Yu Yao Kun-Peng Ning Zhen-Hui Liu Munan Ning Li Yuan HILM LRM AAML 34 177 0 02 Oct 2023
AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Model Avamarie Brueggeman Andrea Madotto Zhaojiang Lin Tushar Nagarajan Matt Smith ... Peyman Heidari Yue Liu Kavya Srinet Babak Damavandi Anuj Kumar MLLM 39 93 0 27 Sep 2023
BT-Adapter: Video Conversation is Feasible Without Video Instruction Tuning Ruyang Liu Chen Li Yixiao Ge Ying Shan Thomas H. Li Ge Li 25 29 0 27 Sep 2023
MSG-BART: Multi-granularity Scene Graph-Enhanced Encoder-Decoder Language Model for Video-grounded Dialogue Generation Hongcheng Liu Zhe Chen Hui Li Pingjie Wang Yanfeng Wang Yu Wang VGen 51 2 0 26 Sep 2023
Connecting Speech Encoder and Large Language Model for ASR Wenyi Yu Changli Tang Guangzhi Sun Xianzhao Chen T. Tan Wei Li Lu Lu Zejun Ma Chao Zhang AuLLM 15 67 0 25 Sep 2023
Investigating the Catastrophic Forgetting in Multimodal Large Language Models Yuexiang Zhai Shengbang Tong Xiao Li Mu Cai Qing Qu Yong Jae Lee Yi Ma VLM MLLM CLL 77 78 0 19 Sep 2023
MusiLingo: Bridging Music and Text with Pre-trained Language Models for Music Captioning and Query Response Zihao Deng Yi Ma Yudong Liu Rongchen Guo Ge Zhang Wenhu Chen Wenhao Huang Emmanouil Benetos MLLM AuLLM 34 20 0 15 Sep 2023
Knowledge-Guided Short-Context Action Anticipation in Human-Centric Videos Sarthak Bhagat Simon Stepputtis Joseph Campbell Katia Sycara 33 4 0 12 Sep 2023
NExT-GPT: Any-to-Any Multimodal LLM Shengqiong Wu Hao Fei Leigang Qu Wei Ji Tat-Seng Chua MLLM 51 461 0 11 Sep 2023
ImageBind-LLM: Multi-modality Instruction Tuning Jiaming Han Renrui Zhang Wenqi Shao Peng Gao Peng Xu ... Yafei Wen Xiaoxin Chen Xiangyu Yue Hongsheng Li Yu Qiao MLLM 54 117 0 07 Sep 2023
Can I Trust Your Answer? Visually Grounded Video Question Answering Junbin Xiao Angela Yao Yicong Li Tat-Seng Chua 50 46 0 04 Sep 2023
Large Content And Behavior Models To Understand, Simulate, And Optimize Content And Behavior Ashmit Khandelwal Aditya Agrawal Aanisha Bhattacharyya Yaman Kumar Singla Somesh Singh ... Ishita Dasgupta Stefano Petrangeli R. Shah Changyou Chen Balaji Krishnamurthy 32 8 0 01 Sep 2023
Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond Jinze Bai Shuai Bai Shusheng Yang Shijie Wang Sinan Tan Peng Wang Junyang Lin Chang Zhou Jingren Zhou MLLM VLM ObjD 50 817 0 24 Aug 2023
Instruction Tuning for Large Language Models: A Survey Shengyu Zhang Linfeng Dong Xiaoya Li Sen Zhang Xiaofei Sun ... Jiwei Li Runyi Hu Tianwei Zhang Fei Wu Guoyin Wang LM&MA 29 551 0 21 Aug 2023
FashionLOGO: Prompting Multimodal Large Language Models for Fashion Logo Embeddings Yulin Su Min Yang Minghui Qiu Jing Wang Tao Wang VLM 40 0 0 17 Aug 2023
Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenes Zehan Wang Haifeng Huang Yang Zhao Ziang Zhang Zhou Zhao 19 62 0 17 Aug 2023
OctoPack: Instruction Tuning Code Large Language Models Niklas Muennighoff Qian Liu A. Zebaze Qinkai Zheng Binyuan Hui Terry Yue Zhuo Swayam Singh Xiangru Tang Leandro von Werra Shayne Longpre VLM ALM 73 120 0 14 Aug 2023
Fine-Tune Language Models as Multi-Modal Differential Equation Solvers Liu Yang Siting Liu Stanley J. Osher 27 0 0 09 Aug 2023
RegionBLIP: A Unified Multi-modal Pre-training Framework for Holistic and Regional Comprehension Qiang-feng Zhou Chaohui Yu Shaofeng Zhang Sitong Wu Zhibin Wang Fan Wang 34 27 0 03 Aug 2023
NLLG Quarterly arXiv Report 06/23: What are the most influential current AI Papers? Steffen Eger Christoph Leiter Jonas Belouadi Ran Zhang Aida Kostikova Daniil Larionov Yanran Chen Vivian Fresen AI4CE 37 4 0 31 Jul 2023
MovieChat: From Dense Token to Sparse Memory for Long Video Understanding Enxin Song Wenhao Chai Guanhong Wang Yucheng Zhang Haoyang Zhou ... Tianbo Ye Yanting Zhang Yang Lu Lei Li Gaoang Wang VLM MLLM 27 266 0 31 Jul 2023
BuboGPT: Enabling Visual Grounding in Multi-Modal LLMs Yang Zhao Zhijie Lin Daquan Zhou Zilong Huang Jiashi Feng Bingyi Kang MLLM 44 108 0 17 Jul 2023
A Comprehensive Overview of Large Language Models Humza Naveed Asad Ullah Khan Shi Qiu Muhammad Saqib Saeed Anwar Muhammad Usman Naveed Akhtar Nick Barnes Ajmal Mian OffRL 72 544 0 12 Jul 2023
SVIT: Scaling up Visual Instruction Tuning Bo Zhao Boya Wu Muyang He Tiejun Huang MLLM 44 120 0 09 Jul 2023
Exploring and Characterizing Large Language Models For Embedded System Development and Debugging Zachary Englhardt Rong-Hua Li Dilini Nissanka Zhihan Zhang Girish Narayanswamy Joseph Breda Xin Liu Shwetak N. Patel Vikram Iyer 32 18 0 07 Jul 2023
What Matters in Training a GPT4-Style Language Model with Multimodal Inputs? Yan Zeng Hanbo Zhang Jiani Zheng Jiangnan Xia Guoqiang Wei Yang Wei Yuchen Zhang Tao Kong MLLM 27 73 0 05 Jul 2023
LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understanding Yanzhe Zhang Ruiyi Zhang Jiuxiang Gu Yufan Zhou Nedim Lipka Diyi Yang Tongfei Sun VLM MLLM 33 219 0 29 Jun 2023
Explainable Multimodal Emotion Recognition Zheng Lian Haiyang Sun Guoying Zhao Hao Gu Zhuofan Wen ... Shan Liang Ya Li Jiangyan Yi B. Liu Jianhua Tao MLLM 35 6 0 27 Jun 2023
FunQA: Towards Surprising Video Comprehension Binzhu Xie Sicheng Zhang Zitang Zhou Bo Li Yuanhan Zhang Jack Hessel Jingkang Yang Ziwei Liu 47 21 0 26 Jun 2023
Large Multimodal Models: Notes on CVPR 2023 Tutorial Chunyuan Li MLLM VLM 24 20 0 26 Jun 2023
A Survey on Multimodal Large Language Models Shukang Yin Chaoyou Fu Sirui Zhao Ke Li Xing Sun Tong Xu Enhong Chen MLLM LRM 62 562 0 23 Jun 2023
Path to Medical AGI: Unify Domain-specific Medical LLMs with the Lowest Cost Juexiao Zhou Preslav Nakov Xin Gao LM&MA AI4CE 96 12 0 19 Jun 2023
LLMVA-GEBC: Large Language Model with Video Adapter for Generic Event Boundary Captioning Yunlong Tang Jinrui Zhang Xiangchen Wang Teng Wang Feng Zheng VLM 76 9 0 17 Jun 2023
Valley: Video Assistant with Large Language model Enhanced abilitY Ruipu Luo Ziwang Zhao Min Yang Junwei Dong Da Li Pengcheng Lu Tao Wang Linmei Hu Ming-Hui Qiu MLLM 57 191 0 12 Jun 2023
Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models Muhammad Maaz H. Rasheed Salman Khan Fahad Shahbaz Khan MLLM 61 592 0 08 Jun 2023
Youku-mPLUG: A 10 Million Large-scale Chinese Video-Language Dataset for Pre-training and Benchmarks Haiyang Xu Qinghao Ye Xuan-Wei Wu Mingshi Yan Yuan Miao ... Qingfang Qian Maofei Que Ji Zhang Xiaoyan Zeng Feiyan Huang VLM MLLM 56 23 0 07 Jun 2023
Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Models Oana Ignat Zhijing Jin Artem Abzaliev Laura Biester Santiago Castro ... Verónica Pérez-Rosas Siqi Shen Zekun Wang Winston Wu Rada Mihalcea LRM 46 6 0 21 May 2023
mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality Qinghao Ye Haiyang Xu Guohai Xu Jiabo Ye Ming Yan ... Junfeng Tian Qiang Qi Ji Zhang Feiyan Huang Jingren Zhou VLM MLLM 212 910 0 27 Apr 2023
Enhancing Chain-of-Thoughts Prompting with Iterative Bootstrapping in Large Language Models Jiashuo Sun Yi Luo Yeyun Gong Chen Lin Yelong Shen Jian Guo Nan Duan LRM 44 19 0 23 Apr 2023
MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models Deyao Zhu Jun Chen Xiaoqian Shen Xiang Li Mohamed Elhoseiny VLM MLLM 75 1,922 0 20 Apr 2023
LASER: A Neuro-Symbolic Framework for Learning Spatial-Temporal Scene Graphs with Weak Supervision Jiani Huang Ziyang Li Mayur Naik Ser-Nam Lim 44 3 0 15 Apr 2023