Zero-Shot Video Question Answering via Frozen Bidirectional Language Models

16 June 2022

Papers citing "Zero-Shot Video Question Answering via Frozen Bidirectional Language Models"

50 / 194 papers shown

Title
YTCommentQA: Video Question Answerability in Instructional Videos Saelyne Yang Sunghyun Park Yunseok Jang Moontae Lee 31 3 0 30 Jan 2024
Weakly Supervised Gaussian Contrastive Grounding with Large Multimodal Models for Video Question Answering Haibo Wang Chenghang Lai Yixuan Sun Weifeng Ge 31 5 0 19 Jan 2024
ELLA-V: Stable Neural Codec Language Modeling with Alignment-guided Sequence Reordering Ya-Zhen Song Zhuo Chen Xiaofei Wang Ziyang Ma Xie Chen AuLLM 21 37 0 14 Jan 2024
Distilling Vision-Language Models on Millions of Videos Yue Zhao Long Zhao Xingyi Zhou Jialin Wu Chun-Te Chu ... Hartwig Adam Ting Liu Boqing Gong Philipp Krahenbuhl Liangzhe Yuan VLM 34 13 0 11 Jan 2024
Detours for Navigating Instructional Videos Kumar Ashutosh Zihui Xue Tushar Nagarajan Kristen Grauman 34 6 0 03 Jan 2024
Glance and Focus: Memory Prompting for Multi-Event Video Question Answering Ziyi Bai Ruiping Wang Xilin Chen 97 8 0 03 Jan 2024
A Simple LLM Framework for Long-Range Video Question-Answering Ce Zhang Taixi Lu Md. Mohaiminul Islam Ziyang Wang Shoubin Yu Mohit Bansal Gedas Bertasius 110 82 0 28 Dec 2023
Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Model Shraman Pramanick Guangxing Han Rui Hou Sayan Nag Ser-Nam Lim Nicolas Ballas Qifan Wang Rama Chellappa Amjad Almahairi VLM MLLM 48 29 0 19 Dec 2023
ViLA: Efficient Video-Language Alignment for Video Question Answering Xijun Wang Junbang Liang Chun-Kai Wang Kenan Deng Yu Lou Ming-Chyuan Lin Shan Yang 40 13 0 13 Dec 2023
READ-PVLA: Recurrent Adapter with Partial Video-Language Alignment for Parameter-Efficient Transfer Learning in Low-Resource Video-Language Modeling Thong Nguyen Xiaobao Wu Xinshuai Dong Khoi M. Le Zhiyuan Hu Cong-Duy Nguyen See-Kiong Ng Anh Tuan Luu 36 2 0 12 Dec 2023
Vista-LLaMA: Reducing Hallucination in Video Language Models via Equal Distance to Visual Tokens Fan Ma Xiaojie Jin Heng Wang Yuchen Xian Jiashi Feng Yi Yang 29 47 0 12 Dec 2023
Learning Hierarchical Prompt with Structured Linguistic Knowledge for Vision-Language Models Yubin Wang Xinyang Jiang De Cheng Dongsheng Li Cairong Zhao VLM 40 15 0 11 Dec 2023
Audio-Visual LLM for Video Understanding Fangxun Shu Lei Zhang Hao Jiang Cihang Xie VLM MLLM 27 38 0 11 Dec 2023
MoVQA: A Benchmark of Versatile Question-Answering for Long-Form Movie Understanding Hongjie Zhang Yi Liu Lu Dong Yifei Huang Z. Ling Yali Wang Limin Wang Yu Qiao 23 25 0 08 Dec 2023
LifelongMemory: Leveraging LLMs for Answering Queries in Long-form Egocentric Videos Ying Wang Yanlai Yang Mengye Ren 49 15 0 07 Dec 2023
VaQuitA: Enhancing Alignment in LLM-Assisted Video Understanding Yizhou Wang Ruiyi Zhang Haoliang Wang Uttaran Bhattacharya Yun Fu Gang Wu MLLM 40 10 0 04 Dec 2023
Zero-Shot Video Question Answering with Procedural Programs Rohan Choudhury Koichiro Niinuma Kris M. Kitani László A. Jeni 24 21 0 01 Dec 2023
X-InstructBLIP: A Framework for aligning X-Modal instruction-aware representations to LLMs and Emergent Cross-modal Reasoning Artemis Panagopoulou Le Xue Ning Yu Junnan Li Dongxu Li Chenyu You Ran Xu Silvio Savarese Caiming Xiong Juan Carlos Niebles VLM MLLM 41 46 0 30 Nov 2023
LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models Yanwei Li Chengyao Wang Jiaya Jia VLM MLLM 43 264 0 28 Nov 2023
MVBench: A Comprehensive Multi-modal Video Understanding Benchmark Kunchang Li Yali Wang Yinan He Yizhuo Li Yi Wang ... Jilan Xu Guo Chen Ping Luo Limin Wang Yu Qiao VLM MLLM 87 410 0 28 Nov 2023
Vamos: Versatile Action Models for Video Understanding Shijie Wang Qi Zhao Minh Quan Do Nakul Agarwal Kwonjoon Lee Chen Sun 29 19 0 22 Nov 2023
PG-Video-LLaVA: Pixel Grounding Large Video-Language Models Shehan Munasinghe Rusiru Thushara Muhammad Maaz H. Rasheed Salman Khan Mubarak Shah Fahad Khan VLM MLLM 35 34 0 22 Nov 2023
Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding Peng Jin Ryuichi Takanobu Caiwan Zhang Xiaochun Cao Li-ming Yuan MLLM 36 226 0 14 Nov 2023
ViLMA: A Zero-Shot Benchmark for Linguistic and Temporal Grounding in Video-Language Models .Ilker Kesen Andrea Pedrotti Mustafa Dogan Michele Cafagna Emre Can Acikgoz ... Iacer Calixto Anette Frank Albert Gatt Aykut Erdem Erkut Erdem 41 15 0 13 Nov 2023
Mirasol3B: A Multimodal Autoregressive model for time-aligned and contextual modalities A. Piergiovanni Isaac Noble Dahun Kim Michael S. Ryoo Victor Gomes A. Angelova 43 19 0 09 Nov 2023
mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration Qinghao Ye Haiyang Xu Jiabo Ye Mingshi Yan Anwen Hu Haowei Liu Qi Qian Ji Zhang Fei Huang Jingren Zhou MLLM VLM 129 385 0 07 Nov 2023
Active Reasoning in an Open-World Environment Manjie Xu Guangyuan Jiang Weihan Liang Chi Zhang Yixin Zhu LLMAG LRM 21 10 0 03 Nov 2023
Harvest Video Foundation Models via Efficient Post-Pretraining Yizhuo Li Kunchang Li Yinan He Yi Wang Yali Wang Limin Wang Yu Qiao Ping Luo CLIP VLM VGen 54 2 0 30 Oct 2023
Apollo: Zero-shot MultiModal Reasoning with Multiple Experts Daniela Ben-David Tzuf Paz-Argaman Reut Tsarfaty MoE 36 0 0 25 Oct 2023
CAD -- Contextual Multi-modal Alignment for Dynamic AVQA Asmar Nadeem Adrian Hilton R. Dawes Graham A. Thomas A. Mustafa 33 9 0 25 Oct 2023
Large Language Models are Temporal and Causal Reasoners for Video Question Answering Dohwan Ko Ji Soo Lee Wooyoung Kang Byungseok Roh Hyunwoo J. Kim LRM 33 31 0 24 Oct 2023
Efficient Cross-Task Prompt Tuning for Few-Shot Conversational Emotion Recognition Yige Xu Zhiwei Zeng Zhiqi Shen VLM 33 3 0 23 Oct 2023
UrbanCLIP: Learning Text-enhanced Urban Region Profiling with Contrastive Language-Image Pretraining from the Web Yibo Yan Haomin Wen Siru Zhong Wei Chen Haodong Chen Qingsong Wen Roger Zimmermann Keli Zhang 21 51 0 22 Oct 2023
Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook Ming Jin Qingsong Wen Keli Zhang Chaoli Zhang Siqiao Xue ... Shirui Pan Vincent S. Tseng Yu Zheng Lei Chen Hui Xiong AI4TS SyDa 40 118 0 16 Oct 2023
Beyond Segmentation: Road Network Generation with Multi-Modal LLMs Sumedh Rasal Sanjay K. Boddhu 35 5 0 15 Oct 2023
IMITATE: Clinical Prior Guided Hierarchical Vision-Language Pre-training Che Liu Sibo Cheng Miaojing Shi Anand Shah Wenjia Bai Rossella Arcucci 24 26 0 11 Oct 2023
Zero-Shot and Few-Shot Video Question Answering with Multi-Modal Prompts Bipin Rajendran Bashir M. Al-Hashimi MLLM VLM 32 2 0 27 Sep 2023
VidChapters-7M: Video Chapters at Scale Antoine Yang Arsha Nagrani Ivan Laptev Josef Sivic Cordelia Schmid VGen 23 26 0 25 Sep 2023
Can I Trust Your Answer? Visually Grounded Video Question Answering Junbin Xiao Angela Yao Yicong Li Tat-Seng Chua 41 46 0 04 Sep 2023
Open-vocabulary Video Question Answering: A New Benchmark for Evaluating the Generalizability of Video Question Answering Models Dohwan Ko Ji Soo Lee M. Choi Jaewon Chu Jihwan Park Hyunwoo J. Kim 22 5 0 18 Aug 2023
EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding K. Mangalam Raiymbek Akshulakov Jitendra Malik 25 253 0 17 Aug 2023
Tem-adapter: Adapting Image-Text Pretraining for Video Question Answer Guangyi Chen Xiao Liu Guangrun Wang Kun Zhang Philip H.S.Torr Xiaoping Zhang Yansong Tang 27 18 0 16 Aug 2023
MovieChat: From Dense Token to Sparse Memory for Long Video Understanding Enxin Song Wenhao Chai Guanhong Wang Yucheng Zhang Haoyang Zhou ... Tianbo Ye Yanting Zhang Yang Lu Lei Li Gaoang Wang VLM MLLM 27 264 0 31 Jul 2023
UnIVAL: Unified Model for Image, Video, Audio and Language Tasks Mustafa Shukor Corentin Dancette Alexandre Ramé Matthieu Cord MoMe MLLM 61 42 0 30 Jul 2023
MultiQG-TI: Towards Question Generation from Multi-modal Sources Zichao Wang Richard Baraniuk 25 5 0 07 Jul 2023
Meta-Personalizing Vision-Language Models to Find Named Instances in Video Chun-Hsiao Yeh Bryan C. Russell Josef Sivic Fabian Caba Heilbron Simon Jenni VLM MLLM 49 9 0 16 Jun 2023
Retrieving-to-Answer: Zero-Shot Video Question Answering with Frozen Large Language Models Junting Pan Ziyi Lin Yuying Ge Xiatian Zhu Renrui Zhang Yi Wang Yu Qiao Hongsheng Li MLLM 32 26 0 15 Jun 2023
COSA: Concatenated Sample Pretrained Vision-Language Foundation Model Sihan Chen Xingjian He Handong Li Xiaojie Jin Jiashi Feng Qingbin Liu VLM CLIP 30 8 0 15 Jun 2023
Dissecting Multimodality in VideoQA Transformer Models by Impairing Modality Fusion Isha Rawal Alexander Matyasko Shantanu Jaiswal Basura Fernando Cheston Tan 26 2 0 15 Jun 2023
Valley: Video Assistant with Large Language model Enhanced abilitY Ruipu Luo Ziwang Zhao Min Yang Junwei Dong Da Li Pengcheng Lu Tao Wang Linmei Hu Ming-Hui Qiu MLLM 54 191 0 12 Jun 2023