All in One: Exploring Unified Video-Language Pre-training

14 March 2022

Rui Yan

Ying Shan

Papers citing "All in One: Exploring Unified Video-Language Pre-training"

50 / 152 papers shown

Title
ActionHub: A Large-scale Action Video Description Dataset for Zero-shot Action Recognition Jiaming Zhou Junwei Liang Kun-Yu Lin Jinrui Yang Wei-Shi Zheng VLM 21 8 0 22 Jan 2024
Weakly Supervised Gaussian Contrastive Grounding with Large Multimodal Models for Video Question Answering Haibo Wang Chenghang Lai Yixuan Sun Weifeng Ge 31 5 0 19 Jan 2024
Glance and Focus: Memory Prompting for Multi-Event Video Question Answering Ziyi Bai Ruiping Wang Xilin Chen 97 8 0 03 Jan 2024
Sports-QA: A Large-Scale Video Question Answering Benchmark for Complex and Professional Sports Haopeng Li Andong Deng Qiuhong Ke Jun Liu Hossein Rahmani Yulan Guo Mohammed Bennamoun Chen Chen 51 17 0 03 Jan 2024
Freeze the backbones: A Parameter-Efficient Contrastive Approach to Robust Medical Vision-Language Pre-training Jiuming Qin Che Liu Sibo Cheng Yike Guo Rossella Arcucci VLM MedIm 20 5 0 02 Jan 2024
ViLA: Efficient Video-Language Alignment for Video Question Answering Xijun Wang Junbang Liang Chun-Kai Wang Kenan Deng Yu Lou Ming-Chyuan Lin Shan Yang 32 13 0 13 Dec 2023
RGNet: A Unified Clip Retrieval and Grounding Network for Long Videos Tanveer Hannan Md. Mohaiminul Islam Thomas Seidl Gedas Bertasius 28 3 0 11 Dec 2023
Audio-Visual LLM for Video Understanding Fangxun Shu Lei Zhang Hao Jiang Cihang Xie VLM MLLM 27 38 0 11 Dec 2023
Vision-Language Models Learn Super Images for Efficient Partially Relevant Video Retrieval Taichi Nishimura Shota Nakada Masayoshi Kondo VLM 21 0 0 01 Dec 2023
Spacewalk-18: A Benchmark for Multimodal and Long-form Procedural Video Understanding in Novel Domains Rohan Myer Krishnan Zitian Tang Zhiqiu Yu Chen Sun 53 1 0 30 Nov 2023
VITATECS: A Diagnostic Dataset for Temporal Concept Understanding of Video-Language Models Shicheng Li Lei Li Shuhuai Ren Yuanxin Liu Yi Liu Rundong Gao Xu Sun Lu Hou 36 29 0 29 Nov 2023
E-ViLM: Efficient Video-Language Model via Masked Video Modeling with Semantic Vector-Quantized Tokenizer Jacob Zhiyuan Fang Skyler Zheng Vasu Sharma Robinson Piramuthu VLM 38 0 0 28 Nov 2023
MVBench: A Comprehensive Multi-modal Video Understanding Benchmark Kunchang Li Yali Wang Yinan He Yizhuo Li Yi Wang ... Jilan Xu Guo Chen Ping Luo Limin Wang Yu Qiao VLM MLLM 56 399 0 28 Nov 2023
Mug-STAN: Adapting Image-Language Pretrained Models for General Video Understanding Ruyang Liu Jingjia Huang Wei-Nan Gao Thomas H. Li Ge Li VLM 29 3 0 25 Nov 2023
Vamos: Versatile Action Models for Video Understanding Shijie Wang Qi Zhao Minh Quan Do Nakul Agarwal Kwonjoon Lee Chen Sun 27 19 0 22 Nov 2023
SPOT! Revisiting Video-Language Models for Event Understanding Gengyuan Zhang Jinhe Bi Jindong Gu Yanyu Chen Volker Tresp 24 2 0 21 Nov 2023
ViLMA: A Zero-Shot Benchmark for Linguistic and Temporal Grounding in Video-Language Models .Ilker Kesen Andrea Pedrotti Mustafa Dogan Michele Cafagna Emre Can Acikgoz ... Iacer Calixto Anette Frank Albert Gatt Aykut Erdem Erkut Erdem 38 15 0 13 Nov 2023
Mirasol3B: A Multimodal Autoregressive model for time-aligned and contextual modalities A. Piergiovanni Isaac Noble Dahun Kim Michael S. Ryoo Victor Gomes A. Angelova 36 19 0 09 Nov 2023
MM-VID: Advancing Video Understanding with GPT-4V(ision) Kevin Qinghong Lin Faisal Ahmed Linjie Li Chung-Ching Lin E. Azarnasab ... Lin Liang Zicheng Liu Yumao Lu Ce Liu Lijuan Wang MLLM 28 63 0 30 Oct 2023
Harvest Video Foundation Models via Efficient Post-Pretraining Yizhuo Li Kunchang Li Yinan He Yi Wang Yali Wang Limin Wang Yu Qiao Ping Luo CLIP VLM VGen 51 2 0 30 Oct 2023
CHAMMI: A benchmark for channel-adaptive models in microscopy imaging Zitong S. Chen Chau Pham Siqi Wang Michael Doron Nikita Moshkov Bryan A. Plummer Juan C. Caicedo 30 11 0 30 Oct 2023
Large Language Models are Temporal and Causal Reasoners for Video Question Answering Dohwan Ko Ji Soo Lee Wooyoung Kang Byungseok Roh Hyunwoo J. Kim LRM 33 31 0 24 Oct 2023
Query-aware Long Video Localization and Relation Discrimination for Deep Video Understanding Yuanxing Xu Yuting Wei Bin Wu 27 0 0 19 Oct 2023
VidCoM: Fast Video Comprehension through Large Language Models with Multimodal Tools Huihui Gong Minjing Dong Siqi Ma S. Çamtepe Chang Xu Lei Hou Surya Nepal VLM MLLM 55 0 0 16 Oct 2023
Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook Ming Jin Qingsong Wen Yuxuan Liang Chaoli Zhang Siqiao Xue ... Shirui Pan Vincent S. Tseng Yu Zheng Lei Chen Hui Xiong AI4TS SyDa 35 117 0 16 Oct 2023
Targeted Image Data Augmentation Increases Basic Skills Captioning Robustness Valentin Barriere Felipe del Rio Andres Carvallo De Ferari Carlos Aspillaga Eugenio Herrera-Berg Cristian Buc Calderon DiffM 27 0 0 27 Sep 2023
Zero-Shot and Few-Shot Video Question Answering with Multi-Modal Prompts Bipin Rajendran Bashir M. Al-Hashimi MLLM VLM 30 2 0 27 Sep 2023
MUTEX: Learning Unified Policies from Multimodal Task Specifications Rutav Shah Roberto Martín-Martín Yuke Zhu OffRL 44 54 0 25 Sep 2023
VidChapters-7M: Video Chapters at Scale Antoine Yang Arsha Nagrani Ivan Laptev Josef Sivic Cordelia Schmid VGen 23 26 0 25 Sep 2023
Unified Coarse-to-Fine Alignment for Video-Text Retrieval Ziyang Wang Yi-Lin Sung Feng Cheng Gedas Bertasius Joey Tianyi Zhou 101 44 0 18 Sep 2023
Can I Trust Your Answer? Visually Grounded Video Question Answering Junbin Xiao Angela Yao Yicong Li Tat-Seng Chua 33 46 0 04 Sep 2023
Multi-event Video-Text Retrieval Gengyuan Zhang Jisen Ren Jindong Gu Volker Tresp 19 13 0 22 Aug 2023
Open-vocabulary Video Question Answering: A New Benchmark for Evaluating the Generalizability of Video Question Answering Models Dohwan Ko Ji Soo Lee M. Choi Jaewon Chu Jihwan Park Hyunwoo J. Kim 22 5 0 18 Aug 2023
Prompt Switch: Efficient CLIP Adaptation for Text-Video Retrieval Chaorui Deng Qi Chen Pengda Qin Dave Zhenyu Chen Qi Wu VLM CLIP 46 29 0 15 Aug 2023
UniVTG: Towards Unified Video-Language Temporal Grounding Kevin Qinghong Lin Pengchuan Zhang Joya Chen Shraman Pramanick Difei Gao Alex Jinpeng Wang Rui Yan Mike Zheng Shou 26 113 0 31 Jul 2023
MovieChat: From Dense Token to Sparse Memory for Long Video Understanding Enxin Song Wenhao Chai Guanhong Wang Yucheng Zhang Haoyang Zhou ... Tianbo Ye Yanting Zhang Yang Lu Jenq-Neng Hwang Gaoang Wang VLM MLLM 22 262 0 31 Jul 2023
UnIVAL: Unified Model for Image, Video, Audio and Language Tasks Mustafa Shukor Corentin Dancette Alexandre Ramé Matthieu Cord MoMe MLLM 61 42 0 30 Jul 2023
InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation Yi Wang Yinan He Yizhuo Li Kunchang Li Jiashuo Yu ... Ping Luo Ziwei Liu Yali Wang Limin Wang Yu Qiao VLM VGen 33 244 0 13 Jul 2023
EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbone Shraman Pramanick Yale Song Sayan Nag Kevin Qinghong Lin Hardik Shah Mike Zheng Shou Ramalingam Chellappa Pengchuan Zhang VLM 39 87 0 11 Jul 2023
Self-Adaptive Sampling for Efficient Video Question-Answering on Image--Text Models Wei Han Hui Chen MingSung Kan Soujanya Poria 24 1 0 09 Jul 2023
All in One: Exploring Unified Vision-Language Tracking with Multi-Modal Alignment Chunhui Zhang Xin Sun Li Liu Yiqian Yang Qiong Liu Xiaoping Zhou Yanfeng Wang 46 15 0 07 Jul 2023
Review of Large Vision Models and Visual Prompt Engineering Jiaqi Wang Zheng Liu Lin Zhao Zihao Wu Chong Ma ... Bao Ge Yixuan Yuan Dinggang Shen Tianming Liu Shu Zhang VLM LRM 55 146 0 03 Jul 2023
Retrieving-to-Answer: Zero-Shot Video Question Answering with Frozen Large Language Models Junting Pan Ziyi Lin Yuying Ge Xiatian Zhu Renrui Zhang Yi Wang Yu Qiao Hongsheng Li MLLM 24 26 0 15 Jun 2023
COSA: Concatenated Sample Pretrained Vision-Language Foundation Model Sihan Chen Xingjian He Handong Li Xiaojie Jin Jiashi Feng Jiaheng Liu VLM CLIP 30 8 0 15 Jun 2023
Dissecting Multimodality in VideoQA Transformer Models by Impairing Modality Fusion Isha Rawal Alexander Matyasko Shantanu Jaiswal Basura Fernando Cheston Tan 21 1 0 15 Jun 2023
Too Large; Data Reduction for Vision-Language Pre-Training Alex Jinpeng Wang Kevin Qinghong Lin David Junhao Zhang Stan Weixian Lei Mike Zheng Shou VLM 30 24 0 31 May 2023
VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset Sihan Chen Handong Li Qunbo Wang Zijia Zhao Ming-Ting Sun Xinxin Zhu Jiaheng Liu 32 97 0 29 May 2023
Cross-view Action Recognition Understanding From Exocentric to Egocentric Perspective Thanh-Dat Truong Khoa Luu EgoV 27 10 0 25 May 2023
VideoLLM: Modeling Video Sequence with Large Language Models Guo Chen Yin-Dong Zheng Jiahao Wang Jilan Xu Yifei Huang ... Yi Wang Yali Wang Yu Qiao Tong Lu Limin Wang MLLM 103 76 0 22 May 2023
VLAB: Enhancing Video Language Pre-training by Feature Adapting and Blending Xingjian He Sihan Chen Fan Ma Zhicheng Huang Xiaojie Jin Zikang Liu Dongmei Fu Yi Yang Jiaheng Liu Jiashi Feng VLM CLIP 23 17 0 22 May 2023