VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding

20 May 2021

Hu Xu

Gargi Ghosh

Po-Yao (Bernie) Huang

Prahal Arora

Masoumeh Aminzadeh

Christoph Feichtenhofer

Florian Metze

Luke Zettlemoyer

ArXiv PDF HTML

Papers citing "VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding"

39 / 39 papers shown

Title
KeyVideoLLM: Towards Large-scale Video Keyframe Selection Hao Liang Jiapeng Li Tianyi Bai Xijie Huang Linzhuang Sun Zhengren Wang Conghui He Bin Cui Chong Chen Wentao Zhang VGen 29 7 0 03 Jul 2024
AutoTVG: A New Vision-language Pre-training Paradigm for Temporal Video Grounding Xing Zhang Jiaxi Gu Haoyu Zhao Shicong Wang Hang Xu Renjing Pei Songcen Xu Zuxuan Wu Yu-Gang Jiang 40 0 0 11 Jun 2024
Instruct-ReID++: Towards Universal Purpose Instruction-Guided Person Re-identification Weizhen He Yiheng Deng Yunfeng Yan Feng Zhu Yizhou Wang Lei Bai Qingsong Xie Donglian Qi Wanli Ouyang Shixiang Tang 95 2 0 28 May 2024
Imagery as Inquiry: Exploring A Multimodal Dataset for Conversational Recommendation Se-eun Yoon Hyunsik Jeon Julian McAuley 38 0 0 23 May 2024
RGNet: A Unified Clip Retrieval and Grounding Network for Long Videos Tanveer Hannan Md. Mohaiminul Islam Thomas Seidl Gedas Bertasius 28 3 0 11 Dec 2023
UnLoc: A Unified Framework for Video Localization Tasks Shengjia Yan Xuehan Xiong Arsha Nagrani Anurag Arnab Zhonghao Wang Weina Ge David A. Ross Cordelia Schmid 26 53 0 21 Aug 2023
Instruct-ReID: A Multi-purpose Person Re-identification Task with Instructions Weizhen He Yihe Deng Shixiang Tang Qihao Chen Qingsong Xie ... Feng Zhu Rui Zhao Wanli Ouyang Donglian Qi Yunfeng Yan 74 19 0 13 Jun 2023
Learning Summary-Worthy Visual Representation for Abstractive Summarization in Video Zenan Xu Xiaojun Meng Yasheng Wang Qinliang Su Zexuan Qiu Xin Jiang Qun Liu 22 3 0 08 May 2023
Differentially Private Attention Computation Yeqi Gao Zhao-quan Song Xin Yang 47 19 0 08 May 2023
Procedure-Aware Pretraining for Instructional Video Understanding Honglu Zhou Roberto Martín-Martín Mubbasir Kapadia Silvio Savarese Juan Carlos Niebles 25 38 0 31 Mar 2023
GOAL: A Challenging Knowledge-grounded Video Captioning Benchmark for Real-time Soccer Commentary Generation Ji Qi Jifan Yu Teng Tu Kunyu Gao Yifan Xu ... Juanzi Li Jie Tang Weidong Guo Hui Liu Yu-Syuan Xu 23 19 0 26 Mar 2023
Text with Knowledge Graph Augmented Transformer for Video Captioning Xin Gu G. Chen Yufei Wang Libo Zhang Tiejian Luo Longyin Wen 27 47 0 22 Mar 2023
Accommodating Audio Modality in CLIP for Multimodal Processing Ludan Ruan Anwen Hu Yuqing Song Liang Zhang S. Zheng Qin Jin VLM 21 10 0 12 Mar 2023
STOA-VLP: Spatial-Temporal Modeling of Object and Action for Video-Language Pre-training Weihong Zhong Mao Zheng Duyu Tang Xuan Luo Heng Gong Xiaocheng Feng Bing Qin 27 8 0 20 Feb 2023
HierVL: Learning Hierarchical Video-Language Embeddings Kumar Ashutosh Rohit Girdhar Lorenzo Torresani Kristen Grauman VLM AI4TS 22 51 0 05 Jan 2023
Learning Trajectory-Word Alignments for Video-Language Tasks Xu Yang Zhang Li Haiyang Xu Hanwang Zhang Qinghao Ye Chenliang Li Ming Yan Yu Zhang Fei Huang Songfang Huang 30 7 0 05 Jan 2023
NaQ: Leveraging Narrations as Queries to Supervise Episodic Memory Santhosh Kumar Ramakrishnan Ziad Al-Halah Kristen Grauman 114 39 0 02 Jan 2023
VideoCoCa: Video-Text Modeling with Zero-Shot Transfer from Contrastive Captioners Shen Yan Tao Zhu Zirui Wang Yuan Cao Mi Zhang Soham Ghosh Yonghui Wu Jiahui Yu VLM VGen 32 46 0 09 Dec 2022
VLG: General Video Recognition with Web Textual Knowledge Jintao Lin Zhaoyang Liu Wenhai Wang Wayne Wu Limin Wang 39 0 0 03 Dec 2022
Grafting Pre-trained Models for Multimodal Headline Generation Lingfeng Qiao Chen Wu Ye Liu Haoyuan Peng Di Yin Bo Ren 35 5 0 14 Nov 2022
Hierarchical3D Adapters for Long Video-to-text Summarization Pinelopi Papalampidi Mirella Lapata VGen 27 12 0 10 Oct 2022
CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignment Hongwei Xue Yuchong Sun Bei Liu Jianlong Fu Rui Song Houqiang Li Jiebo Luo CLIP VLM 25 68 0 14 Sep 2022
Clover: Towards A Unified Video-Language Alignment and Fusion Model Jingjia Huang Yinan Li Jiashi Feng Xinglong Wu Xiaoshuai Sun Rongrong Ji VLM 19 48 0 16 Jul 2022
Robustness Analysis of Video-Language Models Against Visual and Language Perturbations Madeline Chantry Schiappa Shruti Vyas Hamid Palangi Y. S. Rawat Vibhav Vineet VLM 120 17 0 05 Jul 2022
Self-Supervised Learning for Videos: A Survey Madeline Chantry Schiappa Y. S. Rawat M. Shah SSL 34 131 0 18 Jun 2022
LAVENDER: Unifying Video-Language Understanding as Masked Language Modeling Linjie Li Zhe Gan Kevin Qinghong Lin Chung-Ching Lin Zicheng Liu Ce Liu Lijuan Wang MLLM VLM 20 81 0 14 Jun 2022
Multimodal Learning with Transformers: A Survey P. Xu Xiatian Zhu David A. Clifton ViT 50 525 0 13 Jun 2022
Egocentric Video-Language Pretraining Kevin Qinghong Lin Alex Jinpeng Wang Mattia Soldan Michael Wray Rui Yan ... Hongfa Wang Dima Damen Bernard Ghanem Wei Liu Mike Zheng Shou VLM EgoV 46 188 0 03 Jun 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 46 3,334 0 29 Apr 2022
MILES: Visual BERT Pre-training with Injected Language Semantics for Video-text Retrieval Yuying Ge Yixiao Ge Xihui Liu Alex Jinpeng Wang Jianping Wu Ying Shan Xiaohu Qie Ping Luo VLM 13 43 0 26 Apr 2022
End-to-End Modeling via Information Tree for One-Shot Natural Language Spatial Video Grounding Meng Li Tianbao Wang Haoyu Zhang Shengyu Zhang Zhou Zhao ... Wenming Tan Jin Wang Peng Wang Shi Pu Fei Wu 21 45 0 15 Mar 2022
Learnable Irrelevant Modality Dropout for Multimodal Action Recognition on Modality-Specific Annotated Videos Saghir Alfasly Jian Lu C. Xu Yuru Zou 34 18 0 06 Mar 2022
Learning To Recognize Procedural Activities with Distant Supervision Xudong Lin Fabio Petroni Gedas Bertasius Marcus Rohrbach Shih-Fu Chang Lorenzo Torresani 29 82 0 26 Jan 2022
Temporal Sentence Grounding in Videos: A Survey and Future Directions Hao Zhang Aixin Sun Wei Jing Joey Tianyi Zhou 3DGS 36 38 0 20 Jan 2022
Bridging Video-text Retrieval with Multiple Choice Questions Yuying Ge Yixiao Ge Xihui Liu Dian Li Ying Shan Xiaohu Qie Ping Luo BDL 24 108 0 13 Jan 2022
Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions Hongwei Xue Tiankai Hang Yanhong Zeng Yuchong Sun Bei Liu Huan Yang Jianlong Fu B. Guo AI4TS VLM 27 189 0 19 Nov 2021
Survey: Transformer based Video-Language Pre-training Ludan Ruan Qin Jin VLM ViT 72 44 0 21 Sep 2021
Multi-modal Transformer for Video Retrieval Valentin Gabeur Chen Sun Alahari Karteek Cordelia Schmid ViT 418 596 0 21 Jul 2020
Unified Vision-Language Pre-Training for Image Captioning and VQA Luowei Zhou Hamid Palangi Lei Zhang Houdong Hu Jason J. Corso Jianfeng Gao MLLM VLM 252 927 0 24 Sep 2019