VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding

28 September 2021

Hu Xu

Gargi Ghosh

Po-Yao (Bernie) Huang

Florian Metze Luke Zettlemoyer Christoph Feichtenhofer

CLIP

VLM

ArXiv PDF HTML

Papers citing "VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding"

50 / 128 papers shown

Title
Extending Multi-modal Contrastive Representations Zehan Wang Ziang Zhang Luping Liu Yang Zhao Haifeng Huang Tao Jin Zhou Zhao 19 5 0 13 Oct 2023
Delving into CLIP latent space for Video Anomaly Recognition Luca Zanella Benedetta Liberatori Willi Menapace Fabio Poiesi Yiming Wang Elisa Ricci 18 22 0 04 Oct 2023
Training a Large Video Model on a Single Machine in a Day Yue Zhao Philipp Krahenbuhl VLM 27 15 0 28 Sep 2023
CLIP-Hand3D: Exploiting 3D Hand Pose Estimation via Context-Aware Prompting Shaoxiang Guo Qing Cai Lin Qi Junyu Dong 3DH 37 7 0 28 Sep 2023
AttrSeg: Open-Vocabulary Semantic Segmentation via Attribute Decomposition-Aggregation Chaofan Ma Yu-Hao Yang Chen Ju Fei Zhang Ya-Qin Zhang Yanfeng Wang VLM 40 17 0 31 Aug 2023
Cross-Modal Retrieval Meets Inference:Improving Zero-Shot Classification with Cross-Modal Retrieval Seong-Hoon Eom Namgyu Ho Jaehoon Oh Se-Young Yun CLIP VLM 26 0 0 29 Aug 2023
Language Reward Modulation for Pretraining Reinforcement Learning Ademi Adeniji Amber Xie Carmelo Sferrazza Younggyo Seo Stephen James Pieter Abbeel 39 26 0 23 Aug 2023
UnLoc: A Unified Framework for Video Localization Tasks Shengjia Yan Xuehan Xiong Arsha Nagrani Anurag Arnab Zhonghao Wang Weina Ge David A. Ross Cordelia Schmid 24 53 0 21 Aug 2023
Detecting Cloud Presence in Satellite Images Using the RGB-based CLIP Vision-Language Model Mikolaj Czerkawski Robert C. Atkinson Christos Tachtatzis VLM 20 2 0 01 Aug 2023
Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lectures Kun Yuan V. Srivastav Tong Yu Joël L. Lavanchy Pietro Mascagni Pietro Mascagni N. Padoy Nicolas Padoy 22 20 0 27 Jul 2023
Discovering Spatio-Temporal Rationales for Video Question Answering Yicong Li Junbin Xiao Chun Feng Xiang Wang Tat-Seng Chua 19 13 0 22 Jul 2023
Instruct-ReID: A Multi-purpose Person Re-identification Task with Instructions Weizhen He Yihe Deng Shixiang Tang Qihao Chen Qingsong Xie ... Feng Zhu Rui Zhao Wanli Ouyang Donglian Qi Yunfeng Yan 65 19 0 13 Jun 2023
Self-Chained Image-Language Model for Video Localization and Question Answering Shoubin Yu Jaemin Cho Prateek Yadav Mohit Bansal 36 129 0 11 May 2023
LASER: A Neuro-Symbolic Framework for Learning Spatial-Temporal Scene Graphs with Weak Supervision Jiani Huang Ziyang Li Mayur Naik Ser-Nam Lim 35 3 0 15 Apr 2023
Vita-CLIP: Video and text adaptive CLIP via Multimodal Prompting Syed Talal Wasim Muzammal Naseer Salman Khan F. Khan M. Shah VLM VPVLM 28 73 0 06 Apr 2023
VicTR: Video-conditioned Text Representations for Activity Recognition Kumara Kahatapitiya Anurag Arnab Arsha Nagrani Michael S. Ryoo 29 19 0 05 Apr 2023
Procedure-Aware Pretraining for Instructional Video Understanding Honglu Zhou Roberto Martín-Martín Mubbasir Kapadia Silvio Savarese Juan Carlos Niebles 23 38 0 31 Mar 2023
Unmasked Teacher: Towards Training-Efficient Video Foundation Models Kunchang Li Yali Wang Yizhuo Li Yi Wang Yinan He Limin Wang Yu Qiao VGen 43 154 0 28 Mar 2023
GOAL: A Challenging Knowledge-grounded Video Captioning Benchmark for Real-time Soccer Commentary Generation Ji Qi Jifan Yu Teng Tu Kunyu Gao Yifan Xu ... Juanzi Li Jie Tang Weidong Guo Hui Liu Yu-Syuan Xu 23 19 0 26 Mar 2023
Best of Both Worlds: Multimodal Contrastive Learning with Tabular and Imaging Data Paul Hager M. Menten Daniel Rueckert 24 47 0 24 Mar 2023
Multi-modal Prompting for Low-Shot Temporal Action Localization Chen Ju Zeqian Li Peisen Zhao Ya-Qin Zhang Xiaopeng Zhang Qi Tian Yanfeng Wang Weidi Xie 27 18 0 21 Mar 2023
Transformers in Speech Processing: A Survey S. Latif Aun Zaidi Heriberto Cuayáhuitl Fahad Shamshad Moazzam Shoukat Junaid Qadir 42 47 0 21 Mar 2023
VOCALExplore: Pay-as-You-Go Video Data Exploration and Model Building [Technical Report] Maureen Daum Enhao Zhang Dong He Stephen Mussmann Brandon Haynes Ranjay Krishna Magdalena Balazinska 27 4 0 07 Mar 2023
Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning Antoine Yang Arsha Nagrani Paul Hongsuck Seo Antoine Miech Jordi Pont-Tuset Ivan Laptev Josef Sivic Cordelia Schmid AI4TS VLM 23 220 0 27 Feb 2023
STOA-VLP: Spatial-Temporal Modeling of Object and Action for Video-Language Pre-training Weihong Zhong Mao Zheng Duyu Tang Xuan Luo Heng Gong Xiaocheng Feng Bing Qin 25 8 0 20 Feb 2023
ProtST: Multi-Modality Learning of Protein Sequences and Biomedical Texts Minghao Xu Xinyu Yuan Santiago Miret Jian Tang AI4TS 25 94 0 28 Jan 2023
Text-To-4D Dynamic Scene Generation Uriel Singer Shelly Sheynin Adam Polyak Oron Ashual Iurii Makarov ... Naman Goyal Andrea Vedaldi Devi Parikh Justin Johnson Yaniv Taigman DiffM 28 147 0 26 Jan 2023
Temporal Perceiving Video-Language Pre-training Fan Ma Xiaojie Jin Heng Wang Jingjia Huang Linchao Zhu Jiashi Feng Yi Yang VLM 19 15 0 18 Jan 2023
HierVL: Learning Hierarchical Video-Language Embeddings Kumar Ashutosh Rohit Girdhar Lorenzo Torresani Kristen Grauman VLM AI4TS 20 51 0 05 Jan 2023
What You Say Is What You Show: Visual Narration Detection in Instructional Videos Kumar Ashutosh Rohit Girdhar Lorenzo Torresani Kristen Grauman 16 4 0 05 Jan 2023
CiT: Curation in Training for Effective Vision-Language Data Hu Xu Saining Xie Po-Yao (Bernie) Huang Licheng Yu Russ Howes Gargi Ghosh Luke Zettlemoyer Christoph Feichtenhofer VLM DiffM 25 24 0 05 Jan 2023
Learning Trajectory-Word Alignments for Video-Language Tasks Xu Yang Zhang Li Haiyang Xu Hanwang Zhang Qinghao Ye Chenliang Li Ming Yan Yu Zhang Fei Huang Songfang Huang 28 7 0 05 Jan 2023
LidarCLIP or: How I Learned to Talk to Point Clouds Georg Hess Adam Tonderski Christoffer Petersson Kalle AAstrom Lennart Svensson DiffM 19 22 0 13 Dec 2022
VideoCoCa: Video-Text Modeling with Zero-Shot Transfer from Contrastive Captioners Shen Yan Tao Zhu Zirui Wang Yuan Cao Mi Zhang Soham Ghosh Yonghui Wu Jiahui Yu VLM VGen 24 46 0 09 Dec 2022
InternVideo: General Video Foundation Models via Generative and Discriminative Learning Yi Wang Kunchang Li Yizhuo Li Yinan He Bingkun Huang ... Junting Pan Jiashuo Yu Yali Wang Limin Wang Yu Qiao VLM VGen 38 309 0 06 Dec 2022
SMAUG: Sparse Masked Autoencoder for Efficient Video-Language Pre-training Yuanze Lin Chen Wei Huiyu Wang Alan Yuille Cihang Xie 3DGS 26 15 0 21 Nov 2022
Cross-Modal Adapter for Text-Video Retrieval Haojun Jiang Jianke Zhang Rui Huang Chunjiang Ge Zanlin Ni Jiwen Lu Jie Zhou S. Song Gao Huang 40 36 0 17 Nov 2022
Grafting Pre-trained Models for Multimodal Headline Generation Lingfeng Qiao Chen Wu Ye Liu Haoyuan Peng Di Yin Bo Ren 30 5 0 14 Nov 2022
Human in the loop approaches in multi-modal conversational task guidance system development R. Manuvinakurike Sovan Biswas G. Raffa R. Beckwith A. Rhodes Meng Shi Gesem Gudino Mejia Saurav Sahay L. Nachman 27 2 0 03 Nov 2022
LiteVL: Efficient Video-Language Learning with Enhanced Spatial-Temporal Modeling Dongsheng Chen Chaofan Tao Lu Hou Lifeng Shang Xin Jiang Qun Liu VLM 27 18 0 21 Oct 2022
Low-resource Neural Machine Translation with Cross-modal Alignment Zhe Yang Qingkai Fang Yang Feng VLM 29 9 0 13 Oct 2022
Contrastive Video-Language Learning with Fine-grained Frame Sampling Zixu Wang Yujie Zhong Yishu Miao Lin Ma Lucia Specia 44 11 0 10 Oct 2022
CLIP model is an Efficient Continual Learner Vishal G. Thengane Salman Khan Munawar Hayat F. Khan BDL VLM CLL 104 46 0 06 Oct 2022
CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignment Hongwei Xue Yuchong Sun Bei Liu Jianlong Fu Rui Song Houqiang Li Jiebo Luo CLIP VLM 25 68 0 14 Sep 2022
MuMUR : Multilingual Multimodal Universal Retrieval Avinash Madasu Estelle Aflalo Gabriela Ben-Melech Stan Shachar Rosenman Shao-Yen Tseng Gedas Bertasius Vasudev Lal 37 3 0 24 Aug 2022
Expanding Language-Image Pretrained Models for General Video Recognition Bolin Ni Houwen Peng Minghao Chen Songyang Zhang Gaofeng Meng Jianlong Fu Shiming Xiang Haibin Ling VLM CLIP ViT 23 312 0 04 Aug 2022
Clover: Towards A Unified Video-Language Alignment and Fusion Model Jingjia Huang Yinan Li Jiashi Feng Xinglong Wu Xiaoshuai Sun Rongrong Ji VLM 19 48 0 16 Jul 2022
Robustness Analysis of Video-Language Models Against Visual and Language Perturbations Madeline Chantry Schiappa Shruti Vyas Hamid Palangi Y. S. Rawat Vibhav Vineet VLM 120 17 0 05 Jul 2022
Multimodal Frame-Scoring Transformer for Video Summarization Jeiyoon Park Kiho Kwoun Chanhee Lee Heuiseok Lim ViT 25 6 0 05 Jul 2022
Self-Supervised Learning for Videos: A Survey Madeline Chantry Schiappa Y. S. Rawat M. Shah SSL 34 131 0 18 Jun 2022