VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding

28 September 2021

Hu Xu

Gargi Ghosh

Po-Yao (Bernie) Huang

Florian Metze Luke Zettlemoyer Christoph Feichtenhofer

CLIP

VLM

ArXiv PDF HTML

Papers citing "VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding"

50 / 128 papers shown

Title
Fine-Tuning Video-Text Contrastive Model for Primate Behavior Retrieval from Unlabeled Raw Videos Giulio Cesare Mastrocinque Santo Patrícia Izar Irene Delval Victor de Napole Gregolin Nina S. T. Hirata VGen 40 0 0 08 May 2025
T2VPhysBench: A First-Principles Benchmark for Physical Consistency in Text-to-Video Generation Xuyang Guo Jiayan Huo Zhenmei Shi Zhao-quan Song Jiahao Zhang Jiale Zhao EGVM VGen PINN 82 1 0 01 May 2025
VideoComp: Advancing Fine-Grained Compositional and Temporal Alignment in Video-Text Models Dahun Kim A. Piergiovanni Ganesh Mallya A. Angelova CoGe 36 0 0 04 Apr 2025
Is Temporal Prompting All We Need For Limited Labeled Action Recognition? Shreyank N. Gowda Boyan Gao Xiao Gu Xiaobo Jin VLM 34 0 0 02 Apr 2025
A Survey on Music Generation from Single-Modal, Cross-Modal, and Multi-Modal Perspectives Shuyu Li Shulei Ji Zihao W. Wang Songruoyao Wu Jiaxing Yu K. Zhang MGen VGen 70 1 0 01 Apr 2025
Generative Modeling of Class Probability for Multi-Modal Representation Learning Jungkyoo Shin Bumsoo Kim Eunwoo Kim 50 1 0 21 Mar 2025
4D LangSplat: 4D Language Gaussian Splatting via Multimodal Large Language Models Wanhua Li Renping Zhou Jiawei Zhou Yingwei Song Johannes Herter Minghan Qin Gao Huang Hanspeter Pfister 3DGS VLM 66 0 0 13 Mar 2025
Pretrained Image-Text Models are Secretly Video Captioners Chunhui Zhang Yiren Jian Z. Ouyang Soroush Vosoughi VLM 71 4 0 20 Feb 2025
MJ-VIDEO: Fine-Grained Benchmarking and Rewarding Video Preferences in Video Generation Haibo Tong Zhaoyang Wang Z. Chen Haonian Ji Shi Qiu ... Peng Xia Mingyu Ding Rafael Rafailov Chelsea Finn Huaxiu Yao EGVM VGen 95 2 0 03 Feb 2025
Toyteller: AI-powered Visual Storytelling Through Toy-Playing with Character Symbols John Joon Young Chung Melissa Roemmele Max Kreminski VGen 70 0 0 23 Jan 2025
When language and vision meet road safety: leveraging multimodal large language models for video-based traffic accident analysis Ruixuan Zhang Beichen Wang Juexiao Zhang Zilin Bian Chen Feng K. Ozbay 39 2 0 17 Jan 2025
OneLLM: One Framework to Align All Modalities with Language Jiaming Han Kaixiong Gong Yiyuan Zhang Jiaqi Wang Kaipeng Zhang D. Lin Yu Qiao Peng Gao Xiangyu Yue MLLM 104 109 0 10 Jan 2025
A Simple Recipe for Contrastively Pre-training Video-First Encoders Beyond 16 Frames Pinelopi Papalampidi Skanda Koppula Shreya Pathak Justin T Chiu Joseph Heyward Viorica Patraucean Jiajun Shen Antoine Miech Andrew Zisserman Aida Nematzdeh VLM 58 24 0 31 Dec 2024
Can video generation replace cinematographers? Research on the cinematic language of generated video X. Li Kai WU Siyi Yang YiZhan Qu Guohua. Zhang ... Mingliang Xiong Hao Deng Qingwen Liu Gang Li Bin He VGen DiffM 90 1 0 16 Dec 2024
Past, Present, and Future of Sensor-Based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task H. Haresamudram Chi Ian Tang Sungho Suh P. Lukowicz Thomas Ploetz 74 2 0 11 Nov 2024
TRACE: Temporal Grounding Video LLM via Causal Event Modeling Yongxin Guo Jingyu Liu Mingda Li Xiaoying Tang Qingbin Liu Xiaoying Tang 37 14 0 08 Oct 2024
Procedure-Aware Surgical Video-language Pretraining with Hierarchical Knowledge Augmentation Kun Yuan V. Srivastav Nassir Navab N. Padoy 44 7 0 30 Sep 2024
End-to-end Open-vocabulary Video Visual Relationship Detection using Multi-modal Prompting Yongqi Wang Xinxiao Wu Shuo Yang Jiebo Luo 117 1 0 19 Sep 2024
Lexicon3D: Probing Visual Foundation Models for Complex 3D Scene Understanding Yunze Man Shuhong Zheng Zhipeng Bao M. Hebert Liang-Yan Gui Yu-xiong Wang 70 15 0 05 Sep 2024
End-to-end Semantic-centric Video-based Multimodal Affective Computing Ronghao Lin Ying Zeng Sijie Mai Haifeng Hu VGen 40 0 0 14 Aug 2024
ExpertAF: Expert Actionable Feedback from Video Kumar Ashutosh Tushar Nagarajan Georgios Pavlakos Kris M. Kitani Kristen Grauman VGen 44 2 0 01 Aug 2024
Ego-VPA: Egocentric Video Understanding with Parameter-efficient Adaptation Tz-Ying Wu Kyle Min Subarna Tripathi Nuno Vasconcelos EgoV 53 0 0 28 Jul 2024
An Empirical Comparison of Video Frame Sampling Methods for Multi-Modal RAG Retrieval Mahesh Kandhare Thibault Gisselbrecht 35 5 0 22 Jul 2024
Bootstrapping Vision-language Models for Self-supervised Remote Physiological Measurement Zijie Yue Miaojing Shi Hanli Wang Shuai Ding Qijun Chen Shanlin Yang 37 0 0 11 Jul 2024
Sequential Contrastive Audio-Visual Learning Ioannis Tsiamas Santiago Pascual Chunghsin Yeh Joan Serra 33 2 0 08 Jul 2024
CLIPVQA:Video Quality Assessment via CLIP Fengchuang Xing Mingjie Li Yuan-Gen Wang Guopu Zhu Xiaochun Cao CLIP ViT 38 4 0 06 Jul 2024
Tarsier: Recipes for Training and Evaluating Large Video Description Models Jiawei Wang Liping Yuan Yuchen Zhang 33 52 0 30 Jun 2024
MuJo: Multimodal Joint Feature Space Learning for Human Activity Recognition Stefan Gerd Fritsch Cennet Oğuz Vitor Fortes Rey L. Ray Maximilian Kiefer-Emmanouilidis P. Lukowicz HAI 43 0 0 06 Jun 2024
Enhancing Zero-Shot Facial Expression Recognition by LLM Knowledge Transfer Zengqun Zhao Yu Cao Shaogang Gong Ioannis Patras 37 6 0 29 May 2024
Topological Perspectives on Optimal Multimodal Embedding Spaces Abdul Aziz Abdul Rahim BDL 37 0 0 29 May 2024
Instruct-ReID++: Towards Universal Purpose Instruction-Guided Person Re-identification Weizhen He Yiheng Deng Yunfeng Yan Feng Zhu Yizhou Wang Lei Bai Qingsong Xie Donglian Qi Wanli Ouyang Shixiang Tang 93 2 0 28 May 2024
Learning Manipulation Skills through Robot Chain-of-Thought with Sparse Failure Guidance Kaifeng Zhang Zhao-Heng Yin Weirui Ye Yang Gao 62 3 0 22 May 2024
MVP-Shot: Multi-Velocity Progressive-Alignment Framework for Few-Shot Action Recognition Hongyu Qu Rui Yan Xiangbo Shu Haoliang Gao Peng Huang Guo-Sen Xie 49 4 0 03 May 2024
Revisiting Text-to-Image Evaluation with Gecko: On Metrics, Prompts, and Human Ratings Olivia Wiles Chuhan Zhang Isabela Albuquerque Ivana Kajić Su Wang ... Jordi Pont-Tuset Aida Nematzadeh Anant Nawalgaria Jordi Pont-Tuset Aida Nematzadeh EGVM 120 13 0 25 Apr 2024
CatLIP: CLIP-level Visual Recognition Accuracy with 2.7x Faster Pre-training on Web-scale Image-Text Data Sachin Mehta Maxwell Horton Fartash Faghri Mohammad Hossein Sekhavat Mahyar Najibi Mehrdad Farajtabar Oncel Tuzel Mohammad Rastegari VLM CLIP 36 6 0 24 Apr 2024
CREST: Cross-modal Resonance through Evidential Deep Learning for Enhanced Zero-Shot Learning Haojian Huang Xiaozhen Qiao Zhuo Chen Haodong Chen Bingyu Li Zhe Sun Mulin. Chen Xuelong Li 29 10 0 15 Apr 2024
VideoDistill: Language-aware Vision Distillation for Video Question Answering Bo Zou Chao Yang Yu Qiao Chengbin Quan Youjian Zhao VGen 42 1 0 01 Apr 2024
Multi-Stage Multi-Modal Pre-Training for Automatic Speech Recognition Yash Jain David M. Chan Pranav Dheram Aparna Khare Olabanji Shonibare Venkatesh Ravichandran Shalini Ghosh 37 2 0 28 Mar 2024
EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real World Yifei Huang Guo Chen Jilan Xu Mingfang Zhang Lijin Yang ... Hongjie Zhang Lu Dong Yali Wang Limin Wang Yu Qiao EgoV 60 36 0 24 Mar 2024
N-Modal Contrastive Losses with Applications to Social Media Data in Trimodal Space William Theisen Walter J. Scheirer 26 1 0 18 Mar 2024
Unifying Latent and Lexicon Representations for Effective Video-Text Retrieval Haowei Liu Yaya Shi Haiyang Xu Chunfen Yuan Qinghao Ye ... Mingshi Yan Ji Zhang Fei Huang Bing Li Weiming Hu 30 0 0 26 Feb 2024
VideoPrism: A Foundational Visual Encoder for Video Understanding Long Zhao N. B. Gundavarapu Liangzhe Yuan Hao Zhou Shen Yan ... Huisheng Wang Hartwig Adam Mikhail Sirotenko Ting Liu Boqing Gong VGen 29 29 0 20 Feb 2024
Effective pruning of web-scale datasets based on complexity of concept clusters Amro Abbas E. Rusak Kushal Tirumala Wieland Brendel Kamalika Chaudhuri Ari S. Morcos VLM CLIP 34 22 0 09 Jan 2024
RGNet: A Unified Clip Retrieval and Grounding Network for Long Videos Tanveer Hannan Md. Mohaiminul Islam Thomas Seidl Gedas Bertasius 28 3 0 11 Dec 2023
Spacewalk-18: A Benchmark for Multimodal and Long-form Procedural Video Understanding in Novel Domains Rohan Myer Krishnan Zitian Tang Zhiqiu Yu Chen Sun 51 1 0 30 Nov 2023
Sinkhorn Transformations for Single-Query Postprocessing in Text-Video Retrieval Konstantin Yakovlev Gregory Polyakov I. Alimova Alexander Podolskiy A. Bout Sergey I. Nikolenko Irina Piontkovskaya CLIP 14 1 0 14 Nov 2023
OmniVec: Learning robust representations with cross modal sharing Siddharth Srivastava Gaurav Sharma SSL 21 64 0 07 Nov 2023
TESTA: Temporal-Spatial Token Aggregation for Long-form Video-Language Understanding Shuhuai Ren Sishuo Chen Shicheng Li Xu Sun Lu Hou ViT 41 28 0 29 Oct 2023
CAD -- Contextual Multi-modal Alignment for Dynamic AVQA Asmar Nadeem Adrian Hilton R. Dawes Graham A. Thomas A. Mustafa 21 9 0 25 Oct 2023
Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook Ming Jin Qingsong Wen Yuxuan Liang Chaoli Zhang Siqiao Xue ... Shirui Pan Vincent S. Tseng Yu Zheng Lei Chen Hui Xiong AI4TS SyDa 33 117 0 16 Oct 2023