Clover: Towards A Unified Video-Language Alignment and Fusion Model

16 July 2022

Papers citing "Clover: Towards A Unified Video-Language Alignment and Fusion Model"

50 / 54 papers shown

Title
MultiVENT 2.0: A Massive Multilingual Benchmark for Event-Centric Video Retrieval Reno Kriz Kate Sanders David Etter Kenton W. Murray Cameron Carpenter ... Alexander Martin Ronald Colaianni Nolan King Eugene Yang Benjamin Van Durme VGen 128 3 0 15 Oct 2024
Revisiting Temporal Modeling for CLIP-based Image-to-Video Knowledge Transferring Ruyang Liu Jingjia Huang Ge Li Jiashi Feng Xing Wu Thomas H. Li AI4TS CLIP VLM 68 48 0 26 Jan 2023
InternVideo: General Video Foundation Models via Generative and Discriminative Learning Yi Wang Kunchang Li Yizhuo Li Yinan He Bingkun Huang ... Junting Pan Jiashuo Yu Yali Wang Limin Wang Yu Qiao VLM VGen 115 326 0 06 Dec 2022
Text-Adaptive Multiple Visual Prototype Matching for Video-Text Retrieval Che-Hsien Lin Ancong Wu Junwei Liang Jun Zhang Wenhang Ge Wei Zheng Chunhua Shen 114 24 0 27 Sep 2022
CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignment Hongwei Xue Yuchong Sun Bei Liu Jianlong Fu Rui Song Houqiang Li Jiebo Luo CLIP VLM 55 70 0 14 Sep 2022
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval Yiwei Ma Guohai Xu Xiaoshuai Sun Ming Yan Ji Zhang Rongrong Ji CLIP VLM 69 283 0 15 Jul 2022
MILES: Visual BERT Pre-training with Injected Language Semantics for Video-text Retrieval Yuying Ge Yixiao Ge Xihui Liu Alex Jinpeng Wang Jianping Wu Ying Shan Xiaohu Qie Ping Luo VLM 44 44 0 26 Apr 2022
Tencent Text-Video Retrieval: Hierarchical Cross-Modal Interactions with Multi-Level Representations Jie Jiang Shaobo Min Weijie Kong Dihong Gong Hongfa Wang Zhifeng Li Wei Liu VLM 64 19 0 07 Apr 2022
X-Pool: Cross-Modal Language-Video Attention for Text-Video Retrieval S. Gorti Noël Vouitsis Junwei Ma Keyvan Golestan M. Volkovs Animesh Garg Guangwei Yu 75 160 0 28 Mar 2022
All in One: Exploring Unified Video-Language Pre-training Alex Jinpeng Wang Yixiao Ge Rui Yan Yuying Ge Xudong Lin Guanyu Cai Jianping Wu Ying Shan Xiaohu Qie Mike Zheng Shou 80 201 0 14 Mar 2022
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization Alexander Kunitsyn M. Kalashnikov Maksim Dzabraev Andrei Ivaniuta 53 17 0 14 Mar 2022
Bridging Video-text Retrieval with Multiple Choice Questions Yuying Ge Yixiao Ge Xihui Liu Dian Li Ying Shan Xiaohu Qie Ping Luo BDL 65 109 0 13 Jan 2022
Align and Prompt: Video-and-Language Pre-training with Entity Prompts Dongxu Li Junnan Li Hongdong Li Juan Carlos Niebles Guosheng Lin 75 193 0 17 Dec 2021
Object-aware Video-language Pre-training for Retrieval Alex Jinpeng Wang Yixiao Ge Guanyu Cai Rui Yan Xudong Lin Ying Shan Xiaohu Qie Mike Zheng Shou ViT VLM 60 80 0 01 Dec 2021
VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling Tsu-Jui Fu Linjie Li Zhe Gan Kevin Qinghong Lin Wenjie Wang Lijuan Wang Zicheng Liu VLM 87 221 0 24 Nov 2021
Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions Hongwei Xue Tiankai Hang Yanhong Zeng Yuchong Sun Bei Liu Huan Yang Jianlong Fu B. Guo AI4TS VLM 68 193 0 19 Nov 2021
VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding Hu Xu Gargi Ghosh Po-Yao (Bernie) Huang Dmytro Okhonko Armen Aghajanyan Florian Metze Luke Zettlemoyer Florian Metze Luke Zettlemoyer Christoph Feichtenhofer CLIP VLM 309 578 0 28 Sep 2021
Improving Video-Text Retrieval by Multi-Stream Corpus Alignment and Dual Softmax Loss Xingyi Cheng Hezheng Lin Xiangyu Wu Fan Yang Dong Shen 51 154 0 09 Sep 2021
TACo: Token-aware Cascade Contrastive Learning for Video-Text Alignment Jianwei Yang Yonatan Bisk Jianfeng Gao 81 140 0 23 Aug 2021
Align before Fuse: Vision and Language Representation Learning with Momentum Distillation Junnan Li Ramprasaath R. Selvaraju Akhilesh Deepak Gotmare Shafiq Joty Caiming Xiong Guosheng Lin FaML 186 1,953 0 16 Jul 2021
Video Swin Transformer Ze Liu Jia Ning Yue Cao Yixuan Wei Zheng Zhang Stephen Lin Han Hu ViT 94 1,481 0 24 Jun 2021
BEiT: BERT Pre-Training of Image Transformers Hangbo Bao Li Dong Songhao Piao Furu Wei ViT 258 2,824 0 15 Jun 2021
MERLOT: Multimodal Neural Script Knowledge Models Rowan Zellers Ximing Lu Jack Hessel Youngjae Yu J. S. Park Jize Cao Ali Farhadi Yejin Choi VLM LRM 95 381 0 04 Jun 2021
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding Hu Xu Gargi Ghosh Po-Yao (Bernie) Huang Prahal Arora Masoumeh Aminzadeh Christoph Feichtenhofer Florian Metze Luke Zettlemoyer 45 132 0 20 May 2021
Bridge to Answer: Structure-aware Graph Interaction Network for Video Question Answering Jungin Park Jiyoung Lee Kwanghoon Sohn 230 100 0 29 Apr 2021
CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval Huaishao Luo Lei Ji Ming Zhong Yang Chen Wen Lei Nan Duan Tianrui Li CLIP VLM 408 805 0 18 Apr 2021
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval Max Bain Arsha Nagrani Gül Varol Andrew Zisserman VGen 136 1,173 0 01 Apr 2021
Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling Jie Lei Linjie Li Luowei Zhou Zhe Gan Tamara L. Berg Joey Tianyi Zhou Jingjing Liu CLIP 116 661 0 11 Feb 2021
Look Before you Speak: Visually Contextualized Utterances Paul Hongsuck Seo Arsha Nagrani Cordelia Schmid 45 67 0 10 Dec 2020
ActBERT: Learning Global-Local Video-Text Representations Linchao Zhu Yi Yang ViT 117 421 0 14 Nov 2020
COOT: Cooperative Hierarchical Transformer for Video-Text Representation Learning Simon Ging Mohammadreza Zolfaghari Hamed Pirsiavash Thomas Brox ViT CLIP 68 172 0 01 Nov 2020
Location-aware Graph Convolutional Networks for Video Question Answering Deng Huang Peihao Chen Runhao Zeng Qing Du Mingkui Tan Chuang Gan GNN BDL 83 175 0 07 Aug 2020
Multi-modal Transformer for Video Retrieval Valentin Gabeur Chen Sun Alahari Karteek Cordelia Schmid ViT 526 608 0 21 Jul 2020
AVLnet: Learning Audio-Visual Language Representations from Instructional Videos Andrew Rouditchenko Angie Boggust David Harwath Brian Chen D. Joshi ... Rogerio Feris Brian Kingsbury M. Picheny Antonio Torralba James R. Glass SSL 59 142 0 16 Jun 2020
HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training Linjie Li Yen-Chun Chen Yu Cheng Zhe Gan Licheng Yu Jingjing Liu MLLM VLM OffRL AI4TS 106 503 0 01 May 2020
Hierarchical Conditional Relation Networks for Video Question Answering T. Le Vuong Le Svetha Venkatesh T. Tran 77 259 0 25 Feb 2020
UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation Huaishao Luo Lei Ji Botian Shi Haoyang Huang Nan Duan Tianrui Li Jason Li Xilin Chen Ming Zhou VLM 85 439 0 15 Feb 2020
End-to-End Learning of Visual Representations from Uncurated Instructional Videos Antoine Miech Jean-Baptiste Alayrac Lucas Smaira Ivan Laptev Josef Sivic Andrew Zisserman VGen SSL 114 711 0 13 Dec 2019
LXMERT: Learning Cross-Modality Encoder Representations from Transformers Hao Hao Tan Joey Tianyi Zhou VLM MLLM 237 2,479 0 20 Aug 2019
Use What You Have: Video Retrieval Using Representations From Collaborative Experts Yang Liu Samuel Albanie Arsha Nagrani Andrew Zisserman 73 389 0 31 Jul 2019
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips Antoine Miech Dimitri Zhukov Jean-Baptiste Alayrac Makarand Tapaswi Ivan Laptev Josef Sivic VGen 108 1,200 0 07 Jun 2019
Progressive Attention Memory Network for Movie Story Question Answering Junyeong Kim Minuk Ma Kyungsu Kim Sungjin Kim Chang D. Yoo 80 76 0 18 Apr 2019
Heterogeneous Memory Enhanced Multimodal Attention Model for Video Question Answering Chenyou Fan Xiaofan Zhang Shu Zhang Wensheng Wang Chi Zhang Heng-Chiao Huang 47 278 0 08 Apr 2019
VideoBERT: A Joint Model for Video and Language Representation Learning Chen Sun Austin Myers Carl Vondrick Kevin Patrick Murphy Cordelia Schmid VLM SSL 77 1,246 0 03 Apr 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.7K 94,770 0 11 Oct 2018
A Joint Sequence Fusion Model for Video Question Answering and Retrieval Youngjae Yu Jongseok Kim Gunhee Kim 77 345 0 07 Aug 2018
Representation Learning with Contrastive Predictive Coding Aaron van den Oord Yazhe Li Oriol Vinyals DRL SSL 306 10,284 0 10 Jul 2018
Localizing Moments in Video with Natural Language Lisa Anne Hendricks Oliver Wang Eli Shechtman Josef Sivic Trevor Darrell Bryan C. Russell 110 946 0 04 Aug 2017
The Kinetics Human Action Video Dataset W. Kay João Carreira Karen Simonyan Brian Zhang Chloe Hillier ... Tim Green T. Back Apostol Natsev Mustafa Suleyman Andrew Zisserman 250 3,802 0 19 May 2017
TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answering Y. Jang Yale Song Youngjae Yu Youngjin Kim Gunhee Kim 72 555 0 14 Apr 2017