HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training

1 May 2020

Papers citing "HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training"

50 / 175 papers shown

Title
MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretraining Xiaoyi Dong Jianmin Bao Yinglin Zheng Ting Zhang Dongdong Chen ... Weiming Zhang Lu Yuan Dong Chen Fang Wen Nenghai Yu CLIP VLM 54 159 0 25 Aug 2022
MuMUR : Multilingual Multimodal Universal Retrieval Avinash Madasu Estelle Aflalo Gabriela Ben-Melech Stan Shachar Rosenman Shao-Yen Tseng Gedas Bertasius Vasudev Lal 47 3 0 24 Aug 2022
Video Question Answering with Iterative Video-Text Co-Tokenization A. Piergiovanni K. Morton Weicheng Kuo Michael S. Ryoo A. Angelova 39 18 0 01 Aug 2022
Clover: Towards A Unified Video-Language Alignment and Fusion Model Jingjia Huang Yinan Li Jiashi Feng Xinglong Wu Xiaoshuai Sun Rongrong Ji VLM 29 48 0 16 Jul 2022
TS2-Net: Token Shift and Selection Transformer for Text-Video Retrieval Yuqi Liu Pengfei Xiong Luhui Xu Shengming Cao Qin Jin 41 114 0 16 Jul 2022
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval Yiwei Ma Guohai Xu Xiaoshuai Sun Ming Yan Ji Zhang Rongrong Ji CLIP VLM 34 272 0 15 Jul 2022
Self-Supervised Learning for Videos: A Survey Madeline Chantry Schiappa Yogesh S Rawat M. Shah SSL 41 131 0 18 Jun 2022
Zero-Shot Video Question Answering via Frozen Bidirectional Language Models Antoine Yang Antoine Miech Josef Sivic Ivan Laptev Cordelia Schmid 58 228 0 16 Jun 2022
LAVENDER: Unifying Video-Language Understanding as Masked Language Modeling Linjie Li Zhe Gan Kevin Qinghong Lin Chung-Ching Lin Zicheng Liu Ce Liu Lijuan Wang MLLM VLM 20 81 0 14 Jun 2022
Multimodal Learning with Transformers: A Survey Peng Xu Xiatian Zhu David Clifton ViT 79 530 0 13 Jun 2022
Revealing Single Frame Bias for Video-and-Language Learning Jie Lei Tamara L. Berg Joey Tianyi Zhou 24 111 0 07 Jun 2022
Revisiting the "Video" in Video-Language Understanding S. Buch Cristobal Eyzaguirre Adrien Gaidon Jiajun Wu L. Fei-Fei Juan Carlos Niebles 41 158 0 03 Jun 2022
GIT: A Generative Image-to-text Transformer for Vision and Language Jianfeng Wang Zhengyuan Yang Xiaowei Hu Linjie Li Kevin Qinghong Lin Zhe Gan Zicheng Liu Ce Liu Lijuan Wang VLM 61 529 0 27 May 2022
Learning to Retrieve Videos by Asking Questions Avinash Madasu Junier Oliva Gedas Bertasius VGen 32 16 0 11 May 2022
Learning to Answer Visual Questions from Web Videos Antoine Yang Antoine Miech Josef Sivic Ivan Laptev Cordelia Schmid ViT 42 33 0 10 May 2022
i-Code: An Integrative and Composable Multimodal Learning Framework Ziyi Yang Yuwei Fang Chenguang Zhu Reid Pryzant DongDong Chen ... Bin Xiao Yuanxun Lu Takuya Yoshioka Michael Zeng Xuedong Huang 40 45 0 03 May 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 51 3,369 0 29 Apr 2022
Relevance-based Margin for Contrastively-trained Video Retrieval Models Alex Falcon Swathikiran Sudhakaran G. Serra Sergio Escalera Oswald Lanz 40 7 0 27 Apr 2022
MILES: Visual BERT Pre-training with Injected Language Semantics for Video-text Retrieval Yuying Ge Yixiao Ge Xihui Liu Alex Jinpeng Wang Jianping Wu Ying Shan Xiaohu Qie Ping Luo VLM 18 44 0 26 Apr 2022
COTS: Collaborative Two-Stream Vision-Language Pre-Training Model for Cross-Modal Retrieval Haoyu Lu Nanyi Fei Yuqi Huo Yizhao Gao Zhiwu Lu Jiaxin Wen CLIP VLM 27 55 0 15 Apr 2022
Tencent Text-Video Retrieval: Hierarchical Cross-Modal Interactions with Multi-Level Representations Jie Jiang Shaobo Min Weijie Kong Dihong Gong Hongfa Wang Zhifeng Li Wei Liu VLM 20 18 0 07 Apr 2022
ECLIPSE: Efficient Long-range Video Retrieval using Sight and Sound Yan-Bo Lin Jie Lei Joey Tianyi Zhou Gedas Bertasius 54 39 0 06 Apr 2022
Modeling Motion with Multi-Modal Features for Text-Based Video Segmentation Wangbo Zhao Kai Wang Xiangxiang Chu Fuzhao Xue Xinchao Wang Yang You 29 21 0 06 Apr 2022
TubeDETR: Spatio-Temporal Video Grounding with Transformers Antoine Yang Antoine Miech Josef Sivic Ivan Laptev Cordelia Schmid ViT 32 94 0 30 Mar 2022
Disentangled Representation Learning for Text-Video Retrieval Qiang Wang Yanhao Zhang Yun Zheng Pan Pan Xiansheng Hua 45 77 0 14 Mar 2022
Video Question Answering: Datasets, Algorithms and Challenges Yaoyao Zhong Junbin Xiao Wei Ji Yicong Li Wei Deng Tat-Seng Chua 32 87 0 02 Mar 2022
VLP: A Survey on Vision-Language Pre-training Feilong Chen Duzhen Zhang Minglun Han Xiuyi Chen Jing Shi Shuang Xu Bo Xu VLM 82 213 0 18 Feb 2022
Learning To Recognize Procedural Activities with Distant Supervision Xudong Lin Fabio Petroni Gedas Bertasius Marcus Rohrbach Shih-Fu Chang Lorenzo Torresani 35 83 0 26 Jan 2022
End-to-end Generative Pretraining for Multimodal Video Captioning Paul Hongsuck Seo Arsha Nagrani Anurag Arnab Cordelia Schmid 27 164 0 20 Jan 2022
Temporal Sentence Grounding in Videos: A Survey and Future Directions Hao Zhang Aixin Sun Wei Jing Qiufeng Wang 3DGS 41 38 0 20 Jan 2022
Video Transformers: A Survey Javier Selva A. S. Johansen Sergio Escalera Kamal Nasrollahi T. Moeslund Albert Clapés ViT 24 103 0 16 Jan 2022
Bridging Video-text Retrieval with Multiple Choice Questions Yuying Ge Yixiao Ge Xihui Liu Dian Li Ying Shan Xiaohu Qie Ping Luo BDL 29 108 0 13 Jan 2022
Align and Prompt: Video-and-Language Pre-training with Entity Prompts Dongxu Li Junnan Li Hongdong Li Juan Carlos Niebles Guosheng Lin 28 191 0 17 Dec 2021
CoCo-BERT: Improving Video-Language Pre-training with Contrastive Cross-modal Matching and Denoising Jianjie Luo Yehao Li Yingwei Pan Ting Yao Hongyang Chao Tao Mei VLM 18 41 0 14 Dec 2021
VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks Yi-Lin Sung Jaemin Cho Joey Tianyi Zhou VLM VPVLM 37 344 0 13 Dec 2021
Video-Text Pre-training with Learned Regions Rui Yan Mike Zheng Shou Yixiao Ge Alex Jinpeng Wang Xudong Lin Guanyu Cai Jinhui Tang 33 23 0 02 Dec 2021
Object-aware Video-language Pre-training for Retrieval Alex Jinpeng Wang Yixiao Ge Guanyu Cai Rui Yan Xudong Lin Ying Shan Xiaohu Qie Mike Zheng Shou ViT VLM 25 79 0 01 Dec 2021
CLIP Meets Video Captioning: Concept-Aware Representation Learning Does Matter Bang-ju Yang Tong Zhang Yuexian Zou CLIP 25 20 0 30 Nov 2021
VL-LTR: Learning Class-wise Visual-Linguistic Representation for Long-Tailed Visual Recognition Changyao Tian Wenhai Wang Xizhou Zhu Jifeng Dai Yu Qiao VLM 32 70 0 26 Nov 2021
VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling Tsu-Jui Fu Linjie Li Zhe Gan Kevin Qinghong Lin Wenjie Wang Lijuan Wang Zicheng Liu VLM 53 218 0 24 Nov 2021
Scaling Up Vision-Language Pre-training for Image Captioning Xiaowei Hu Zhe Gan Jianfeng Wang Zhengyuan Yang Zicheng Liu Yumao Lu Lijuan Wang MLLM VLM 34 246 0 24 Nov 2021
Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions Hongwei Xue Tiankai Hang Yanhong Zeng Yuchong Sun Bei Liu Huan Yang Jianlong Fu B. Guo AI4TS VLM 31 189 0 19 Nov 2021
EMScore: Evaluating Video Captioning via Coarse-Grained and Fine-Grained Embedding Matching Yaya Shi Xu Yang Haiyang Xu Chunfen Yuan Bing Li Weiming Hu Zhengjun Zha 39 33 0 17 Nov 2021
Masking Modalities for Cross-modal Video Retrieval Valentin Gabeur Arsha Nagrani Chen Sun Alahari Karteek Cordelia Schmid 19 29 0 01 Nov 2021
MEmoBERT: Pre-training Model with Prompt-based Learning for Multimodal Emotion Recognition Jinming Zhao Ruichen Li Qin Jin Xinchao Wang Haizhou Li 19 25 0 27 Oct 2021
A CLIP-Enhanced Method for Video-Language Understanding Guohao Li Feng He Zhifan Feng CLIP 31 12 0 14 Oct 2021
VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding Hu Xu Gargi Ghosh Po-Yao (Bernie) Huang Dmytro Okhonko Armen Aghajanyan Florian Metze Luke Zettlemoyer Florian Metze Luke Zettlemoyer Christoph Feichtenhofer CLIP VLM 259 561 0 28 Sep 2021
Survey: Transformer based Video-Language Pre-training Ludan Ruan Qin Jin VLM ViT 72 44 0 21 Sep 2021
Dyadformer: A Multi-modal Transformer for Long-Range Modeling of Dyadic Interactions D. Curto Albert Clapés Javier Selva Sorina Smeureanu Julio C. S. Jacques Junior ... G. Guilera D. Leiva T. Moeslund Sergio Escalera Cristina Palmero 51 29 0 20 Sep 2021
M5Product: Self-harmonized Contrastive Learning for E-commercial Multi-modal Pretraining Xiao Dong Xunlin Zhan Yangxin Wu Yunchao Wei Michael C. Kampffmeyer Xiaoyong Wei Minlong Lu Yaowei Wang Xiaodan Liang 35 37 0 09 Sep 2021