VideoOFA: Two-Stage Pre-Training for Video-to-Text Generation

4 May 2023

Yashar Mehdad

Papers citing "VideoOFA: Two-Stage Pre-Training for Video-to-Text Generation"

31 / 31 papers shown

Title
GIT: A Generative Image-to-text Transformer for Vision and Language Jianfeng Wang Zhengyuan Yang Xiaowei Hu Linjie Li Kevin Qinghong Lin Zhe Gan Zicheng Liu Ce Liu Lijuan Wang VLM 129 552 0 27 May 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 382 3,535 0 29 Apr 2022
All in One: Exploring Unified Video-Language Pre-training Alex Jinpeng Wang Yixiao Ge Rui Yan Yuying Ge Xudong Lin Guanyu Cai Jianping Wu Ying Shan Xiaohu Qie Mike Zheng Shou 80 201 0 14 Mar 2022
OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework Peng Wang An Yang Rui Men Junyang Lin Shuai Bai Zhikang Li Jianxin Ma Chang Zhou Jingren Zhou Hongxia Yang MLLM ObjD 147 873 0 07 Feb 2022
End-to-end Generative Pretraining for Multimodal Video Captioning Paul Hongsuck Seo Arsha Nagrani Anurag Arnab Cordelia Schmid 68 168 0 20 Jan 2022
SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning Kevin Qinghong Lin Linjie Li Chung-Ching Lin Faisal Ahmed Zhe Gan Zicheng Liu Yumao Lu Lijuan Wang ViT 78 243 0 25 Nov 2021
VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling Tsu-Jui Fu Linjie Li Zhe Gan Kevin Qinghong Lin Wenjie Wang Lijuan Wang Zicheng Liu VLM 95 221 0 24 Nov 2021
UniTAB: Unifying Text and Box Outputs for Grounded Vision-Language Modeling Zhengyuan Yang Zhe Gan Jianfeng Wang Xiaowei Hu Faisal Ahmed Zicheng Liu Yumao Lu Lijuan Wang 87 116 0 23 Nov 2021
Video Swin Transformer Ze Liu Jia Ning Yue Cao Yixuan Wei Zheng Zhang Stephen Lin Han Hu ViT 100 1,481 0 24 Jun 2021
VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation Linjie Li Jie Lei Zhe Gan Licheng Yu Yen-Chun Chen ... Tamara L. Berg Joey Tianyi Zhou Jingjing Liu Lijuan Wang Zicheng Liu VLM 85 100 0 08 Jun 2021
ViViT: A Video Vision Transformer Anurag Arnab Mostafa Dehghani G. Heigold Chen Sun Mario Lucic Cordelia Schmid ViT 222 2,149 0 29 Mar 2021
Open-book Video Captioning with Retrieve-Copy-Generate Network Ziqi Zhang Zhongang Qi Chun Yuan Ying Shan Bing Li Ying Deng Weiming Hu 52 94 0 09 Mar 2021
Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling Jie Lei Linjie Li Luowei Zhou Zhe Gan Tamara L. Berg Joey Tianyi Zhou Jingjing Liu CLIP 121 661 0 11 Feb 2021
ActBERT: Learning Global-Local Video-Text Representations Linchao Zhu Yi Yang ViT 117 421 0 14 Nov 2020
A Short Note on the Kinetics-700-2020 Human Action Dataset Lucas Smaira João Carreira Eric Noland Ellen Clancy Amy Wu Andrew Zisserman 82 139 0 21 Oct 2020
Learning Modality Interaction for Temporal Sentence Localization and Event Captioning in Videos Shaoxiang Chen Wenhao Jiang Wei Liu Yu-Gang Jiang 62 102 0 28 Jul 2020
HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training Linjie Li Yen-Chun Chen Yu Cheng Zhe Gan Licheng Yu Jingjing Liu MLLM VLM OffRL AI4TS 108 503 0 01 May 2020
Don't Stop Pretraining: Adapt Language Models to Domains and Tasks Suchin Gururangan Ana Marasović Swabha Swayamdipta Kyle Lo Iz Beltagy Doug Downey Noah A. Smith VLM AI4CE CLL 152 2,424 0 23 Apr 2020
Spatio-Temporal Graph for Video Captioning with Knowledge Distillation Boxiao Pan Haoye Cai De-An Huang Kuan-Hui Lee Adrien Gaidon Ehsan Adeli Juan Carlos Niebles 56 236 0 31 Mar 2020
Object Relational Graph with Teacher-Recommended Learning for Video Captioning Ziqi Zhang Yaya Shi Chunfen Yuan Bing Li Peijin Wang Weiming Hu Zhengjun Zha VLM 78 272 0 26 Feb 2020
VL-BERT: Pre-training of Generic Visual-Linguistic Representations Weijie Su Xizhou Zhu Yue Cao Bin Li Lewei Lu Furu Wei Jifeng Dai VLM MLLM SSL 153 1,663 0 22 Aug 2019
LXMERT: Learning Cross-Modality Encoder Representations from Transformers Hao Hao Tan Joey Tianyi Zhou VLM MLLM 237 2,479 0 20 Aug 2019
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips Antoine Miech Dimitri Zhukov Jean-Baptiste Alayrac Makarand Tapaswi Ivan Laptev Josef Sivic VGen 110 1,200 0 07 Jun 2019
VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research Xin Eric Wang Jiawei Wu Junkun Chen Lei Li Yuan-fang Wang William Yang Wang 99 550 0 06 Apr 2019
fairseq: A Fast, Extensible Toolkit for Sequence Modeling Myle Ott Sergey Edunov Alexei Baevski Angela Fan Sam Gross Nathan Ng David Grangier Michael Auli VLM FaML 105 3,149 0 01 Apr 2019
Self-critical Sequence Training for Image Captioning Steven J. Rennie E. Marcheret Youssef Mroueh Jerret Ross Vaibhava Goel 107 1,887 0 02 Dec 2016
Leveraging Video Descriptions to Learn Video Question Answering Kuo-Hao Zeng Tseng-Hung Chen Ching-Yao Chuang Yuan-Hong Liao Juan Carlos Niebles Min Sun 84 179 0 12 Nov 2016
Describing Videos by Exploiting Temporal Structure L. Yao Atousa Torabi Kyunghyun Cho Nicolas Ballas C. Pal Hugo Larochelle Aaron Courville 144 1,064 0 27 Feb 2015
Translating Videos to Natural Language Using Deep Recurrent Neural Networks Subhashini Venugopalan Huijuan Xu Jeff Donahue Marcus Rohrbach Raymond J. Mooney Kate Saenko 135 952 0 15 Dec 2014
CIDEr: Consensus-based Image Description Evaluation Ramakrishna Vedantam C. L. Zitnick Devi Parikh 286 4,484 0 20 Nov 2014
Sequence to Sequence Learning with Neural Networks Ilya Sutskever Oriol Vinyals Quoc V. Le AIMat 434 20,553 0 10 Sep 2014