Unified Vision-Language Pre-Training for Image Captioning and VQA

24 September 2019

Lei Zhang

Papers citing "Unified Vision-Language Pre-Training for Image Captioning and VQA"

50 / 250 papers shown

Title
LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval Siqi Sun Yen-Chun Chen Linjie Li Shuohang Wang Yuwei Fang Jingjing Liu VLM 41 82 0 16 Mar 2021
Causal Attention for Vision-Language Tasks Xu Yang Hanwang Zhang Guojun Qi Jianfei Cai CML 36 150 0 05 Mar 2021
WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning Krishna Srinivasan K. Raman Jiecao Chen Michael Bendersky Marc Najork VLM 216 311 0 02 Mar 2021
CrossMap Transformer: A Crossmodal Masked Path Transformer Using Double Back-Translation for Vision-and-Language Navigation A. Magassouba K. Sugiura Hisashi Kawai 53 10 0 01 Mar 2021
M6: A Chinese Multimodal Pretrainer Junyang Lin Rui Men An Yang Chan Zhou Ming Ding ... Yong Li Wei Lin Jingren Zhou J. Tang Hongxia Yang VLM MoE 37 133 0 01 Mar 2021
Natural Language Video Localization: A Revisit in Span-based Question Answering Framework Hao Zhang Aixin Sun Wei Jing Liangli Zhen Qiufeng Wang Rick Siow Mong Goh 113 85 0 26 Feb 2021
UniT: Multimodal Multitask Learning with a Unified Transformer Ronghang Hu Amanpreet Singh ViT 30 296 0 22 Feb 2021
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 323 1,090 0 17 Feb 2021
Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling Jie Lei Linjie Li Luowei Zhou Zhe Gan Tamara L. Berg Joey Tianyi Zhou Jingjing Liu CLIP 46 649 0 11 Feb 2021
Iconographic Image Captioning for Artworks E. Cetinic 32 24 0 07 Feb 2021
Unifying Vision-and-Language Tasks via Text Generation Jaemin Cho Jie Lei Hao Tan Joey Tianyi Zhou MLLM 277 527 0 04 Feb 2021
Latent Variable Models for Visual Question Answering Zixu Wang Yishu Miao Lucia Specia 25 5 0 16 Jan 2021
Transformers in Vision: A Survey Salman Khan Muzammal Naseer Munawar Hayat Syed Waqas Zamir Fahad Shahbaz Khan M. Shah ViT 233 2,434 0 04 Jan 2021
KM-BART: Knowledge Enhanced Multimodal BART for Visual Commonsense Generation Yiran Xing Z. Shi Zhao Meng Gerhard Lakemeyer Yunpu Ma Roger Wattenhofer VLM 74 40 0 02 Jan 2021
A Closer Look at the Robustness of Vision-and-Language Pre-trained Models Linjie Li Zhe Gan Jingjing Liu VLM 33 42 0 15 Dec 2020
MiniVLM: A Smaller and Faster Vision-Language Model Jianfeng Wang Xiaowei Hu Pengchuan Zhang Xiujun Li Lijuan Wang Lefei Zhang Jianfeng Gao Zicheng Liu VLM MLLM 35 59 0 13 Dec 2020
TAP: Text-Aware Pre-training for Text-VQA and Text-Caption Zhengyuan Yang Yijuan Lu Jianfeng Wang Xi Yin D. Florêncio Lijuan Wang Cha Zhang Lei Zhang Jiebo Luo VLM 28 141 0 08 Dec 2020
Multimodal Pretraining Unmasked: A Meta-Analysis and a Unified Framework of Vision-and-Language BERTs Emanuele Bugliarello Ryan Cotterell Naoaki Okazaki Desmond Elliott 35 119 0 30 Nov 2020
CapWAP: Captioning with a Purpose Adam Fisch Kenton Lee Ming-Wei Chang J. Clark Regina Barzilay 13 11 0 09 Nov 2020
Cross-Media Keyphrase Prediction: A Unified Framework with Multi-Modality Multi-Head Attention and Image Wordings Yue Wang Jing Li M. Lyu Irwin King 19 16 0 03 Nov 2020
COOT: Cooperative Hierarchical Transformer for Video-Text Representation Learning Simon Ging Mohammadreza Zolfaghari Hamed Pirsiavash Thomas Brox ViT CLIP 35 169 0 01 Nov 2020
Learning Dual Semantic Relations with Graph Attention for Image-Text Matching Keyu Wen Xiaodong Gu Qingrong Cheng 27 95 0 22 Oct 2020
X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal Transformers Jaemin Cho Jiasen Lu Dustin Schwenk Hannaneh Hajishirzi Aniruddha Kembhavi VLM MLLM 30 102 0 23 Sep 2020
A Multimodal Memes Classification: A Survey and Open Research Issues Tariq Habib Afridi A. Alam Muhammad Numan Khan Jawad Khan Young-Koo Lee 29 37 0 17 Sep 2020
Polysemy Deciphering Network for Robust Human-Object Interaction Detection Xubin Zhong Changxing Ding X. Qu Dacheng Tao 27 58 0 07 Aug 2020
Spatially Aware Multimodal Transformers for TextVQA Yash Kant Dhruv Batra Peter Anderson Alex Schwing Devi Parikh Jiasen Lu Harsh Agrawal 27 85 0 23 Jul 2020
Length-Controllable Image Captioning Chaorui Deng Ning Ding Mingkui Tan Qi Wu VLM 33 56 0 19 Jul 2020
ERNIE-ViL: Knowledge Enhanced Vision-Language Representations Through Scene Graph Fei Yu Jiji Tang Weichong Yin Yu Sun Hao Tian Hua Wu Haifeng Wang 31 377 0 30 Jun 2020
Neuro-Symbolic Visual Reasoning: Disentangling "Visual" from "Reasoning" Saeed Amizadeh Hamid Palangi Oleksandr Polozov Yichen Huang K. Koishida NAI LRM 44 58 0 20 Jun 2020
Contrastive Learning for Weakly Supervised Phrase Grounding Tanmay Gupta Arash Vahdat Gal Chechik Xiaodong Yang Jan Kautz Derek Hoiem ObjD SSL 47 141 0 17 Jun 2020
VirTex: Learning Visual Representations from Textual Annotations Karan Desai Justin Johnson SSL VLM 30 433 0 11 Jun 2020
Large-Scale Adversarial Training for Vision-and-Language Representation Learning Zhe Gan Yen-Chun Chen Linjie Li Chen Zhu Yu Cheng Jingjing Liu ObjD VLM 35 489 0 11 Jun 2020
Emergent Multi-Agent Communication in the Deep Learning Era Angeliki Lazaridou Marco Baroni AI4CE 48 197 0 03 Jun 2020
Graph Density-Aware Losses for Novel Compositions in Scene Graph Generation Boris Knyazev H. D. Vries Cătălina Cangea Graham W. Taylor Aaron Courville Eugene Belilovsky 32 56 0 17 May 2020
Behind the Scene: Revealing the Secrets of Pre-trained Vision-and-Language Models Jize Cao Zhe Gan Yu Cheng Licheng Yu Yen-Chun Chen Jingjing Liu VLM 24 127 0 15 May 2020
HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training Linjie Li Yen-Chun Chen Yu Cheng Zhe Gan Licheng Yu Jingjing Liu MLLM VLM OffRL AI4TS 64 494 0 01 May 2020
Improving Vision-and-Language Navigation with Image-Text Pairs from the Web Arjun Majumdar Ayush Shrivastava Stefan Lee Peter Anderson Devi Parikh Dhruv Batra LM&Ro 52 230 0 30 Apr 2020
VD-BERT: A Unified Vision and Dialog Transformer with BERT Yue Wang Chenyu You Michael R. Lyu Irwin King Caiming Xiong Guosheng Lin 24 102 0 28 Apr 2020
Experience Grounds Language Yonatan Bisk Ari Holtzman Jesse Thomason Jacob Andreas Yoshua Bengio ... Angeliki Lazaridou Jonathan May Aleksandr Nisnevich Nicolas Pinto Joseph P. Turian 24 351 0 21 Apr 2020
Are we pretraining it right? Digging deeper into visio-linguistic pretraining Amanpreet Singh Vedanuj Goswami Devi Parikh VLM 46 48 0 19 Apr 2020
XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation Yaobo Liang Nan Duan Yeyun Gong Ning Wu Fenfei Guo ... Shuguang Liu Fan Yang Daniel Fernando Campos Rangan Majumder Ming Zhou ELM VLM 63 343 0 03 Apr 2020
Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers Zhicheng Huang Zhaoyang Zeng Bei Liu Dongmei Fu Jianlong Fu ViT 50 436 0 02 Apr 2020
XGPT: Cross-modal Generative Pre-Training for Image Captioning Qiaolin Xia Haoyang Huang Nan Duan Dongdong Zhang Lei Ji Zhifang Sui Edward Cui Taroon Bharti Xin Liu Ming Zhou MLLM VLM 27 74 0 03 Mar 2020
Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-training Weituo Hao Chunyuan Li Xiujun Li Lawrence Carin Jianfeng Gao LM&Ro 20 274 0 25 Feb 2020
ImageBERT: Cross-modal Pre-training with Large-scale Weak-supervised Image-Text Data Di Qi Lin Su Jianwei Song Edward Cui Taroon Bharti Arun Sacheti VLM 40 259 0 22 Jan 2020
In Defense of Grid Features for Visual Question Answering Huaizu Jiang Ishan Misra Marcus Rohrbach Erik Learned-Miller Xinlei Chen OOD ObjD 23 320 0 10 Jan 2020
Probing Contextualized Sentence Representations with Visual Awareness Zhuosheng Zhang Rui Wang Kehai Chen Masao Utiyama Eiichiro Sumita Hai Zhao 19 2 0 07 Nov 2019
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks Jiasen Lu Dhruv Batra Devi Parikh Stefan Lee SSL VLM 147 3,645 0 06 Aug 2019
Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods Aditya Mogadala M. Kalimuthu Dietrich Klakow VLM 32 133 0 22 Jul 2019
Neural Baby Talk Jiasen Lu Jianwei Yang Dhruv Batra Devi Parikh VLM 202 434 0 27 Mar 2018