Unified Vision-Language Pre-Training for Image Captioning and VQA

24 September 2019

Lei Zhang

Papers citing "Unified Vision-Language Pre-Training for Image Captioning and VQA"

50 / 254 papers shown

Title
Injecting Semantic Concepts into End-to-End Image Captioning Zhiyuan Fang Jianfeng Wang Xiaowei Hu Lin Liang Zhe Gan Lijuan Wang Yezhou Yang Zicheng Liu ViT VLM 40 86 0 09 Dec 2021
FLAVA: A Foundational Language And Vision Alignment Model Amanpreet Singh Ronghang Hu Vedanuj Goswami Guillaume Couairon Wojciech Galuba Marcus Rohrbach Douwe Kiela CLIP VLM 42 693 0 08 Dec 2021
Grounded Language-Image Pre-training Liunian Harold Li Pengchuan Zhang Haotian Zhang Jianwei Yang Chunyuan Li ... Lu Yuan Lei Zhang Lei Li Kai-Wei Chang Jianfeng Gao ObjD VLM 36 1,026 0 07 Dec 2021
Joint Learning of Localized Representations from Medical Images and Reports Philipp Muller Georgios Kaissis Cong Zou Daniel Munich 140 81 0 06 Dec 2021
Uni-Perceiver: Pre-training Unified Architecture for Generic Perception for Zero-shot and Few-shot Tasks Xizhou Zhu Jinguo Zhu Hao Li Xiaoshi Wu Xiaogang Wang Hongsheng Li Xiaohua Wang Jifeng Dai 56 129 0 02 Dec 2021
Object-Centric Unsupervised Image Captioning Zihang Meng David Yang Xuefei Cao Ashish Shah Ser-Nam Lim OCL VLM 32 11 0 02 Dec 2021
Generating More Pertinent Captions by Leveraging Semantics and Style on Multi-Source Datasets Marcella Cornia Lorenzo Baraldi G. Fiameni Rita Cucchiara 36 12 0 24 Nov 2021
VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling Tsu-Jui Fu Linjie Li Zhe Gan Kevin Qinghong Lin Wenjie Wang Lijuan Wang Zicheng Liu VLM 58 219 0 24 Nov 2021
Scaling Up Vision-Language Pre-training for Image Captioning Xiaowei Hu Zhe Gan Jianfeng Wang Zhengyuan Yang Zicheng Liu Yumao Lu Lijuan Wang MLLM VLM 39 246 0 24 Nov 2021
RedCaps: web-curated image-text data created by the people, for the people Karan Desai Gaurav Kaul Zubin Aysola Justin Johnson 31 162 0 22 Nov 2021
UFO: A UniFied TransfOrmer for Vision-Language Representation Learning Jianfeng Wang Xiaowei Hu Zhe Gan Zhengyuan Yang Xiyang Dai Zicheng Liu Yumao Lu Lijuan Wang ViT 33 57 0 19 Nov 2021
Transparent Human Evaluation for Image Captioning Jungo Kasai Keisuke Sakaguchi Lavinia Dunagan Jacob Morrison Ronan Le Bras Yejin Choi Noah A. Smith 33 47 0 17 Nov 2021
A Survey of Visual Transformers Yang Liu Yao Zhang Yixin Wang Feng Hou Jin Yuan Jiang Tian Yang Zhang Zhongchao Shi Jianping Fan Zhiqiang He 3DGS ViT 79 332 0 11 Nov 2021
Unifying Multimodal Transformer for Bi-directional Image and Text Generation Yupan Huang Hongwei Xue Bei Liu Yutong Lu 26 57 0 19 Oct 2021
Unsupervised Finetuning Suichan Li Dongdong Chen Yinpeng Chen Lu Yuan Lei Zhang Qi Chu B. Liu Nenghai Yu 30 8 0 18 Oct 2021
Self-Annotated Training for Controllable Image Captioning Zhangzi Zhu Tianlei Wang Hong Qu 37 2 0 16 Oct 2021
CLIP4Caption: CLIP for Video Caption Mingkang Tang Zhanyu Wang Zhenhua Liu Fengyun Rao Dian Li Xiu Li CLIP VLM 37 150 0 13 Oct 2021
$Pano-AVQA: Grounded Audio-Visual Question Answering on 360$^\circ$ Videos$ Pano-AVQA: Grounded Audio-Visual Question Answering on 360 $^\circ$ Videos Heeseung Yun Youngjae Yu Wonsuk Yang Kangil Lee Gunhee Kim 51 79 0 11 Oct 2021
Exploring Teacher-Student Learning Approach for Multi-lingual Speech-to-Intent Classification Bidisha Sharma Maulik C. Madhavi Xuehao Zhou Haizhou Li 28 2 0 28 Sep 2021
Dependency Induction Through the Lens of Visual Perception Ruisi Su Shruti Rijhwani Hao Zhu Junxian He Xinyu Wang Yonatan Bisk Graham Neubig 46 2 0 20 Sep 2021
Vision Guided Generative Pre-trained Language Models for Multimodal Abstractive Summarization Tiezheng Yu Wenliang Dai Zihan Liu Pascale Fung 37 73 0 06 Sep 2021
Learning to Generate Scene Graph from Natural Language Supervision Yiwu Zhong Jing Shi Jianwei Yang Chenliang Xu Yin Li SSL 53 77 0 06 Sep 2021
Product-oriented Machine Translation with Cross-modal Cross-lingual Pre-training Yuqing Song Shizhe Chen Qin Jin Wei Luo Jun Xie Fei Huang 46 19 0 25 Aug 2021
TACo: Token-aware Cascade Contrastive Learning for Video-Text Alignment Jianwei Yang Yonatan Bisk Jianfeng Gao 27 138 0 23 Aug 2021
Knowledge Perceived Multi-modal Pretraining in E-commerce Yushan Zhu Huaixiao Tou Wen Zhang Ganqiang Ye Hui Chen Ningyu Zhang Huajun Chen 36 33 0 20 Aug 2021
X-modaler: A Versatile and High-performance Codebase for Cross-modal Analytics Yehao Li Yingwei Pan Jingwen Chen Ting Yao Tao Mei VLM 21 31 0 18 Aug 2021
Knowledge Distillation from BERT Transformer to Speech Transformer for Intent Classification Yiding Jiang Bidisha Sharma Maulik C. Madhavi Haizhou Li 41 25 0 05 Aug 2021
Exceeding the Limits of Visual-Linguistic Multi-Task Learning Cameron R. Wolfe Keld T. Lundgaard VLM 50 2 0 27 Jul 2021
From Show to Tell: A Survey on Deep Learning-based Image Captioning Matteo Stefanini Marcella Cornia Lorenzo Baraldi S. Cascianelli G. Fiameni Rita Cucchiara 3DV VLM MLLM 69 258 0 14 Jul 2021
How Much Can CLIP Benefit Vision-and-Language Tasks? Sheng Shen Liunian Harold Li Hao Tan Joey Tianyi Zhou Anna Rohrbach Kai-Wei Chang Z. Yao Kurt Keutzer CLIP VLM MLLM 206 407 0 13 Jul 2021
End-to-end Multi-modal Video Temporal Grounding Yi-Wen Chen Yi-Hsuan Tsai Ming-Hsuan Yang 11 51 0 12 Jul 2021
OPT: Omni-Perception Pre-Trainer for Cross-Modal Understanding and Generation Jing Liu Xinxin Zhu Fei Liu Longteng Guo Zijia Zhao ... Weining Wang Hanqing Lu Shiyu Zhou Jiajun Zhang Jinqiao Wang 44 37 0 01 Jul 2021
Probing Inter-modality: Visual Parsing with Self-Attention for Vision-Language Pre-training Hongwei Xue Yupan Huang Bei Liu Houwen Peng Jianlong Fu Houqiang Li Jiebo Luo 38 89 0 25 Jun 2021
A Transformer-based Cross-modal Fusion Model with Adversarial Training for VQA Challenge 2021 Keda Lu Bo Fang Kuan-Yu Chen ViT 34 2 0 24 Jun 2021
Efficient Self-supervised Vision Transformers for Representation Learning Chunyuan Li Jianwei Yang Pengchuan Zhang Mei Gao Bin Xiao Xiyang Dai Lu Yuan Jianfeng Gao ViT 40 211 0 17 Jun 2021
Pre-Trained Models: Past, Present and Future Xu Han Zhengyan Zhang Ning Ding Yuxian Gu Xiao Liu ... Jie Tang Ji-Rong Wen Jinhui Yuan Wayne Xin Zhao Jun Zhu AIFin MQ AI4MH 63 819 0 14 Jun 2021
Assessing Multilingual Fairness in Pre-trained Multimodal Representations Jialu Wang Yang Liu Xinze Wang EGVM 33 35 0 12 Jun 2021
M6-UFC: Unifying Multi-Modal Controls for Conditional Image Synthesis via Non-Autoregressive Generative Transformers Zhu Zhang Jianxin Ma Chang Zhou Rui Men Zhikang Li Ming Ding Jie Tang Jingren Zhou Hongxia Yang 34 46 0 29 May 2021
Multi-modal Understanding and Generation for Medical Images and Text via Vision-Language Pre-Training Jong Hak Moon HyunGyung Lee W. Shin Young-Hak Kim Edward Choi MedIm 34 153 0 24 May 2021
Pretrained Language Models for Text Generation: A Survey Junyi Li Tianyi Tang Wayne Xin Zhao Ji-Rong Wen LM&MA VLM SyDa 30 185 0 21 May 2021
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding Hu Xu Gargi Ghosh Po-Yao (Bernie) Huang Prahal Arora Masoumeh Aminzadeh Christoph Feichtenhofer Florian Metze Luke Zettlemoyer 31 130 0 20 May 2021
Show Why the Answer is Correct! Towards Explainable AI using Compositional Temporal Attention Nihar Bendre K. Desai Peyman Najafirad CoGe 38 6 0 15 May 2021
Connecting What to Say With Where to Look by Modeling Human Attention Traces Zihang Meng Licheng Yu Ning Zhang Tamara L. Berg Babak Damavandi Vikas Singh Amy Bearman 40 25 0 12 May 2021
Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey Jinjie Ni Tom Young Vlad Pandelea Fuzhao Xue Min Zhang 73 270 0 10 May 2021
Playing Lottery Tickets with Vision and Language Zhe Gan Yen-Chun Chen Linjie Li Tianlong Chen Yu Cheng Shuohang Wang Jingjing Liu Lijuan Wang Zicheng Liu VLM 114 54 0 23 Apr 2021
Understanding Synonymous Referring Expressions via Contrastive Features Yi-Wen Chen Yi-Hsuan Tsai Ming-Hsuan Yang ObjD 27 4 0 20 Apr 2021
Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning Zhicheng Huang Zhaoyang Zeng Yupan Huang Bei Liu Dongmei Fu Jianlong Fu VLM ViT 51 271 0 07 Apr 2021
Compressing Visual-linguistic Model via Knowledge Distillation Zhiyuan Fang Jianfeng Wang Xiaowei Hu Lijuan Wang Yezhou Yang Zicheng Liu VLM 46 97 0 05 Apr 2021
Towards General Purpose Vision Systems Tanmay Gupta Amita Kamath Aniruddha Kembhavi Derek Hoiem 18 50 0 01 Apr 2021
UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training Mingyang Zhou Luowei Zhou Shuohang Wang Yu Cheng Linjie Li Zhou Yu Jingjing Liu MLLM VLM 36 89 0 01 Apr 2021