Unified Vision-Language Pre-Training for Image Captioning and VQA

24 September 2019

Lei Zhang

Papers citing "Unified Vision-Language Pre-Training for Image Captioning and VQA"

50 / 523 papers shown

Title
UFO: A UniFied TransfOrmer for Vision-Language Representation Learning Jianfeng Wang Xiaowei Hu Zhe Gan Zhengyuan Yang Xiyang Dai Zicheng Liu Yumao Lu Lijuan Wang ViT 41 57 0 19 Nov 2021
ClipCap: CLIP Prefix for Image Captioning Ron Mokady Amir Hertz Amit H. Bermano CLIP VLM 28 662 0 18 Nov 2021
Transparent Human Evaluation for Image Captioning Jungo Kasai Keisuke Sakaguchi Lavinia Dunagan Jacob Morrison Ronan Le Bras Yejin Choi Noah A. Smith 38 48 0 17 Nov 2021
A Survey of Visual Transformers Yang Liu Yao Zhang Yixin Wang Feng Hou Jin Yuan Jiang Tian Yang Zhang Zhongchao Shi Jianping Fan Zhiqiang He 3DGS ViT 81 335 0 11 Nov 2021
Multimodal Transformer with Variable-length Memory for Vision-and-Language Navigation Chuang Lin Yi Jiang Jianfei Cai Zhuang Li Gholamreza Haffari Zehuan Yuan 46 32 0 10 Nov 2021
VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts Hangbo Bao Wenhui Wang Li Dong Qiang Liu Owais Khan Mohammed Kriti Aggarwal Subhojit Som Furu Wei VLM MLLM MoE 25 539 0 03 Nov 2021
Unifying Multimodal Transformer for Bi-directional Image and Text Generation Yupan Huang Hongwei Xue Bei Liu Yutong Lu 26 57 0 19 Oct 2021
Unsupervised Finetuning Suichan Li Dongdong Chen Yinpeng Chen Lu Yuan Lei Zhang Qi Chu B. Liu Nenghai Yu 35 8 0 18 Oct 2021
A Good Prompt Is Worth Millions of Parameters: Low-resource Prompt-based Learning for Vision-Language Models Woojeong Jin Yu Cheng Yelong Shen Weizhu Chen Xiang Ren VLM VPVLM MLLM 35 132 0 16 Oct 2021
Self-Annotated Training for Controllable Image Captioning Zhangzi Zhu Tianlei Wang Hong Qu 50 2 0 16 Oct 2021
CLIP4Caption: CLIP for Video Caption Mingkang Tang Zhanyu Wang Zhenhua Liu Fengyun Rao Dian Li Xiu Li CLIP VLM 40 150 0 13 Oct 2021
$Pano-AVQA: Grounded Audio-Visual Question Answering on 360$^\circ$ Videos$ Pano-AVQA: Grounded Audio-Visual Question Answering on 360 $^\circ$ Videos Heeseung Yun Youngjae Yu Wonsuk Yang Kangil Lee Gunhee Kim 56 80 0 11 Oct 2021
Geometry Attention Transformer with Position-aware LSTMs for Image Captioning Chi-Yin Wang Yulin Shen Luping Ji ViT 64 35 0 01 Oct 2021
IntentVizor: Towards Generic Query Guided Interactive Video Summarization Guande Wu Jianzhe Lin Claudio T. Silva 50 24 0 30 Sep 2021
Exploring Teacher-Student Learning Approach for Multi-lingual Speech-to-Intent Classification Bidisha Sharma Maulik C. Madhavi Xuehao Zhou Haizhou Li 33 2 0 28 Sep 2021
Detecting Harmful Memes and Their Targets Shraman Pramanick Dimitar Dimitrov Rituparna Mukherjee Shivam Sharma Md. Shad Akhtar Preslav Nakov Tanmoy Chakraborty 28 111 0 24 Sep 2021
KD-VLP: Improving End-to-End Vision-and-Language Pretraining with Object Knowledge Distillation Yongfei Liu Chenfei Wu Shao-Yen Tseng Vasudev Lal Xuming He Nan Duan CLIP VLM 53 28 0 22 Sep 2021
Dependency Induction Through the Lens of Visual Perception Ruisi Su Shruti Rijhwani Hao Zhu Junxian He Xinyu Wang Yonatan Bisk Graham Neubig 46 3 0 20 Sep 2021
A Survey on Multi-modal Summarization Anubhav Jangra Sourajit Mukherjee Adam Jatowt S. Saha M. Hasanuzzaman 44 60 0 11 Sep 2021
MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets Shraman Pramanick Shivam Sharma Dimitar Dimitrov Md. Shad Akhtar Preslav Nakov Tanmoy Chakraborty 33 120 0 11 Sep 2021
Towards Developing a Multilingual and Code-Mixed Visual Question Answering System by Knowledge Distillation H. Khan D. Gupta Asif Ekbal 32 14 0 10 Sep 2021
Vision Guided Generative Pre-trained Language Models for Multimodal Abstractive Summarization Tiezheng Yu Wenliang Dai Zihan Liu Pascale Fung 37 73 0 06 Sep 2021
Learning to Generate Scene Graph from Natural Language Supervision Yiwu Zhong Jing Shi Jianwei Yang Chenliang Xu Yin Li SSL 56 78 0 06 Sep 2021
LAViTeR: Learning Aligned Visual and Textual Representations Assisted by Image and Caption Generation Mohammad Abuzar Shaikh Zhanghexuan Ji Dana Moukheiber Yan Shen S. Srihari Mingchen Gao VLM 27 1 0 04 Sep 2021
Point-of-Interest Type Prediction using Text and Images Danae Sánchez Villegas Nikolaos Aletras 43 14 0 01 Sep 2021
WebQA: Multihop and Multimodal QA Yingshan Chang M. Narang Hisami Suzuki Guihong Cao Jianfeng Gao Yonatan Bisk LRM 18 79 0 01 Sep 2021
Product-oriented Machine Translation with Cross-modal Cross-lingual Pre-training Yuqing Song Shizhe Chen Qin Jin Wei Luo Jun Xie Fei Huang 54 19 0 25 Aug 2021
TACo: Token-aware Cascade Contrastive Learning for Video-Text Alignment Jianwei Yang Yonatan Bisk Jianfeng Gao 45 138 0 23 Aug 2021
Knowledge Perceived Multi-modal Pretraining in E-commerce Yushan Zhu Huaixiao Tou Wen Zhang Ganqiang Ye Hui Chen Ningyu Zhang Huajun Chen 36 33 0 20 Aug 2021
X-modaler: A Versatile and High-performance Codebase for Cross-modal Analytics Yehao Li Yingwei Pan Jingwen Chen Ting Yao Tao Mei VLM 24 31 0 18 Aug 2021
Who's Waldo? Linking People Across Text and Images Claire Yuqing Cui Apoorv Khandelwal Yoav Artzi Noah Snavely Hadar Averbuch-Elor 41 21 0 16 Aug 2021
BERTHop: An Effective Vision-and-Language Model for Chest X-ray Disease Diagnosis Masoud Monajatipoor Mozhdeh Rouhsedaghat Liunian Harold Li Aichi Chien C.-C. Jay Kuo Fabien Scalzo Kai-Wei Chang LM&MA MedIm 34 30 0 10 Aug 2021
Embodied BERT: A Transformer Model for Embodied, Language-guided Visual Task Completion Alessandro Suglia Qiaozi Gao Jesse Thomason Govind Thattai Gaurav Sukhatme LM&Ro 59 77 0 10 Aug 2021
OVIS: Open-Vocabulary Visual Instance Search via Visual-Semantic Aligned Representation Learning Sheng Liu Kevin Qinghong Lin Lijuan Wang Junsong Yuan Zicheng Liu VLM 16 3 0 08 Aug 2021
Knowledge Distillation from BERT Transformer to Speech Transformer for Intent Classification Yiding Jiang Bidisha Sharma Maulik C. Madhavi Haizhou Li 46 25 0 05 Aug 2021
Exceeding the Limits of Visual-Linguistic Multi-Task Learning Cameron R. Wolfe Keld T. Lundgaard VLM 50 2 0 27 Jul 2021
Multi-stage Pre-training over Simplified Multimodal Pre-training Models Tongtong Liu Fangxiang Feng Xiaojie Wang 21 14 0 22 Jul 2021
From Show to Tell: A Survey on Deep Learning-based Image Captioning Matteo Stefanini Marcella Cornia Lorenzo Baraldi S. Cascianelli G. Fiameni Rita Cucchiara 3DV VLM MLLM 71 259 0 14 Jul 2021
How Much Can CLIP Benefit Vision-and-Language Tasks? Sheng Shen Liunian Harold Li Hao Tan Joey Tianyi Zhou Anna Rohrbach Kai-Wei Chang Z. Yao Kurt Keutzer CLIP VLM MLLM 217 407 0 13 Jul 2021
End-to-end Multi-modal Video Temporal Grounding Yi-Wen Chen Yi-Hsuan Tsai Ming-Hsuan Yang 21 51 0 12 Jul 2021
VidLanKD: Improving Language Understanding via Video-Distilled Knowledge Transfer Zineng Tang Jaemin Cho Hao Tan Joey Tianyi Zhou VLM 38 29 0 06 Jul 2021
OPT: Omni-Perception Pre-Trainer for Cross-Modal Understanding and Generation Jing Liu Xinxin Zhu Fei Liu Longteng Guo Zijia Zhao ... Weining Wang Hanqing Lu Shiyu Zhou Jiajun Zhang Jinqiao Wang 44 37 0 01 Jul 2021
Saying the Unseen: Video Descriptions via Dialog Agents Ye Zhu Yu Wu Yi Yang Yan Yan 33 6 0 26 Jun 2021
Probing Inter-modality: Visual Parsing with Self-Attention for Vision-Language Pre-training Hongwei Xue Yupan Huang Bei Liu Houwen Peng Jianlong Fu Houqiang Li Jiebo Luo 38 89 0 25 Jun 2021
A Transformer-based Cross-modal Fusion Model with Adversarial Training for VQA Challenge 2021 Keda Lu Bo Fang Kuan-Yu Chen ViT 34 2 0 24 Jun 2021
Efficient Self-supervised Vision Transformers for Representation Learning Chunyuan Li Jianwei Yang Pengchuan Zhang Mei Gao Bin Xiao Xiyang Dai Lu Yuan Jianfeng Gao ViT 47 212 0 17 Jun 2021
Semi-Autoregressive Transformer for Image Captioning Yuanen Zhou Yong Zhang Zhenzhen Hu Meng Wang VLM 39 25 0 17 Jun 2021
Pre-Trained Models: Past, Present and Future Xu Han Zhengyan Zhang Ning Ding Yuxian Gu Xiao Liu ... Jie Tang Ji-Rong Wen Jinhui Yuan Wayne Xin Zhao Jun Zhu AIFin MQ AI4MH 74 825 0 14 Jun 2021
Assessing Multilingual Fairness in Pre-trained Multimodal Representations Jialu Wang Yang Liu Xinze Wang EGVM 33 36 0 12 Jun 2021
Chasing Sparsity in Vision Transformers: An End-to-End Exploration Tianlong Chen Yu Cheng Zhe Gan Lu Yuan Lei Zhang Zhangyang Wang ViT 24 216 0 08 Jun 2021