Unified Vision-Language Pre-Training for Image Captioning and VQA

24 September 2019

Lei Zhang

Papers citing "Unified Vision-Language Pre-Training for Image Captioning and VQA"

50 / 523 papers shown

Title
DU-VLG: Unifying Vision-and-Language Generation via Dual Sequence-to-Sequence Pre-training Luyang Huang Guocheng Niu Jiachen Liu Xinyan Xiao Hua Wu VLM CoGe 24 8 0 17 Mar 2022
The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy Tianlong Chen Zhenyu Zhang Yu Cheng Ahmed Hassan Awadallah Zhangyang Wang ViT 46 37 0 12 Mar 2022
StyleBabel: Artistic Style Tagging and Captioning Dan Ruta Andrew Gilbert Pranav Aggarwal Naveen Marri Ajinkya Kale ... Hailin Jin Baldo Faieta Alex Filipkowski Zhe Lin John Collomosse 32 12 0 10 Mar 2022
NLX-GPT: A Model for Natural Language Explanations in Vision and Vision-Language Tasks Fawaz Sammani Tanmoy Mukherjee Nikos Deligiannis MILM ELM LRM 53 67 0 09 Mar 2022
Visual-Language Navigation Pretraining via Prompt-based Environmental Self-exploration Xiwen Liang Fengda Zhu Lingling Li Hang Xu Xiaodan Liang LM&Ro VLM 38 29 0 08 Mar 2022
Vision-Language Intelligence: Tasks, Representation Learning, and Large Models Feng Li Hao Zhang Yi-Fan Zhang Shixuan Liu Jian Guo L. Ni Pengchuan Zhang Lei Zhang AI4TS VLM 34 37 0 03 Mar 2022
Unsupervised Vision-and-Language Pre-training via Retrieval-based Multi-Granular Alignment Mingyang Zhou Licheng Yu Amanpreet Singh Mengjiao MJ Wang Zhou Yu Ning Zhang VLM 40 31 0 01 Mar 2022
GroupViT: Semantic Segmentation Emerges from Text Supervision Jiarui Xu Shalini De Mello Sifei Liu Wonmin Byeon Thomas Breuel Jan Kautz Xinyu Wang ViT VLM 219 511 0 22 Feb 2022
VU-BERT: A Unified framework for Visual Dialog Tong Ye Shijing Si Jianzong Wang Rui Wang Ning Cheng Jing Xiao MLLM 70 5 0 22 Feb 2022
CaMEL: Mean Teacher Learning for Image Captioning Manuele Barraco Matteo Stefanini Marcella Cornia S. Cascianelli Lorenzo Baraldi Rita Cucchiara ViT VLM 48 27 0 21 Feb 2022
A Survey of Vision-Language Pre-Trained Models Yifan Du Zikang Liu Junyi Li Wayne Xin Zhao VLM 59 181 0 18 Feb 2022
VLP: A Survey on Vision-Language Pre-training Feilong Chen Duzhen Zhang Minglun Han Xiuyi Chen Jing Shi Shuang Xu Bo Xu VLM 82 216 0 18 Feb 2022
CommerceMM: Large-Scale Commerce MultiModal Representation Learning with Omni Retrieval Licheng Yu Jun Chen Animesh Sinha Mengjiao MJ Wang Hugo Chen Tamara L. Berg Ning Zhang VLM 44 39 0 15 Feb 2022
I-Tuning: Tuning Frozen Language Models with Image for Lightweight Image Captioning Ziyang Luo Zhipeng Hu Yadong Xi Rongsheng Zhang Jing Ma VLM 28 13 0 14 Feb 2022
Image Difference Captioning with Pre-training and Contrastive Learning Linli Yao Weiying Wang Qin Jin SSL VLM 38 41 0 09 Feb 2022
OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework Peng Wang An Yang Rui Men Junyang Lin Shuai Bai Zhikang Li Jianxin Ma Chang Zhou Jingren Zhou Hongxia Yang MLLM ObjD 74 857 0 07 Feb 2022
Catch Me if You Can: A Novel Task for Detection of Covert Geo-Locations (CGL) Binoy Saha Sukhendu Das 27 1 0 05 Feb 2022
Deep Learning Approaches on Image Captioning: A Review Taraneh Ghandi H. Pourreza H. Mahyar VLM 30 90 0 31 Jan 2022
A Frustratingly Simple Approach for End-to-End Image Captioning Ziyang Luo Yadong Xi Rongsheng Zhang Jing Ma VLM MLLM 30 16 0 30 Jan 2022
MVPTR: Multi-Level Semantic Alignment for Vision-Language Pre-Training via Multi-Stage Learning Zejun Li Zhihao Fan Huaixiao Tou Jingjing Chen Zhongyu Wei Xuanjing Huang 30 16 0 29 Jan 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong Guosheng Lin MLLM BDL VLM CLIP 399 4,250 0 28 Jan 2022
Do Smart Glasses Dream of Sentimental Visions? Deep Emotionship Analysis for Eyewear Devices Yingying Zhao Yuhu Chang Yutian Lu Yujiang Wang Mingzhi Dong ... Robert P. Dick Fan Yang Tun Lu Ning Gu L. Shang 46 9 0 24 Jan 2022
A Pre-trained Audio-Visual Transformer for Emotion Recognition Minh Tran M. Soleymani 61 25 0 23 Jan 2022
Cross-modal Contrastive Distillation for Instructional Activity Anticipation Zhengyuan Yang Jingen Liu Jing-ling Huang Xiaodong He Tao Mei Chenliang Xu Jiebo Luo 36 6 0 18 Jan 2022
CLIP-TD: CLIP Targeted Distillation for Vision-Language Tasks Zhecan Wang Noel Codella Yen-Chun Chen Luowei Zhou Jianwei Yang Xiyang Dai Bin Xiao Haoxuan You Shih-Fu Chang Lu Yuan CLIP VLM 27 39 0 15 Jan 2022
CLIP-Event: Connecting Text and Images with Event Structures Manling Li Ruochen Xu Shuohang Wang Luowei Zhou Xudong Lin Chenguang Zhu Michael Zeng Heng Ji Shih-Fu Chang VLM CLIP 32 124 0 13 Jan 2022
Uni-EDEN: Universal Encoder-Decoder Network by Multi-Granular Vision-Language Pre-training Yehao Li Jiahao Fan Yingwei Pan Ting Yao Weiyao Lin Tao Mei MLLM ObjD 38 19 0 11 Jan 2022
On the Efficacy of Co-Attention Transformer Layers in Visual Question Answering Ankur Sikarwar Gabriel Kreiman ViT 24 1 0 11 Jan 2022
Self-Training Vision Language BERTs with a Unified Conditional Model Xiaofeng Yang Fengmao Lv Fayao Liu Guosheng Lin SSL VLM 54 14 0 06 Jan 2022
Compact Bidirectional Transformer for Image Captioning Yuanen Zhou Zhenzhen Hu Daqing Liu Huixia Ben Meng Wang VLM 35 16 0 06 Jan 2022
ERNIE-ViLG: Unified Generative Pre-training for Bidirectional Vision-Language Generation Han Zhang Weichong Yin Yewei Fang Lanxin Li Boqiang Duan Zhihua Wu Yu Sun Hao Tian Hua Wu Haifeng Wang 34 59 0 31 Dec 2021
LaTr: Layout-Aware Transformer for Scene-Text VQA Ali Furkan Biten Ron Litman Yusheng Xie Srikar Appalaraju R. Manmatha ViT 52 100 0 23 Dec 2021
RegionCLIP: Region-based Language-Image Pretraining Yiwu Zhong Jianwei Yang Pengchuan Zhang Chunyuan Li Noel Codella ... Luowei Zhou Xiyang Dai Lu Yuan Yin Li Jianfeng Gao VLM CLIP 45 565 0 16 Dec 2021
Distilled Dual-Encoder Model for Vision-Language Understanding Zekun Wang Wenhui Wang Haichao Zhu Ming Liu Bing Qin Furu Wei VLM FedML 34 31 0 16 Dec 2021
CoCo-BERT: Improving Video-Language Pre-training with Contrastive Cross-modal Matching and Denoising Jianjie Luo Yehao Li Yingwei Pan Ting Yao Hongyang Chao Tao Mei VLM 30 42 0 14 Dec 2021
VUT: Versatile UI Transformer for Multi-Modal Multi-Task User Interface Modeling Yang Li Gang Li Xin Zhou Mostafa Dehghani A. Gritsenko MLLM 47 35 0 10 Dec 2021
Unified Multimodal Pre-training and Prompt-based Tuning for Vision-Language Understanding and Generation Tianyi Liu Zuxuan Wu Wenhan Xiong Jingjing Chen Yu-Gang Jiang VLM MLLM 32 10 0 10 Dec 2021
Injecting Semantic Concepts into End-to-End Image Captioning Zhiyuan Fang Jianfeng Wang Xiaowei Hu Lin Liang Zhe Gan Lijuan Wang Yezhou Yang Zicheng Liu ViT VLM 40 86 0 09 Dec 2021
FLAVA: A Foundational Language And Vision Alignment Model Amanpreet Singh Ronghang Hu Vedanuj Goswami Guillaume Couairon Wojciech Galuba Marcus Rohrbach Douwe Kiela CLIP VLM 42 695 0 08 Dec 2021
MLP Architectures for Vision-and-Language Modeling: An Empirical Study Yi-Liang Nie Linjie Li Zhe Gan Shuohang Wang Chenguang Zhu Michael Zeng Zicheng Liu Joey Tianyi Zhou Lijuan Wang 29 6 0 08 Dec 2021
Bidimensional Leaderboards: Generate and Evaluate Language Hand in Hand Jungo Kasai Keisuke Sakaguchi Ronan Le Bras Lavinia Dunagan Jacob Morrison Alexander R. Fabbri Yejin Choi Noah A. Smith 57 39 0 08 Dec 2021
Grounded Language-Image Pre-training Liunian Harold Li Pengchuan Zhang Haotian Zhang Jianwei Yang Chunyuan Li ... Lu Yuan Lei Zhang Lei Li Kai-Wei Chang Jianfeng Gao ObjD VLM 36 1,036 0 07 Dec 2021
CMA-CLIP: Cross-Modality Attention CLIP for Image-Text Classification Huidong Liu Shaoyuan Xu Jinmiao Fu Yang Liu Ning Xie Chien Wang Bryan Wang Yi Sun CLIP VLM 32 27 0 07 Dec 2021
Joint Learning of Localized Representations from Medical Images and Reports Philipp Muller Georgios Kaissis Cong Zou Daniel Munich 140 81 0 06 Dec 2021
Uni-Perceiver: Pre-training Unified Architecture for Generic Perception for Zero-shot and Few-shot Tasks Xizhou Zhu Jinguo Zhu Hao Li Xiaoshi Wu Xiaogang Wang Hongsheng Li Xiaohua Wang Jifeng Dai 56 129 0 02 Dec 2021
Object-Centric Unsupervised Image Captioning Zihang Meng David Yang Xuefei Cao Ashish Shah Ser-Nam Lim OCL VLM 32 12 0 02 Dec 2021
Generating More Pertinent Captions by Leveraging Semantics and Style on Multi-Source Datasets Marcella Cornia Lorenzo Baraldi G. Fiameni Rita Cucchiara 39 12 0 24 Nov 2021
VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling Tsu-Jui Fu Linjie Li Zhe Gan Kevin Qinghong Lin Wenjie Wang Lijuan Wang Zicheng Liu VLM 60 219 0 24 Nov 2021
Scaling Up Vision-Language Pre-training for Image Captioning Xiaowei Hu Zhe Gan Jianfeng Wang Zhengyuan Yang Zicheng Liu Yumao Lu Lijuan Wang MLLM VLM 53 247 0 24 Nov 2021
RedCaps: web-curated image-text data created by the people, for the people Karan Desai Gaurav Kaul Zubin Aysola Justin Johnson 38 163 0 22 Nov 2021