Unified Vision-Language Pre-Training for Image Captioning and VQA

24 September 2019

Lei Zhang

Papers citing "Unified Vision-Language Pre-Training for Image Captioning and VQA"

50 / 250 papers shown

Title
CLIP4IDC: CLIP for Image Difference Captioning Zixin Guo Tong Wang Jorma T. Laaksonen VLM 29 27 0 01 Jun 2022
GIT: A Generative Image-to-text Transformer for Vision and Language Jianfeng Wang Zhengyuan Yang Xiaowei Hu Linjie Li Kevin Qinghong Lin Zhe Gan Zicheng Liu Ce Liu Lijuan Wang VLM 64 531 0 27 May 2022
DisinfoMeme: A Multimodal Dataset for Detecting Meme Intentionally Spreading Out Disinformation Jingnong Qu Liunian Harold Li Jieyu Zhao Sunipa Dev Kai-Wei Chang 26 12 0 25 May 2022
On Advances in Text Generation from Images Beyond Captioning: A Case Study in Self-Rationalization Shruti Palaskar Akshita Bhagia Yonatan Bisk Florian Metze A. Black Ana Marasović 36 4 0 24 May 2022
Novel Multicolumn Kernel Extreme Learning Machine for Food Detection via Optimal Features from CNN G. Tahir Tahir Chu C. K. Loo 26 2 0 15 May 2022
Automated Audio Captioning: An Overview of Recent Progress and New Challenges Xinhao Mei Xubo Liu Mark D. Plumbley Wenwu Wang 34 38 0 12 May 2022
Learning to Answer Visual Questions from Web Videos Antoine Yang Antoine Miech Josef Sivic Ivan Laptev Cordelia Schmid ViT 48 34 0 10 May 2022
Language Models Can See: Plugging Visual Controls in Text Generation Yixuan Su Tian Lan Yahui Liu Fangyu Liu Dani Yogatama Yan Wang Lingpeng Kong Nigel Collier VLM MLLM 62 97 0 05 May 2022
All You May Need for VQA are Image Captions Soravit Changpinyo Doron Kukliansky Idan Szpektor Xi Chen Nan Ding Radu Soricut 32 70 0 04 May 2022
Answer-Me: Multi-Task Open-Vocabulary Visual Question Answering A. Piergiovanni Wei Li Weicheng Kuo M. Saffar Fred Bertsch A. Angelova 17 16 0 02 May 2022
Molecular Identification from AFM images using the IUPAC Nomenclature and Attribute Multimodal Recurrent Neural Networks Jaime Carracedo-Cosme C. Romero-Muñiz Pablo Pou R. Pérez 19 10 0 01 May 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 53 3,381 0 29 Apr 2022
Training and challenging models for text-guided fashion image retrieval Eric Dodds Jack Culpepper Gaurav Srivastava 26 8 0 23 Apr 2022
Multimodal Adaptive Distillation for Leveraging Unimodal Encoders for Vision-Language Tasks Zhecan Wang Noel Codella Yen-Chun Chen Luowei Zhou Xiyang Dai ... Jianwei Yang Haoxuan You Kai-Wei Chang Shih-Fu Chang Lu Yuan VLM OffRL 33 22 0 22 Apr 2022
Guiding Attention using Partial-Order Relationships for Image Captioning Murad Popattia Muhammad Rafi Rizwan Qureshi Shah Nawaz 21 5 0 15 Apr 2022
Image Captioning In the Transformer Age Yangliu Xu Li Li Haiyang Xu Songfang Huang Fei Huang Jianfei Cai ViT 27 5 0 15 Apr 2022
Vision-and-Language Pretrained Models: A Survey Siqu Long Feiqi Cao S. Han Haiqing Yang VLM 38 63 0 15 Apr 2022
TubeDETR: Spatio-Temporal Video Grounding with Transformers Antoine Yang Antoine Miech Josef Sivic Ivan Laptev Cordelia Schmid ViT 35 94 0 30 Mar 2022
Reshaping Robot Trajectories Using Natural Language Commands: A Study of Multi-Modal Data Alignment Using Transformers A. Bucker Luis F. C. Figueredo Sami Haddadin Ashish Kapoor Shuang Ma Rogerio Bonatti LM&Ro 47 49 0 25 Mar 2022
STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translation Qingkai Fang Rong Ye Lei Li Yang Feng Mingxuan Wang 51 95 0 20 Mar 2022
DU-VLG: Unifying Vision-and-Language Generation via Dual Sequence-to-Sequence Pre-training Luyang Huang Guocheng Niu Jiachen Liu Xinyan Xiao Hua Wu VLM CoGe 19 7 0 17 Mar 2022
The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy Tianlong Chen Zhenyu Zhang Yu Cheng Ahmed Hassan Awadallah Zhangyang Wang ViT 46 37 0 12 Mar 2022
StyleBabel: Artistic Style Tagging and Captioning Dan Ruta Andrew Gilbert Pranav Aggarwal Naveen Marri Ajinkya Kale ... Hailin Jin Baldo Faieta Alex Filipkowski Zhe Lin John Collomosse 32 12 0 10 Mar 2022
NLX-GPT: A Model for Natural Language Explanations in Vision and Vision-Language Tasks Fawaz Sammani Tanmoy Mukherjee Nikos Deligiannis MILM ELM LRM 32 67 0 09 Mar 2022
Visual-Language Navigation Pretraining via Prompt-based Environmental Self-exploration Xiwen Liang Fengda Zhu Lingling Li Hang Xu Xiaodan Liang LM&Ro VLM 36 29 0 08 Mar 2022
Unsupervised Vision-and-Language Pre-training via Retrieval-based Multi-Granular Alignment Mingyang Zhou Licheng Yu Amanpreet Singh Mengjiao MJ Wang Zhou Yu Ning Zhang VLM 35 31 0 01 Mar 2022
GroupViT: Semantic Segmentation Emerges from Text Supervision Jiarui Xu Shalini De Mello Sifei Liu Wonmin Byeon Thomas Breuel Jan Kautz Xinyu Wang ViT VLM 200 506 0 22 Feb 2022
VU-BERT: A Unified framework for Visual Dialog Tong Ye Shijing Si Jianzong Wang Rui Wang Ning Cheng Jing Xiao MLLM 38 5 0 22 Feb 2022
CaMEL: Mean Teacher Learning for Image Captioning Manuele Barraco Matteo Stefanini Marcella Cornia S. Cascianelli Lorenzo Baraldi Rita Cucchiara ViT VLM 43 27 0 21 Feb 2022
A Survey of Vision-Language Pre-Trained Models Yifan Du Zikang Liu Junyi Li Wayne Xin Zhao VLM 47 180 0 18 Feb 2022
VLP: A Survey on Vision-Language Pre-training Feilong Chen Duzhen Zhang Minglun Han Xiuyi Chen Jing Shi Shuang Xu Bo Xu VLM 82 215 0 18 Feb 2022
OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework Peng Wang An Yang Rui Men Junyang Lin Shuai Bai Zhikang Li Jianxin Ma Chang Zhou Jingren Zhou Hongxia Yang MLLM ObjD 74 852 0 07 Feb 2022
Catch Me if You Can: A Novel Task for Detection of Covert Geo-Locations (CGL) Binoy Saha Sukhendu Das 22 1 0 05 Feb 2022
Deep Learning Approaches on Image Captioning: A Review Taraneh Ghandi H. Pourreza H. Mahyar VLM 30 89 0 31 Jan 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong Guosheng Lin MLLM BDL VLM CLIP 397 4,204 0 28 Jan 2022
Do Smart Glasses Dream of Sentimental Visions? Deep Emotionship Analysis for Eyewear Devices Yingying Zhao Yuhu Chang Yutian Lu Yujiang Wang Mingzhi Dong ... Robert P. Dick Fan Yang Tun Lu Ning Gu L. Shang 41 9 0 24 Jan 2022
Cross-modal Contrastive Distillation for Instructional Activity Anticipation Zhengyuan Yang Jingen Liu Jing-ling Huang Xiaodong He Tao Mei Chenliang Xu Jiebo Luo 31 6 0 18 Jan 2022
CLIP-TD: CLIP Targeted Distillation for Vision-Language Tasks Zhecan Wang Noel Codella Yen-Chun Chen Luowei Zhou Jianwei Yang Xiyang Dai Bin Xiao Haoxuan You Shih-Fu Chang Lu Yuan CLIP VLM 24 39 0 15 Jan 2022
CLIP-Event: Connecting Text and Images with Event Structures Manling Li Ruochen Xu Shuohang Wang Luowei Zhou Xudong Lin Chenguang Zhu Michael Zeng Heng Ji Shih-Fu Chang VLM CLIP 27 124 0 13 Jan 2022
Uni-EDEN: Universal Encoder-Decoder Network by Multi-Granular Vision-Language Pre-training Yehao Li Jiahao Fan Yingwei Pan Ting Yao Weiyao Lin Tao Mei MLLM ObjD 33 19 0 11 Jan 2022
On the Efficacy of Co-Attention Transformer Layers in Visual Question Answering Ankur Sikarwar Gabriel Kreiman ViT 24 1 0 11 Jan 2022
Self-Training Vision Language BERTs with a Unified Conditional Model Xiaofeng Yang Fengmao Lv Fayao Liu Guosheng Lin SSL VLM 54 14 0 06 Jan 2022
Compact Bidirectional Transformer for Image Captioning Yuanen Zhou Zhenzhen Hu Daqing Liu Huixia Ben Meng Wang VLM 25 16 0 06 Jan 2022
ERNIE-ViLG: Unified Generative Pre-training for Bidirectional Vision-Language Generation Han Zhang Weichong Yin Yewei Fang Lanxin Li Boqiang Duan Zhihua Wu Yu Sun Hao Tian Hua Wu Haifeng Wang 32 59 0 31 Dec 2021
LaTr: Layout-Aware Transformer for Scene-Text VQA Ali Furkan Biten Ron Litman Yusheng Xie Srikar Appalaraju R. Manmatha ViT 41 100 0 23 Dec 2021
RegionCLIP: Region-based Language-Image Pretraining Yiwu Zhong Jianwei Yang Pengchuan Zhang Chunyuan Li Noel Codella ... Luowei Zhou Xiyang Dai Lu Yuan Yin Li Jianfeng Gao VLM CLIP 40 558 0 16 Dec 2021
CoCo-BERT: Improving Video-Language Pre-training with Contrastive Cross-modal Matching and Denoising Jianjie Luo Yehao Li Yingwei Pan Ting Yao Hongyang Chao Tao Mei VLM 18 42 0 14 Dec 2021
VUT: Versatile UI Transformer for Multi-Modal Multi-Task User Interface Modeling Yang Li Gang Li Xin Zhou Mostafa Dehghani A. Gritsenko MLLM 45 35 0 10 Dec 2021
Unified Multimodal Pre-training and Prompt-based Tuning for Vision-Language Understanding and Generation Tianyi Liu Zuxuan Wu Wenhan Xiong Jingjing Chen Yu-Gang Jiang VLM MLLM 32 10 0 10 Dec 2021
Injecting Semantic Concepts into End-to-End Image Captioning Zhiyuan Fang Jianfeng Wang Xiaowei Hu Lin Liang Zhe Gan Lijuan Wang Yezhou Yang Zicheng Liu ViT VLM 34 86 0 09 Dec 2021