TAP: Text-Aware Pre-training for Text-VQA and Text-Caption

8 December 2020

Lei Zhang

Papers citing "TAP: Text-Aware Pre-training for Text-VQA and Text-Caption"

44 / 44 papers shown

Title
Granite Vision: a lightweight, open-source multimodal model for enterprise Intelligence Granite Vision Team Leonid Karlinsky Assaf Arbelle Abraham Daniels A. Nassar ... Sriram Raghavan Tanveer Syeda-Mahmood Peter W. J. Staar Tal Drory Rogerio Feris VLM AI4TS 159 2 0 14 Feb 2025
List Items One by One: A New Data Source and Learning Paradigm for Multimodal LLMs An Yan Zhengyuan Yang Junda Wu Wanrong Zhu Jianwei Yang ... Kevin Qinghong Lin Jianfeng Wang Julian McAuley Jianfeng Gao Lijuan Wang LRM 62 12 0 25 Apr 2024
Prophet: Prompting Large Language Models with Complementary Answer Heuristics for Knowledge-based Visual Question Answering Zhou Yu Xuecheng Ouyang Zhenwei Shao Mei Wang Jun Yu MLLM 149 11 0 03 Mar 2023
Confidence-aware Non-repetitive Multimodal Transformers for TextCaps Zhaokai Wang Renda Bao Qi Wu Si Liu 95 26 0 07 Dec 2020
Finding the Evidence: Localization-aware Answer Prediction for Text Visual Question Answering Wei Han Hantao Huang Tao Han 36 51 0 06 Oct 2020
VIVO: Visual Vocabulary Pre-Training for Novel Object Captioning Xiaowei Hu Xi Yin Kevin Qinghong Lin Lijuan Wang Lefei Zhang Jianfeng Gao Zicheng Liu VLM 62 56 0 28 Sep 2020
Spatially Aware Multimodal Transformers for TextVQA Yash Kant Dhruv Batra Peter Anderson Alex Schwing Devi Parikh Jiasen Lu Harsh Agrawal 68 85 0 23 Jul 2020
Structured Multimodal Attentions for TextVQA Chenyu Gao Qi Zhu Peng Wang Hui Li Yuliang Liu Anton Van Den Hengel Qi Wu 67 59 0 01 Jun 2020
Behind the Scene: Revealing the Secrets of Pre-trained Vision-and-Language Models Jize Cao Zhe Gan Yu Cheng Licheng Yu Yen-Chun Chen Jingjing Liu VLM 61 130 0 15 May 2020
Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks Xiujun Li Xi Yin Chunyuan Li Pengchuan Zhang Xiaowei Hu ... Houdong Hu Li Dong Furu Wei Yejin Choi Jianfeng Gao VLM 90 1,934 0 13 Apr 2020
Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers Zhicheng Huang Zhaoyang Zeng Bei Liu Dongmei Fu Jianlong Fu ViT 124 438 0 02 Apr 2020
Multi-Modal Graph Neural Network for Joint Reasoning on Vision and Scene Text Difei Gao Ke Li Ruiping Wang Shiguang Shan Xilin Chen 57 112 0 31 Mar 2020
TextCaps: a Dataset for Image Captioning with Reading Comprehension Oleksii Sidorov Ronghang Hu Marcus Rohrbach Amanpreet Singh 58 411 0 24 Mar 2020
On the General Value of Evidence, and Bilingual Scene-Text Visual Question Answering Xinyu Wang Yuliang Liu Chunhua Shen Chun Chet Ng Canjie Luo Lianwen Jin C. Chan Anton Van Den Hengel Liangwei Wang 81 97 0 24 Feb 2020
12-in-1: Multi-Task Vision and Language Representation Learning Jiasen Lu Vedanuj Goswami Marcus Rohrbach Devi Parikh Stefan Lee VLM ObjD 73 478 0 05 Dec 2019
Iterative Answer Prediction with Pointer-Augmented Multimodal Transformers for TextVQA Ronghang Hu Amanpreet Singh Trevor Darrell Marcus Rohrbach 67 197 0 14 Nov 2019
Rosetta: Large scale system for text detection and recognition in images Fedor Borisyuk Albert Gordo V. Sivakumar 56 298 0 11 Oct 2019
UNITER: UNiversal Image-TExt Representation Learning Yen-Chun Chen Linjie Li Licheng Yu Ahmed El Kholy Faisal Ahmed Zhe Gan Yu Cheng Jingjing Liu VLM OT 99 447 0 25 Sep 2019
Unified Vision-Language Pre-Training for Image Captioning and VQA Luowei Zhou Hamid Palangi Lei Zhang Houdong Hu Jason J. Corso Jianfeng Gao MLLM VLM 337 937 0 24 Sep 2019
VL-BERT: Pre-training of Generic Visual-Linguistic Representations Weijie Su Xizhou Zhu Yue Cao Bin Li Lewei Lu Furu Wei Jifeng Dai VLM MLLM SSL 145 1,661 0 22 Aug 2019
LXMERT: Learning Cross-Modality Encoder Representations from Transformers Hao Hao Tan Joey Tianyi Zhou VLM MLLM 227 2,474 0 20 Aug 2019
Attention on Attention for Image Captioning Lun Huang Wenmin Wang Jie Chen Xiao-Yong Wei 58 829 0 19 Aug 2019
A Fast and Accurate One-Stage Approach to Visual Grounding Zhengyuan Yang Boqing Gong Liwei Wang Wenbing Huang Dong Yu Jiebo Luo ObjD 46 362 0 18 Aug 2019
Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-training Gen Li Nan Duan Yuejian Fang Ming Gong Daxin Jiang Ming Zhou SSL VLM MLLM 200 900 0 16 Aug 2019
Fusion of Detected Objects in Text for Visual Question Answering Chris Alberti Jeffrey Ling Michael Collins David Reitter 56 173 0 14 Aug 2019
VisualBERT: A Simple and Performant Baseline for Vision and Language Liunian Harold Li Mark Yatskar Da Yin Cho-Jui Hsieh Kai-Wei Chang VLM 130 1,948 0 09 Aug 2019
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks Jiasen Lu Dhruv Batra Devi Parikh Stefan Lee SSL VLM 217 3,667 0 06 Aug 2019
Scene Text Visual Question Answering Ali Furkan Biten Rubèn Pérez Tito Andrés Mafla Lluís Gómez Marçal Rusiñol Ernest Valveny C. V. Jawahar Dimosthenis Karatzas 82 355 0 31 May 2019
Towards VQA Models That Can Read Amanpreet Singh Vivek Natarajan Meet Shah Yu Jiang Xinlei Chen Dhruv Batra Devi Parikh Marcus Rohrbach EgoV 69 1,210 0 18 Apr 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.6K 94,511 0 11 Oct 2018
Exploring Visual Relationship for Image Captioning Ting Yao Yingwei Pan Yehao Li Tao Mei 74 831 0 19 Sep 2018
Pythia v0.1: the Winning Entry to the VQA Challenge 2018 Yu Jiang Vivek Natarajan Xinlei Chen Marcus Rohrbach Dhruv Batra Devi Parikh VLM 54 203 0 26 Jul 2018
VizWiz Grand Challenge: Answering Visual Questions from Blind People Danna Gurari Qing Li Abigale Stangl Anhong Guo Chi Lin Kristen Grauman Jiebo Luo Jeffrey P. Bigham CoGe 82 844 0 22 Feb 2018
Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments Peter Anderson Qi Wu Damien Teney Jake Bruce Mark Johnson Niko Sünderhauf Ian Reid Stephen Gould Anton Van Den Hengel LM&Ro 95 1,304 0 20 Nov 2017
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering Peter Anderson Xiaodong He Chris Buehler Damien Teney Mark Johnson Stephen Gould Lei Zhang AIMat 113 4,208 0 25 Jul 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 646 130,942 0 12 Jun 2017
SPICE: Semantic Propositional Image Caption Evaluation Peter Anderson Basura Fernando Mark Johnson Stephen Gould EGVM 90 1,909 0 29 Jul 2016
Enriching Word Vectors with Subword Information Piotr Bojanowski Edouard Grave Armand Joulin Tomas Mikolov NAI SSL VLM 220 9,957 0 15 Jul 2016
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations Ranjay Krishna Yuke Zhu Oliver Groth Justin Johnson Kenji Hata ... Yannis Kalantidis Li Li David A. Shamma Michael S. Bernstein Fei-Fei Li 196 5,726 0 23 Feb 2016
COCO-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Images Andreas Veit Tomas Matera Lukás Neumann Jirí Matas Serge J. Belongie 236 527 0 26 Jan 2016
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks Shaoqing Ren Kaiming He Ross B. Girshick Jian Sun AIMat ObjD 478 62,122 0 04 Jun 2015
Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models Bryan A. Plummer Liwei Wang Christopher M. Cervantes Juan C. Caicedo Julia Hockenmaier Svetlana Lazebnik 187 2,047 0 19 May 2015
VQA: Visual Question Answering Aishwarya Agrawal Jiasen Lu Stanislaw Antol Margaret Mitchell C. L. Zitnick Dhruv Batra Devi Parikh CoGe 182 5,452 0 03 May 2015
CIDEr: Consensus-based Image Description Evaluation Ramakrishna Vedantam C. L. Zitnick Devi Parikh 264 4,471 0 20 Nov 2014