LXMERT: Learning Cross-Modality Encoder Representations from Transformers

20 August 2019

Papers citing "LXMERT: Learning Cross-Modality Encoder Representations from Transformers"

50 / 1,512 papers shown

Title
Rethinking Attention-Model Explainability through Faithfulness Violation Test Yue Liu Haoliang Li Yangyang Guo Chen Kong Jing Li Shiqi Wang FAtt 121 43 0 28 Jan 2022
IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages Emanuele Bugliarello Fangyu Liu Jonas Pfeiffer Siva Reddy Desmond Elliott Edoardo Ponti Ivan Vulić MLLM VLM ELM 55 62 0 27 Jan 2022
MGA-VQA: Multi-Granularity Alignment for Visual Question Answering Peixi Xiong Yilin Shen Hongxia Jin 24 5 0 25 Jan 2022
SA-VQA: Structured Alignment of Visual and Semantic Representations for Visual Question Answering Peixi Xiong Quanzeng You Pei Yu Zicheng Liu Ying Wu 24 5 0 25 Jan 2022
Do Smart Glasses Dream of Sentimental Visions? Deep Emotionship Analysis for Eyewear Devices Yingying Zhao Yuhu Chang Yutian Lu Yujiang Wang Mingzhi Dong ... Robert P. Dick Fan Yang Tun Lu Ning Gu L. Shang 41 9 0 24 Jan 2022
Question Generation for Evaluating Cross-Dataset Shifts in Multi-modal Grounding Arjun Reddy Akula OOD 23 3 0 24 Jan 2022
Supervised Visual Attention for Simultaneous Multimodal Machine Translation Veneta Haralampieva Ozan Caglayan Lucia Specia LRM 50 4 0 23 Jan 2022
OntoProtein: Protein Pretraining With Gene Ontology Embedding Ningyu Zhang Zhen Bi Xiaozhuan Liang Shuyang Cheng Haosen Hong Shumin Deng J. Lian Qiang Zhang Huajun Chen 99 90 0 23 Jan 2022
A Pre-trained Audio-Visual Transformer for Emotion Recognition Minh Tran M. Soleymani 61 25 0 23 Jan 2022
Omnivore: A Single Model for Many Visual Modalities Rohit Girdhar Mannat Singh Nikhil Ravi Laurens van der Maaten Armand Joulin Ishan Misra 229 226 0 20 Jan 2022
CLIP-TD: CLIP Targeted Distillation for Vision-Language Tasks Zhecan Wang Noel Codella Yen-Chun Chen Luowei Zhou Jianwei Yang Xiyang Dai Bin Xiao Haoxuan You Shih-Fu Chang Lu Yuan CLIP VLM 24 39 0 15 Jan 2022
A Thousand Words Are Worth More Than a Picture: Natural Language-Centric Outside-Knowledge Visual Question Answering Feng Gao Q. Ping Govind Thattai Aishwarya N. Reganti Yingting Wu Premkumar Natarajan 18 17 0 14 Jan 2022
CLIP-Event: Connecting Text and Images with Event Structures Manling Li Ruochen Xu Shuohang Wang Luowei Zhou Xudong Lin Chenguang Zhu Michael Zeng Heng Ji Shih-Fu Chang VLM CLIP 27 124 0 13 Jan 2022
Uni-EDEN: Universal Encoder-Decoder Network by Multi-Granular Vision-Language Pre-training Yehao Li Jiahao Fan Yingwei Pan Ting Yao Weiyao Lin Tao Mei MLLM ObjD 33 19 0 11 Jan 2022
On the Efficacy of Co-Attention Transformer Layers in Visual Question Answering Ankur Sikarwar Gabriel Kreiman ViT 24 1 0 11 Jan 2022
Music2Video: Automatic Generation of Music Video with fusion of audio and text Yoonjeon Kim Joel Jang Sumin Shin DiffM VGen 38 7 0 11 Jan 2022
A Comprehensive Empirical Study of Vision-Language Pre-trained Model for Supervised Cross-Modal Retrieval Zhixiong Zeng Wenji Mao VLM 15 17 0 08 Jan 2022
MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound Rowan Zellers Jiasen Lu Ximing Lu Youngjae Yu Yanpeng Zhao Mohammadreza Salehi Aditya Kusupati Jack Hessel Ali Farhadi Yejin Choi 48 207 0 07 Jan 2022
Self-Training Vision Language BERTs with a Unified Conditional Model Xiaofeng Yang Fengmao Lv Fayao Liu Guosheng Lin SSL VLM 54 13 0 06 Jan 2022
Semantically Grounded Visual Embeddings for Zero-Shot Learning Shah Nawaz Jacopo Cavazza Alessio Del Bue ObjD FedML VLM 28 3 0 03 Jan 2022
LaTr: Layout-Aware Transformer for Scene-Text VQA Ali Furkan Biten Ron Litman Yusheng Xie Srikar Appalaraju R. Manmatha ViT 39 100 0 23 Dec 2021
Comprehensive Visual Question Answering on Point Clouds through Compositional Scene Manipulation Xu Yan Zhihao Yuan Yuhao Du Yinghong Liao Yao Guo Zhen Li Shuguang Cui 3DPC CoGe 23 14 0 22 Dec 2021
MuMuQA: Multimedia Multi-Hop News Question Answering via Cross-Media Knowledge Extraction and Grounding Revanth Reddy Gangi Reddy Xilin Rui Manling Li Xudong Lin Haoyang Wen ... Joey Tianyi Zhou Avirup Sil Shih-Fu Chang Alex Schwing Heng Ji 25 31 0 20 Dec 2021
Hateful Memes Challenge: An Enhanced Multimodal Framework Aijing Gao Bingjun Wang Jiaqi Yin Yating Tian 21 2 0 20 Dec 2021
RegionCLIP: Region-based Language-Image Pretraining Yiwu Zhong Jianwei Yang Pengchuan Zhang Chunyuan Li Noel Codella ... Luowei Zhou Xiyang Dai Lu Yuan Yin Li Jianfeng Gao VLM CLIP 40 557 0 16 Dec 2021
Distilled Dual-Encoder Model for Vision-Language Understanding Zekun Wang Wenhui Wang Haichao Zhu Ming Liu Bing Qin Furu Wei VLM FedML 29 30 0 16 Dec 2021
KAT: A Knowledge Augmented Transformer for Vision-and-Language Liangke Gui Borui Wang Qiuyuan Huang Alexander G. Hauptmann Yonatan Bisk Jianfeng Gao 25 153 0 16 Dec 2021
SGEITL: Scene Graph Enhanced Image-Text Learning for Visual Commonsense Reasoning Zhecan Wang Haoxuan You Liunian Harold Li Alireza Zareian Suji Park Yiqing Liang Kai-Wei Chang Shih-Fu Chang ReLM LRM 18 30 0 16 Dec 2021
Hierarchical Cross-Modality Semantic Correlation Learning Model for Multimodal Summarization Litian Zhang Xiaoming Zhang Junshu Pan Feiran Huang 18 47 0 16 Dec 2021
3D Question Answering Shuquan Ye Dongdong Chen Songfang Han Jing Liao ViT 31 47 0 15 Dec 2021
Improving Conversational Recommendation Systems' Quality with Context-Aware Item Meta Information Bowen Yang Cong Han Yu Li Lei Zuo Zhou Yu 17 41 0 15 Dec 2021
Dual-Key Multimodal Backdoors for Visual Question Answering Matthew Walmer Karan Sikka Indranil Sur Abhinav Shrivastava Susmit Jha AAML 29 34 0 14 Dec 2021
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena Letitia Parcalabescu Michele Cafagna Lilitta Muradjan Anette Frank Iacer Calixto Albert Gatt CoGe 34 110 0 14 Dec 2021
CoCo-BERT: Improving Video-Language Pre-training with Contrastive Cross-modal Matching and Denoising Jianjie Luo Yehao Li Yingwei Pan Ting Yao Hongyang Chao Tao Mei VLM 18 41 0 14 Dec 2021
Towards a Unified Foundation Model: Jointly Pre-Training Transformers on Unpaired Images and Text Qing Li Boqing Gong Huayu Chen Dan Kondratyuk Xianzhi Du Ming-Hsuan Yang Matthew A. Brown ViT 19 17 0 14 Dec 2021
Improving and Diagnosing Knowledge-Based Visual Question Answering via Entity Enhanced Knowledge Injection Diego Garcia-Olano Yasumasa Onoe Joydeep Ghosh 21 18 0 13 Dec 2021
VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks Yi-Lin Sung Jaemin Cho Joey Tianyi Zhou VLM VPVLM 37 344 0 13 Dec 2021
ITA: Image-Text Alignments for Multi-Modal Named Entity Recognition Xinyu Wang Min Gui Yong-jia Jiang Zixia Jia Nguyen Bach Tao Wang Zhongqiang Huang Fei Huang Kewei Tu 44 52 0 13 Dec 2021
VUT: Versatile UI Transformer for Multi-Modal Multi-Task User Interface Modeling Yang Li Gang Li Xin Zhou Mostafa Dehghani A. Gritsenko MLLM 45 35 0 10 Dec 2021
Unified Multimodal Pre-training and Prompt-based Tuning for Vision-Language Understanding and Generation Tianyi Liu Zuxuan Wu Wenhan Xiong Jingjing Chen Yu-Gang Jiang VLM MLLM 32 10 0 10 Dec 2021
MAGMA -- Multimodal Augmentation of Generative Models through Adapter-based Finetuning C. Eichenberg Sid Black Samuel Weinbach Letitia Parcalabescu Anette Frank MLLM VLM 23 100 0 09 Dec 2021
CLIP2StyleGAN: Unsupervised Extraction of StyleGAN Edit Directions Rameen Abdal Peihao Zhu John C. Femiani Niloy J. Mitra Peter Wonka CLIP 39 103 0 09 Dec 2021
HairCLIP: Design Your Hair by Text and Reference Image Tianyi Wei Dongdong Chen Wenbo Zhou Jing Liao Zhentao Tan Lu Yuan Weiming Zhang Nenghai Yu CLIP 38 108 0 09 Dec 2021
FLAVA: A Foundational Language And Vision Alignment Model Amanpreet Singh Ronghang Hu Vedanuj Goswami Guillaume Couairon Wojciech Galuba Marcus Rohrbach Douwe Kiela CLIP VLM 40 691 0 08 Dec 2021
MLP Architectures for Vision-and-Language Modeling: An Empirical Study Yi-Liang Nie Linjie Li Zhe Gan Shuohang Wang Chenguang Zhu Michael Zeng Zicheng Liu Joey Tianyi Zhou Lijuan Wang 27 6 0 08 Dec 2021
Contrastive Instruction-Trajectory Learning for Vision-Language Navigation Xiwen Liang Fengda Zhu Yi Zhu Bingqian Lin Bing Wang Xiaodan Liang 27 23 0 08 Dec 2021
Grounded Language-Image Pre-training Liunian Harold Li Pengchuan Zhang Haotian Zhang Jianwei Yang Chunyuan Li ... Lu Yuan Lei Zhang Lei Li Kai-Wei Chang Jianfeng Gao ObjD VLM 36 1,025 0 07 Dec 2021
UNITER-Based Situated Coreference Resolution with Rich Multimodal Input Yichen Huang Yuchen Wang Yik-Cheung Tam 38 8 0 07 Dec 2021
Semantic Segmentation In-the-Wild Without Seeing Any Segmentation Examples Nir Zabari Yedid Hoshen VLM 38 26 0 06 Dec 2021
General Facial Representation Learning in a Visual-Linguistic Manner Yinglin Zheng Hao Yang Ting Zhang Jianmin Bao Dongdong Chen Yangyu Huang Lu Yuan Dong Chen Ming Zeng Fang Wen CVBM 146 164 0 06 Dec 2021