LXMERT: Learning Cross-Modality Encoder Representations from Transformers

20 August 2019

Papers citing "LXMERT: Learning Cross-Modality Encoder Representations from Transformers"

50 / 1,513 papers shown

Title
LAVENDER: Unifying Video-Language Understanding as Masked Language Modeling Linjie Li Zhe Gan Kevin Qinghong Lin Chung-Ching Lin Zicheng Liu Ce Liu Lijuan Wang MLLM VLM 25 82 0 14 Jun 2022
LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learning Yi-Lin Sung Jaemin Cho Joey Tianyi Zhou VLM 26 238 0 13 Jun 2022
Multimodal Learning with Transformers: A Survey Peng Xu Xiatian Zhu David Clifton ViT 82 531 0 13 Jun 2022
Compositional Mixture Representations for Vision and Text Stephan Alaniz Marco Federici Zeynep Akata CoGe OCL VLM 25 2 0 13 Jun 2022
Bringing Image Scene Structure to Video via Frame-Clip Consistency of Object Tokens Elad Ben-Avraham Roei Herzig K. Mangalam Amir Bar Anna Rohrbach Leonid Karlinsky Trevor Darrell Amir Globerson 19 0 0 13 Jun 2022
Language Models are General-Purpose Interfaces Y. Hao Haoyu Song Li Dong Shaohan Huang Zewen Chi Wenhui Wang Shuming Ma Furu Wei MLLM 35 96 0 13 Jun 2022
INDIGO: Intrinsic Multimodality for Domain Generalization Puneet Mangla Shivam Chandhok Milan Aggarwal V. Balasubramanian Balaji Krishnamurthy VLM 46 2 0 13 Jun 2022
GLIPv2: Unifying Localization and Vision-Language Understanding Haotian Zhang Pengchuan Zhang Xiaowei Hu Yen-Chun Chen Liunian Harold Li Xiyang Dai Lijuan Wang Lu Yuan Lei Li Jianfeng Gao ObjD VLM 30 291 0 12 Jun 2022
A Unified Continuous Learning Framework for Multi-modal Knowledge Discovery and Pre-training Zhihao Fan Zhongyu Wei Jingjing Chen Siyuan Wang Zejun Li Jiarong Xu Xuanjing Huang CLL 17 6 0 11 Jun 2022
Uni-Perceiver-MoE: Learning Sparse Generalist Models with Conditional MoEs Jinguo Zhu Xizhou Zhu Wenhai Wang Xiaohua Wang Hongsheng Li Xiaogang Wang Jifeng Dai MoMe MoE 39 66 0 09 Jun 2022
Revealing Single Frame Bias for Video-and-Language Learning Jie Lei Tamara L. Berg Joey Tianyi Zhou 24 111 0 07 Jun 2022
cViL: Cross-Lingual Training of Vision-Language Models using Knowledge Distillation Kshitij Gupta Devansh Gautam R. Mamidi VLM 29 3 0 07 Jun 2022
Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts Basil Mustafa C. Riquelme J. Puigcerver Rodolphe Jenatton N. Houlsby VLM MoE 33 185 0 06 Jun 2022
What do tokens know about their characters and how do they know it? Ayush Kaushal Kyle Mahowald 40 28 0 06 Jun 2022
Towards Fast Adaptation of Pretrained Contrastive Models for Multi-channel Video-Language Retrieval Xudong Lin Simran Tiwari Shiyuan Huang Manling Li Mike Zheng Shou Heng Ji Shih-Fu Chang 38 20 0 05 Jun 2022
A-OKVQA: A Benchmark for Visual Question Answering using World Knowledge Dustin Schwenk Apoorv Khandelwal Christopher Clark Kenneth Marino Roozbeh Mottaghi 18 511 0 03 Jun 2022
VL-BEiT: Generative Vision-Language Pretraining Hangbo Bao Wenhui Wang Li Dong Furu Wei VLM 18 45 0 02 Jun 2022
Cross-View Language Modeling: Towards Unified Cross-Lingual Cross-Modal Pre-training Yan Zeng Wangchunshu Zhou Ao Luo Ziming Cheng Xinsong Zhang VLM 29 30 0 01 Jun 2022
Neural Retriever and Go Beyond: A Thesis Proposal Man Luo 42 1 0 31 May 2022
VLUE: A Multi-Task Benchmark for Evaluating Vision-Language Models Wangchunshu Zhou Yan Zeng Shizhe Diao Xinsong Zhang CoGe VLM 37 13 0 30 May 2022
Prompt-aligned Gradient for Prompt Tuning Beier Zhu Yulei Niu Yucheng Han Yuehua Wu Hanwang Zhang VLM 189 276 0 30 May 2022
UPB at SemEval-2022 Task 5: Enhancing UNITER with Image Sentiment and Graph Convolutional Networks for Multimedia Automatic Misogyny Identification Andrei Paraschiv M. Dascalu Dumitru-Clementin Cercel 27 3 0 29 May 2022
VD-PCR: Improving Visual Dialog with Pronoun Coreference Resolution Xintong Yu Hongming Zhang Ruixin Hong Yangqiu Song Changshui Zhang 17 13 0 29 May 2022
Multimodal Masked Autoencoders Learn Transferable Representations Xinyang Geng Hao Liu Lisa Lee Dale Schuurams Sergey Levine Pieter Abbeel 26 115 0 27 May 2022
GIT: A Generative Image-to-text Transformer for Vision and Language Jianfeng Wang Zhengyuan Yang Xiaowei Hu Linjie Li Kevin Qinghong Lin Zhe Gan Zicheng Liu Ce Liu Lijuan Wang VLM 66 531 0 27 May 2022
Multimodal Knowledge Alignment with Reinforcement Learning Youngjae Yu Jiwan Chung Heeseung Yun Jack Hessel Jinho Park ... Prithviraj Ammanabrolu Rowan Zellers Ronan Le Bras Gunhee Kim Yejin Choi VLM 123 36 0 25 May 2022
DisinfoMeme: A Multimodal Dataset for Detecting Meme Intentionally Spreading Out Disinformation Jingnong Qu Liunian Harold Li Jieyu Zhao Sunipa Dev Kai-Wei Chang 26 12 0 25 May 2022
Guiding Visual Question Answering with Attention Priors T. Le Vuong Le Sunil R. Gupta Svetha Venkatesh T. Tran 33 6 0 25 May 2022
HiVLP: Hierarchical Vision-Language Pre-Training for Fast Image-Text Retrieval Feilong Chen Xiuyi Chen Jiaxin Shi Duzhen Zhang Jianlong Chang Qi Tian VLM CLIP 36 6 0 24 May 2022
mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections Chenliang Li Haiyang Xu Junfeng Tian Wei Wang Ming Yan ... Ji Zhang Songfang Huang Feiran Huang Jingren Zhou Luo Si VLM MLLM 36 213 0 24 May 2022
ItemSage: Learning Product Embeddings for Shopping Recommendations at Pinterest Paul Baltescu Haoyu Chen Nikil Pancha Andrew Zhai J. Leskovec Charles R. Rosenberg DML 45 33 0 24 May 2022
On Advances in Text Generation from Images Beyond Captioning: A Case Study in Self-Rationalization Shruti Palaskar Akshita Bhagia Yonatan Bisk Florian Metze A. Black Ana Marasović 36 4 0 24 May 2022
PEVL: Position-enhanced Pre-training and Prompt Tuning for Vision-language Models Yuan Yao Qi-An Chen Ao Zhang Wei Ji Zhiyuan Liu Tat-Seng Chua Maosong Sun VLM MLLM 34 38 0 23 May 2022
Supporting Vision-Language Model Inference with Confounder-pruning Knowledge Prompt Jiangmeng Li Wenyi Mo Jingyao Wang Fuchun Sun Changwen Zheng Hui Xiong Ji-Rong Wen VLM 18 0 0 23 May 2022
Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners Zhenhailong Wang Manling Li Ruochen Xu Luowei Zhou Jie Lei ... Chenguang Zhu Derek Hoiem Shih-Fu Chang Joey Tianyi Zhou Heng Ji MLLM VLM 170 138 0 22 May 2022
Training Vision-Language Transformers from Captions Liangke Gui Yingshan Chang Qiuyuan Huang Subhojit Som Alexander G. Hauptmann Jianfeng Gao Yonatan Bisk VLM ViT 174 11 0 19 May 2022
On the Limits of Evaluating Embodied Agent Model Generalization Using Validation Sets Hyounghun Kim Aishwarya Padmakumar Di Jin Joey Tianyi Zhou Dilek Z. Hakkani-Tür 14 0 0 18 May 2022
Gender and Racial Bias in Visual Question Answering Datasets Yusuke Hirota Yuta Nakashima Noa Garcia FaML 148 46 0 17 May 2022
What do Models Learn From Training on More Than Text? Measuring Visual Commonsense Knowledge Lovisa Hagström Richard Johansson VLM 37 4 0 14 May 2022
Multimodal Conversational AI: A Survey of Datasets and Approaches Anirudh S. Sundar Larry Heck 48 29 0 13 May 2022
Localized Vision-Language Matching for Open-vocabulary Object Detection M. A. Bravo Sudhanshu Mittal Thomas Brox VLM ObjD 20 25 0 12 May 2022
Learning to Answer Visual Questions from Web Videos Antoine Yang Antoine Miech Josef Sivic Ivan Laptev Cordelia Schmid ViT 48 34 0 10 May 2022
Beyond a Pre-Trained Object Detector: Cross-Modal Textual and Visual Context for Image Captioning Chia-Wen Kuo Z. Kira 32 52 0 09 May 2022
Beyond Bounding Box: Multimodal Knowledge Learning for Object Detection Wei Feng Xingyuan Bu Chenchen Zhang Xubin Li VLM 12 4 0 09 May 2022
Chart Question Answering: State of the Art and Future Directions Enamul Hoque P. Kavehzadeh Ahmed Masry 21 42 0 08 May 2022
ConvMAE: Masked Convolution Meets Masked Autoencoders Peng Gao Teli Ma Hongsheng Li Ziyi Lin Jifeng Dai Yu Qiao ViT 24 122 0 08 May 2022
Good Visual Guidance Makes A Better Extractor: Hierarchical Visual Prefix for Multimodal Entity and Relation Extraction Xiang Chen Ningyu Zhang Lei Li Yunzhi Yao Shumin Deng Chuanqi Tan Fei Huang Luo Si Huajun Chen 34 33 0 07 May 2022
DouFu: A Double Fusion Joint Learning Method For Driving Trajectory Representation Han Wang Zhou Huang Xiao Zhou Ganmin Yin Yi Bao Yihang Bao 37 4 0 05 May 2022
Declaration-based Prompt Tuning for Visual Question Answering Yuhang Liu Wei Wei Daowan Peng Feida Zhu MLLM VLM 27 19 0 05 May 2022
Hybrid Transformer with Multi-level Fusion for Multimodal Knowledge Graph Completion Xiang Chen Ningyu Zhang Lei Li Shumin Deng Chuanqi Tan Changliang Xu Fei Huang Luo Si Huajun Chen 36 127 0 04 May 2022