Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers

2 April 2020

Zhicheng Huang

Papers citing "Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers"

50 / 287 papers shown

Title
Accelerating Vision-Language Pretraining with Free Language Modeling Teng Wang Yixiao Ge Feng Zheng Ran Cheng Ying Shan Xiaohu Qie Ping Luo VLM MLLM 93 9 0 24 Mar 2023
Transformers in Speech Processing: A Survey S. Latif Aun Zaidi Heriberto Cuayáhuitl Fahad Shamshad Moazzam Shoukat Junaid Qadir 42 47 0 21 Mar 2023
Investigating the Role of Attribute Context in Vision-Language Models for Object Recognition and Detection Kyle Buettner Adriana Kovashka 22 0 0 17 Mar 2023
From Visual Prompt Learning to Zero-Shot Transfer: Mapping Is All You Need Ziqing Yang Zeyang Sha Michael Backes Yang Zhang VPVLM VLM 37 3 0 09 Mar 2023
Text-Visual Prompting for Efficient 2D Temporal Video Grounding Yimeng Zhang Xin Chen Jinghan Jia Sijia Liu Ke Ding 23 25 0 09 Mar 2023
A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT Yihan Cao Siyu Li Yixin Liu Zhiling Yan Yutong Dai Philip S. Yu Lichao Sun 29 507 0 07 Mar 2023
Large-scale Multi-Modal Pre-trained Models: A Comprehensive Survey Tianlin Li Guangyao Chen Guangwu Qian Pengcheng Gao Xiaoyong Wei Yaowei Wang Yonghong Tian Wen Gao AI4CE VLM 31 202 0 20 Feb 2023
Towards Unifying Medical Vision-and-Language Pre-training via Soft Prompts Zhihong Chen Shizhe Diao Benyou Wang Guanbin Li Xiang Wan MedIm 25 29 0 17 Feb 2023
Differentiable Outlier Detection Enable Robust Deep Multimodal Analysis Zhu Wang Sourav Medya Sathya Ravi VLM 28 0 0 11 Feb 2023
Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applications Muhammad Arslan Manzoor S. Albarri Ziting Xian Zaiqiao Meng Preslav Nakov Shangsong Liang AI4TS 31 26 0 01 Feb 2023
Debiased Fine-Tuning for Vision-language Models by Prompt Regularization Beier Zhu Yulei Niu Saeil Lee Minhoe Hur Hanwang Zhang VLM VPVLM 32 22 0 29 Jan 2023
Summarize the Past to Predict the Future: Natural Language Descriptions of Context Boost Multimodal Object Interaction Anticipation Razvan-George Pasca Alexey Gavryushin Muhammad Hamza Yen-Ling Kuo Kaichun Mo Luc Van Gool Otmar Hilliges Xi Wang 27 14 0 22 Jan 2023
Effective End-to-End Vision Language Pretraining with Semantic Visual Loss Xiaofeng Yang Fayao Liu Guosheng Lin VLM 26 7 0 18 Jan 2023
GIVL: Improving Geographical Inclusivity of Vision-Language Models with Pre-Training Methods Da Yin Feng Gao Govind Thattai Michael F. Johnston Kai-Wei Chang VLM 32 15 0 05 Jan 2023
VQA and Visual Reasoning: An Overview of Recent Datasets, Methods and Challenges R. Zakari Jim Wilson Owusu Hailin Wang Ke Qin Zaharaddeen Karami Lawal Yue-hong Dong LRM 33 16 0 26 Dec 2022
CLIPPO: Image-and-Language Understanding from Pixels Only Michael Tschannen Basil Mustafa N. Houlsby CLIP VLM 32 47 0 15 Dec 2022
Unifying Vision, Text, and Layout for Universal Document Processing Zineng Tang Ziyi Yang Guoxin Wang Yuwei Fang Yang Liu Chenguang Zhu Michael Zeng Chao-Yue Zhang Joey Tianyi Zhou VLM 32 105 0 05 Dec 2022
What do you MEME? Generating Explanations for Visual Semantic Role Labelling in Memes Shivam Sharma Siddhant Agarwal Tharun Suresh Preslav Nakov Md. Shad Akhtar Tanmoy Charkraborty VLM 28 18 0 01 Dec 2022
Improving Cross-Modal Retrieval with Set of Diverse Embeddings Dongwon Kim Nam-Won Kim Suha Kwak 24 37 0 30 Nov 2022
SegCLIP: Patch Aggregation with Learnable Centers for Open-Vocabulary Semantic Segmentation Huaishao Luo Junwei Bao Youzheng Wu Xiaodong He Tianrui Li VLM 32 144 0 27 Nov 2022
Seeing What You Miss: Vision-Language Pre-training with Semantic Completion Learning Yatai Ji Rong-Cheng Tu Jie Jiang Weijie Kong Chengfei Cai Wenzhe Zhao Hongfa Wang Yujiu Yang Wei Liu VLM 32 13 0 24 Nov 2022
X $^2$ -VLM: All-In-One Pre-trained Model For Vision-Language Tasks Yan Zeng Xinsong Zhang Hang Li Jiawei Wang Jipeng Zhang Hkust Wangchunshu Zhou VLM MLLM 28 14 0 22 Nov 2022
Perceiver-VL: Efficient Vision-and-Language Modeling with Iterative Latent Attention Zineng Tang Jaemin Cho Jie Lei Joey Tianyi Zhou VLM 24 9 0 21 Nov 2022
A survey on knowledge-enhanced multimodal learning Maria Lymperaiou Giorgos Stamou 41 13 0 19 Nov 2022
Detect Only What You Specify : Object Detection with Linguistic Target Moyuru Yamada ObjD VLM 33 0 0 18 Nov 2022
DiMBERT: Learning Vision-Language Grounded Representations with Disentangled Multimodal-Attention Fenglin Liu Xian Wu Shen Ge Xuancheng Ren Wei Fan Xu Sun Yuexian Zou VLM 75 12 0 28 Oct 2022
VLT: Vision-Language Transformer and Query Generation for Referring Segmentation Henghui Ding Chang Liu Suchen Wang Xudong Jiang 77 115 0 28 Oct 2022
Masked Vision-Language Transformer in Fashion Ge-Peng Ji Mingchen Zhuge D. Gao Deng-Ping Fan Daniel Gehrig Luc Van Gool 21 25 0 27 Oct 2022
MedCLIP: Contrastive Learning from Unpaired Medical Images and Text Zifeng Wang Zhenbang Wu Dinesh Agarwal Jimeng Sun CLIP VLM MedIm 49 399 0 18 Oct 2022
EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning Tiannan Wang Wangchunshu Zhou Yan Zeng Xinsong Zhang VLM 36 36 0 14 Oct 2022
That's the Wrong Lung! Evaluating and Improving the Interpretability of Unsupervised Multimodal Encoders for Medical Data Denis Jered McInerney Geoffrey S. Young Jan-Willem van de Meent Byron C. Wallace 18 0 0 12 Oct 2022
One does not fit all! On the Complementarity of Vision Encoders for Vision and Language Tasks Gregor Geigle Chen Cecilia Liu Jonas Pfeiffer Iryna Gurevych VLM 28 1 0 12 Oct 2022
Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learning Yuchong Sun Hongwei Xue Ruihua Song Bei Liu Huan Yang Jianlong Fu AI4TS VLM 20 68 0 12 Oct 2022
VoLTA: Vision-Language Transformer with Weakly-Supervised Local-Feature Alignment Shraman Pramanick Li Jing Sayan Nag Jiachen Zhu Hardik Shah Yann LeCun Ramalingam Chellappa 29 21 0 09 Oct 2022
RepsNet: Combining Vision with Language for Automated Medical Reports A. Tanwani Joelle Barral Daniel Freedman MedIm 35 20 0 27 Sep 2022
Align, Reason and Learn: Enhancing Medical Vision-and-Language Pre-training with Knowledge Zhihong Chen Guanbin Li Xiang Wan 127 65 0 15 Sep 2022
Multi-Modal Masked Autoencoders for Medical Vision-and-Language Pre-Training Zhihong Chen Yu Du Jinpeng Hu Yang Liu Guanbin Li Xiang Wan Tsung-Hui Chang 89 111 0 15 Sep 2022
Correlation Information Bottleneck: Towards Adapting Pretrained Multimodal Models for Robust Visual Question Answering Jingjing Jiang Zi-yi Liu Nanning Zheng 26 8 0 14 Sep 2022
CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignment Hongwei Xue Yuchong Sun Bei Liu Jianlong Fu Rui Song Houqiang Li Jiebo Luo CLIP VLM 25 68 0 14 Sep 2022
Efficient Vision-Language Pretraining with Visual Concepts and Hierarchical Alignment Mustafa Shukor Guillaume Couairon Matthieu Cord VLM CLIP 24 27 0 29 Aug 2022
Towards Open-vocabulary Scene Graph Generation with Prompt-based Finetuning Tao He Lianli Gao Jingkuan Song Yuan-Fang Li VLM 31 50 0 17 Aug 2022
Language-Guided Face Animation by Recurrent StyleGAN-based Generator Tiankai Hang Huan Yang Bei Liu Jianlong Fu Xin Geng B. Guo VGen 29 13 0 11 Aug 2022
Self-supervised Multi-modal Training from Uncurated Image and Reports Enables Zero-shot Oversight Artificial Intelligence in Radiology Sangjoon Park Eunha Lee Kyung Sook Shin Jeonghyeon Lee Jong Chul Ye 27 2 0 10 Aug 2022
GRIT-VLP: Grouped Mini-batch Sampling for Efficient Vision and Language Pre-training Jaeseok Byun Taebaek Hwang Jianlong Fu Taesup Moon VLM 20 11 0 08 Aug 2022
Fine-Grained Semantically Aligned Vision-Language Pre-Training Juncheng Li Xin He Longhui Wei Long Qian Linchao Zhu Lingxi Xie Yueting Zhuang Qi Tian Siliang Tang VLM 41 79 0 04 Aug 2022
FashionViL: Fashion-Focused Vision-and-Language Representation Learning Xiaoping Han Licheng Yu Xiatian Zhu Li Zhang Yi-Zhe Song Tao Xiang AI4TS 16 49 0 17 Jul 2022
Boosting Multi-Modal E-commerce Attribute Value Extraction via Unified Learning Scheme and Dynamic Range Minimization Meng-yang Liu Chao Zhu Hongyu Gao Weibo Gu Hongfa Wang Wei Liu Xu-Cheng Yin 24 2 0 15 Jul 2022
Global-local Motion Transformer for Unsupervised Skeleton-based Action Learning Boeun Kim H. Chang Jungho Kim J. Choi ViT 16 48 0 13 Jul 2022
Vision-and-Language Pretraining Thong Nguyen Cong-Duy Nguyen Xiaobao Wu See-Kiong Ng A. Luu VLM CLIP 24 2 0 05 Jul 2022
Contrastive Cross-Modal Knowledge Sharing Pre-training for Vision-Language Representation Learning and Retrieval Keyu Wen Zhenshan Tan Qingrong Cheng Cheng Chen X. Gu VLM 26 0 0 02 Jul 2022