ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision

5 February 2021

Papers citing "ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision"

50 / 336 papers shown

Title
Scratching Visual Transformer's Back with Uniform Attention Nam Hyeon-Woo Kim Yu-Ji Byeongho Heo Doonyoon Han Seong Joon Oh Tae-Hyun Oh 364 23 0 16 Oct 2022
Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training Wenliang Dai Zihan Liu Ziwei Ji Dan Su Pascale Fung MLLM VLM 32 62 0 14 Oct 2022
One does not fit all! On the Complementarity of Vision Encoders for Vision and Language Tasks Gregor Geigle Chen Cecilia Liu Jonas Pfeiffer Iryna Gurevych VLM 28 1 0 12 Oct 2022
LiveSeg: Unsupervised Multimodal Temporal Segmentation of Long Livestream Videos Jielin Qiu Franck Dernoncourt Trung Bui Zhaowen Wang Ding Zhao Hailin Jin AI4TS 22 5 0 12 Oct 2022
Contrastive Video-Language Learning with Fine-grained Frame Sampling Zixu Wang Yujie Zhong Yishu Miao Lin Ma Lucia Specia 52 11 0 10 Oct 2022
CLIP-Diffusion-LM: Apply Diffusion Model on Image Captioning Shi-You Xu VLM DiffM 32 11 0 10 Oct 2022
CLIP model is an Efficient Continual Learner Vishal G. Thengane Salman Khan Munawar Hayat Fahad Shahbaz Khan BDL VLM CLL 112 46 0 06 Oct 2022
Uncertainty Estimation for Multi-view Data: The Power of Seeing the Whole Picture M. Jung He Zhao Joanna Dipnall Belinda Gabbe Lan Du UQCV EDL 57 12 0 06 Oct 2022
PLOT: Prompt Learning with Optimal Transport for Vision-Language Models Guangyi Chen Weiran Yao Xiangchen Song Xinyue Li Yongming Rao Kun Zhang VPVLM VLM 8 62 0 03 Oct 2022
Multimodal Analogical Reasoning over Knowledge Graphs Ningyu Zhang Lei Li Xiang Chen Xiaozhuan Liang Shumin Deng Huajun Chen 54 26 0 01 Oct 2022
TVLT: Textless Vision-Language Transformer Zineng Tang Jaemin Cho Yixin Nie Joey Tianyi Zhou VLM 51 28 0 28 Sep 2022
TokenFlow: Rethinking Fine-grained Cross-modal Alignment in Vision-Language Retrieval Xiaohan Zou Changqiao Wu Lele Cheng Zhongyuan Wang 94 6 0 28 Sep 2022
Show, Interpret and Tell: Entity-aware Contextualised Image Captioning in Wikipedia K. Nguyen Ali Furkan Biten Andrés Mafla Lluís Gómez Dimosthenis Karatzas 36 10 0 21 Sep 2022
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Pan Lu Swaroop Mishra Tony Xia Liang Qiu Kai-Wei Chang Song-Chun Zhu Oyvind Tafjord Peter Clark Ashwin Kalyan ELM ReLM LRM 211 1,113 0 20 Sep 2022
Non-Linguistic Supervision for Contrastive Learning of Sentence Embeddings Yiren Jian Chongyang Gao Soroush Vosoughi SSL 31 15 0 20 Sep 2022
VIPHY: Probing "Visible" Physical Commonsense Knowledge Shikhar Singh Ehsan Qasemi Muhao Chen 46 6 0 15 Sep 2022
CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignment Hongwei Xue Yuchong Sun Bei Liu Jianlong Fu Rui Song Houqiang Li Jiebo Luo CLIP VLM 25 68 0 14 Sep 2022
VL-Taboo: An Analysis of Attribute-based Zero-shot Capabilities of Vision-Language Models Felix Vogel Nina Shvetsova Leonid Karlinsky Hilde Kuehne VLM 63 7 0 12 Sep 2022
FETA: Towards Specializing Foundation Models for Expert Task Applications Amit Alfassy Assaf Arbelle Oshri Halimi Sivan Harary Roei Herzig ... Christoph Auer Kate Saenko Peter W. J. Staar Rogerio Feris Leonid Karlinsky 23 19 0 08 Sep 2022
MuMUR : Multilingual Multimodal Universal Retrieval Avinash Madasu Estelle Aflalo Gabriela Ben-Melech Stan Shachar Rosenman Shao-Yen Tseng Gedas Bertasius Vasudev Lal 44 3 0 24 Aug 2022
Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks Wenhui Wang Hangbo Bao Li Dong Johan Bjorck Zhiliang Peng ... Kriti Aggarwal O. Mohammed Saksham Singhal Subhojit Som Furu Wei MLLM VLM ViT 54 629 0 22 Aug 2022
VLMAE: Vision-Language Masked Autoencoder Su He Taian Guo Tao Dai Ruizhi Qiao Chen Wu Xiujun Shu Bohan Ren VLM 34 11 0 19 Aug 2022
See Finer, See More: Implicit Modality Alignment for Text-based Person Retrieval Xiujun Shu Wei Wen Haoqian Wu Keyun Chen Yi-Zhe Song Ruizhi Qiao Bohan Ren Xiao Wang 27 91 0 18 Aug 2022
Multimodal Lecture Presentations Dataset: Understanding Multimodality in Educational Slides Dong Won Lee Chaitanya Ahuja Paul Pu Liang Sanika Natu Louis-Philippe Morency 23 7 0 17 Aug 2022
Your ViT is Secretly a Hybrid Discriminative-Generative Diffusion Model Xiulong Yang Sheng-Min Shih Yinlin Fu Xiaoting Zhao Shihao Ji DiffM 33 56 0 16 Aug 2022
ARMANI: Part-level Garment-Text Alignment for Unified Cross-Modal Fashion Design Xujie Zhang Yuyang Sha Michael C. Kampffmeyer Zhenyu Xie Zequn Jie Chengwen Huang Jianqing Peng Xiaodan Liang 14 18 0 11 Aug 2022
GRIT-VLP: Grouped Mini-batch Sampling for Efficient Vision and Language Pre-training Jaeseok Byun Taebaek Hwang Jianlong Fu Taesup Moon VLM 23 11 0 08 Aug 2022
Masked Vision and Language Modeling for Multi-modal Representation Learning Gukyeong Kwon Zhaowei Cai Avinash Ravichandran Erhan Bas Rahul Bhotika Stefano Soatto 36 67 0 03 Aug 2022
Augmenting Vision Language Pretraining by Learning Codebook with Visual Semantics Xiaoyuan Guo Jiali Duan C.-C. Jay Kuo J. Gichoya Imon Banerjee VLM 22 1 0 31 Jul 2022
Visual correspondence-based explanations improve AI robustness and human-AI team accuracy Giang Nguyen Mohammad Reza Taesiri Anh Totti Nguyen 30 42 0 26 Jul 2022
GRIT: Faster and Better Image captioning Transformer Using Dual Visual Features Van-Quang Nguyen Masanori Suganuma Takayuki Okatani ViT 36 106 0 20 Jul 2022
Boosting Multi-Modal E-commerce Attribute Value Extraction via Unified Learning Scheme and Dynamic Range Minimization Meng-yang Liu Chao Zhu Hongyu Gao Weibo Gu Hongfa Wang Wei Liu Xu-Cheng Yin 24 2 0 15 Jul 2022
Open-Vocabulary Multi-Label Classification via Multi-Modal Knowledge Transfer Su He Taian Guo Tao Dai Ruizhi Qiao Bo Ren Shutao Xia VLM 78 49 0 05 Jul 2022
Counterfactually Measuring and Eliminating Social Bias in Vision-Language Pre-training Models Yi Zhang Junyan Wang Jitao Sang 22 27 0 03 Jul 2022
VL-CheckList: Evaluating Pre-trained Vision-Language Models with Objects, Attributes and Relations Tiancheng Zhao Tianqi Zhang Mingwei Zhu Haozhan Shen Kyusong Lee Xiaopeng Lu Jianwei Yin VLM CoGe MLLM 45 91 0 01 Jul 2022
LViT: Language meets Vision Transformer in Medical Image Segmentation Zihan Li Yunxiang Li Qingde Li Puyang Wang Dazhou Guo Le Lu D. Jin You Zhang Qingqi Hong VLM MedIm 67 134 0 29 Jun 2022
VLMixer: Unpaired Vision-Language Pre-training via Cross-Modal CutMix Teng Wang Wenhao Jiang Zhichao Lu Feng Zheng Ran Cheng Chengguo Yin Ping Luo VLM 34 42 0 17 Jun 2022
Entity-Graph Enhanced Cross-Modal Pretraining for Instance-level Product Retrieval Xiao Dong Xunlin Zhan Yunchao Wei Xiaoyong Wei Yaowei Wang Minlong Lu Xiaochun Cao Xiaodan Liang 27 11 0 17 Jun 2022
Zero-Shot Video Question Answering via Frozen Bidirectional Language Models Antoine Yang Antoine Miech Josef Sivic Ivan Laptev Cordelia Schmid 36 228 0 16 Jun 2022
Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone Zi-Yi Dou Aishwarya Kamath Zhe Gan Pengchuan Zhang Jianfeng Wang ... Ce Liu Yann LeCun Nanyun Peng Jianfeng Gao Lijuan Wang VLM ObjD 30 124 0 15 Jun 2022
Multimodal Learning with Transformers: A Survey P. Xu Xiatian Zhu David A. Clifton ViT 72 528 0 13 Jun 2022
Revealing Single Frame Bias for Video-and-Language Learning Jie Lei Tamara L. Berg Joey Tianyi Zhou 24 111 0 07 Jun 2022
VLUE: A Multi-Task Benchmark for Evaluating Vision-Language Models Wangchunshu Zhou Yan Zeng Shizhe Diao Xinsong Zhang CoGe VLM 32 13 0 30 May 2022
Prompt-aligned Gradient for Prompt Tuning Beier Zhu Yulei Niu Yucheng Han Yuehua Wu Hanwang Zhang VLM 189 272 0 30 May 2022
GIT: A Generative Image-to-text Transformer for Vision and Language Jianfeng Wang Zhengyuan Yang Xiaowei Hu Linjie Li Kevin Qinghong Lin Zhe Gan Zicheng Liu Ce Liu Lijuan Wang VLM 59 529 0 27 May 2022
HiVLP: Hierarchical Vision-Language Pre-Training for Fast Image-Text Retrieval Feilong Chen Xiuyi Chen Jiaxin Shi Duzhen Zhang Jianlong Chang Qi Tian VLM CLIP 34 6 0 24 May 2022
Utilizing Language-Image Pretraining for Efficient and Robust Bilingual Word Alignment Tuan Dinh Jy-yong Sohn Shashank Rajput Timothy Ossowski Yifei Ming Junjie Hu Dimitris Papailiopoulos Kangwook Lee 28 0 0 23 May 2022
PEVL: Position-enhanced Pre-training and Prompt Tuning for Vision-language Models Yuan Yao Qi-An Chen Ao Zhang Wei Ji Zhiyuan Liu Tat-Seng Chua Maosong Sun VLM MLLM 26 38 0 23 May 2022
CoCa: Contrastive Captioners are Image-Text Foundation Models Jiahui Yu Zirui Wang Vijay Vasudevan Legg Yeung Mojtaba Seyedhosseini Yonghui Wu VLM CLIP OffRL 79 1,262 0 04 May 2022
Answer-Me: Multi-Task Open-Vocabulary Visual Question Answering A. Piergiovanni Wei Li Weicheng Kuo M. Saffar Fred Bertsch A. Angelova 17 16 0 02 May 2022