Understanding the Effect of using Semantically Meaningful Tokens for Visual Representation Learning

26 May 2024

Papers citing "Understanding the Effect of using Semantically Meaningful Tokens for Visual Representation Learning"

21 / 21 papers shown

Title
Groma: Localized Visual Tokenization for Grounding Multimodal Large Language Models Chuofan Ma Yi Jiang Jiannan Wu Zehuan Yuan Xiaojuan Qi VLM ObjD 47 56 0 19 Apr 2024
DAT++: Spatially Dynamic Vision Transformer with Deformable Attention Zhuofan Xia Xuran Pan Shiji Song Li Erran Li Gao Huang ViT 45 25 0 04 Sep 2023
When and why vision-language models behave like bags-of-words, and what to do about it? Mert Yuksekgonul Federico Bianchi Pratyusha Kalluri Dan Jurafsky James Zou VLM CoGe 54 378 0 04 Oct 2022
Panoptic Scene Graph Generation Jingkang Yang Yi Zhe Ang Zujin Guo Kaiyang Zhou Wayne Zhang Ziwei Liu 87 111 0 22 Jul 2022
Vision GNN: An Image is Worth Graph of Nodes Kai Han Yunhe Wang Jianyuan Guo Yehui Tang Enhua Wu GNN 3DH 56 361 0 01 Jun 2022
Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality Tristan Thrush Ryan Jiang Max Bartolo Amanpreet Singh Adina Williams Douwe Kiela Candace Ross CoGe 80 413 0 07 Apr 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong Guosheng Lin MLLM BDL VLM CLIP 436 4,283 0 28 Jan 2022
FLAVA: A Foundational Language And Vision Alignment Model Amanpreet Singh Ronghang Hu Vedanuj Goswami Guillaume Couairon Wojciech Galuba Marcus Rohrbach Douwe Kiela CLIP VLM 64 695 0 08 Dec 2021
Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Concepts Yan Zeng Xinsong Zhang Hang Li VLM CLIP 42 302 0 16 Nov 2021
DPT: Deformable Patch-based Transformer for Visual Recognition Zhiyang Chen Yousong Zhu Chaoyang Zhao Guosheng Hu Wei Zeng Jinqiao Wang Ming Tang ViT 24 99 0 30 Jul 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng Zhang Stephen Lin B. Guo ViT 295 21,051 0 25 Mar 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 666 28,659 0 26 Feb 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 275 6,657 0 23 Dec 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 340 40,217 0 22 Oct 2020
VisualBERT: A Simple and Performant Baseline for Vision and Language Liunian Harold Li Mark Yatskar Da Yin Cho-Jui Hsieh Kai-Wei Chang VLM 104 1,939 0 09 Aug 2019
Deep Learning using Rectified Linear Units (ReLU) Abien Fred Agarap 54 3,195 0 22 Mar 2018
Semi-Supervised Classification with Graph Convolutional Networks Thomas Kipf Max Welling GNN SSL 432 28,795 0 09 Sep 2016
Semantic Understanding of Scenes through the ADE20K Dataset Bolei Zhou Hang Zhao Xavier Puig Tete Xiao Sanja Fidler Adela Barriuso Antonio Torralba SSeg 327 1,850 0 18 Aug 2016
Fully Convolutional Networks for Semantic Segmentation Evan Shelhamer Jonathan Long Trevor Darrell VOS SSeg 293 37,704 0 20 May 2016
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations Ranjay Krishna Yuke Zhu Oliver Groth Justin Johnson Kenji Hata ... Yannis Kalantidis Li Li David A. Shamma Michael S. Bernstein Fei-Fei Li 165 5,706 0 23 Feb 2016
U-Net: Convolutional Networks for Biomedical Image Segmentation Olaf Ronneberger Philipp Fischer Thomas Brox SSeg 3DV 1.1K 76,547 0 18 May 2015