From Colors to Classes: Emergence of Concepts in Vision Transformers

From Colors to Classes: Emergence of Concepts in Vision Transformers

31 March 2025

Teresa Dorszewski

Lenka Tětková

Lars Kai Hansen

Kristoffer Wickstrøm

Papers citing "From Colors to Classes: Emergence of Concepts in Vision Transformers"

19 / 19 papers shown

Title
From What to How: Attributing CLIP's Latent Components Reveals Unexpected Semantic Reliance Maximilian Dreyer Lorenz Hufe J. Berend Thomas Wiegand Sebastian Lapuschkin Wojciech Samek 37 0 0 26 May 2025
Beyond Scalars: Concept-Based Alignment Analysis in Vision Transformers Johanna Vielhaben Dilyara Bareeva J. Berend Wojciech Samek Nils Strodthoff 89 3 0 09 Dec 2024
CRAFT: Concept Recursive Activation FacTorization for Explainability Thomas Fel Agustin Picard Louis Bethune Thibaut Boissin David Vigouroux Julien Colin Rémi Cadène Thomas Serre 60 114 0 17 Nov 2022
CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networks Tuomas P. Oikarinen Tsui-Wei Weng VLM 50 88 1 23 Apr 2022
Natural Language Descriptions of Deep Visual Features Evan Hernandez Sarah Schwettmann David Bau Teona Bagashvili Antonio Torralba Jacob Andreas MILM 294 122 0 26 Jan 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 451 7,739 0 11 Nov 2021
Are Convolutional Neural Networks or Transformers more like human vision? Shikhar Tuli Ishita Dasgupta Erin Grant Thomas Griffiths ViT FaML 56 185 0 15 May 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 909 29,372 0 26 Feb 2021
A Survey on Visual Transformer Kai Han Yunhe Wang Hanting Chen Xinghao Chen Jianyuan Guo ... Chunjing Xu Yixing Xu Zhaohui Yang Yiman Zhang Dacheng Tao ViT 193 2,223 0 23 Dec 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 637 41,003 0 22 Oct 2020
Shortcut Learning in Deep Neural Networks Robert Geirhos J. Jacobsen Claudio Michaelis R. Zemel Wieland Brendel Matthias Bethge Felix Wichmann 201 2,048 0 16 Apr 2020
Adversarial Examples Are Not Bugs, They Are Features Andrew Ilyas Shibani Santurkar Dimitris Tsipras Logan Engstrom Brandon Tran Aleksander Madry SILM 89 1,839 0 06 May 2019
Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV) Been Kim Martin Wattenberg Justin Gilmer Carrie J. Cai James Wexler F. Viégas Rory Sayres FAtt 211 1,842 0 30 Nov 2017
Network Dissection: Quantifying Interpretability of Deep Visual Representations David Bau Bolei Zhou A. Khosla A. Oliva Antonio Torralba MILM FAtt 146 1,516 1 19 Apr 2017
Overcoming catastrophic forgetting in neural networks J. Kirkpatrick Razvan Pascanu Neil C. Rabinowitz J. Veness Guillaume Desjardins ... A. Grabska-Barwinska Demis Hassabis Claudia Clopath D. Kumaran R. Hadsell CLL 357 7,504 0 02 Dec 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 193,878 0 10 Dec 2015
How transferable are features in deep neural networks? J. Yosinski Jeff Clune Yoshua Bengio Hod Lipson OOD 231 8,336 0 06 Nov 2014
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 1.7K 39,525 0 01 Sep 2014
Visualizing and Understanding Convolutional Networks Matthew D. Zeiler Rob Fergus FAtt SSL 591 15,876 0 12 Nov 2013