Interpreting the Second-Order Effects of Neurons in CLIP

Interpreting the Second-Order Effects of Neurons in CLIP

6 June 2024

Yossi Gandelsman

Alexei A. Efros

Jacob Steinhardt

Papers citing "Interpreting the Second-Order Effects of Neurons in CLIP"

17 / 17 papers shown

Title
Prisma: An Open Source Toolkit for Mechanistic Interpretability in Vision and Video Sonia Joseph Praneet Suresh Lorenz Hufe Edward Stevinson Robert Graham Yash Vadi Danilo Bzdok Sebastian Lapuschkin Lee Sharkey Blake A. Richards 72 0 0 28 Apr 2025
Decoding Vision Transformers: the Diffusion Steering Lens Ryota Takatsuki Sonia Joseph Ippei Fujisawa Ryota Kanai DiffM 30 0 0 18 Apr 2025
Steering CLIP's vision transformer with sparse autoencoders Sonia Joseph Praneet Suresh Ethan Goldfarb Lorenz Hufe Yossi Gandelsman Robert Graham Danilo Bzdok Wojciech Samek Blake A. Richards 51 2 0 11 Apr 2025
Sparse Autoencoders Learn Monosemantic Features in Vision-Language Models Mateusz Pach Shyamgopal Karthik Quentin Bouniot Serge Belongie Zeynep Akata VLM 66 0 0 03 Apr 2025
Quantifying Interpretability in CLIP Models with Concept Consistency Avinash Madasu Vasudev Lal Phillip Howard VLM 69 0 0 14 Mar 2025
Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey Yunkai Dang Kaichen Huang Jiahao Huo Yibo Yan S. Huang ... Kun Wang Yong Liu Jing Shao Hui Xiong Xuming Hu LRM 101 15 0 03 Dec 2024
GIFT: A Framework for Global Interpretable Faithful Textual Explanations of Vision Classifiers Éloi Zablocki Valentin Gerard Amaia Cardiel Eric Gaussier Matthieu Cord Eduardo Valle 79 0 0 23 Nov 2024
Comparing Bottom-Up and Top-Down Steering Approaches on In-Context Learning Tasks Madeline Brumley Joe Kwon David M. Krueger Dmitrii Krasheninnikov Usman Anwar LLMSV 39 6 0 11 Nov 2024
Towards Universality: Studying Mechanistic Similarity Across Language Model Architectures Junxuan Wang Xuyang Ge Wentao Shu Qiong Tang Yunhua Zhou Zhengfu He Xipeng Qiu 29 7 0 09 Oct 2024
Towards Interpreting Visual Information Processing in Vision-Language Models Clement Neo Luke Ong Philip H. S. Torr Mor Geva David M. Krueger Fazl Barez 86 6 0 09 Oct 2024
Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations Nick Jiang Anish Kachinthaya Suzie Petryk Yossi Gandelsman VLM 34 15 0 03 Oct 2024
Quantifying and Enabling the Interpretability of CLIP-like Models Avinash Madasu Yossi Gandelsman Vasudev Lal Phillip Howard VLM 48 2 0 10 Sep 2024
Decomposing and Editing Predictions by Modeling Model Computation Harshay Shah Andrew Ilyas A. Madry KELM 34 14 0 17 Apr 2024
Toy Models of Superposition Nelson Elhage Tristan Hume Catherine Olsson Nicholas Schiefer T. Henighan ... Sam McCandlish Jared Kaplan Dario Amodei Martin Wattenberg C. Olah AAML MILM 125 318 0 21 Sep 2022
Natural Language Descriptions of Deep Visual Features Evan Hernandez Sarah Schwettmann David Bau Teona Bagashvili Antonio Torralba Jacob Andreas MILM 201 117 0 26 Jan 2022
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 255 4,781 0 24 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 298 3,700 0 11 Feb 2021