A is for Absorption: Studying Feature Splitting and Absorption in Sparse
Autoencoders

A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders

22 September 2024

James Wilken-Smith

Hardik Bhatnagar

Papers citing "A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders"

16 / 16 papers shown

Title
Probing the Vulnerability of Large Language Models to Polysemantic Interventions Bofan Gong Shiyang Lai Dawn Song AAML MILM 4 0 0 16 May 2025
Feature Hedging: Correlated Features Break Narrow Sparse Autoencoders David Chanin Tomáš Dulka Adrià Garriga-Alonso 4 0 0 16 May 2025
Are Sparse Autoencoders Useful for Java Function Bug Detection? Rui Melo Claudia Mamede Andre Catarino Rui Abreu Henrique Lopes Cardoso 31 0 0 15 May 2025
Unveiling Language-Specific Features in Large Language Models via Sparse Autoencoders Boyi Deng Boyi Deng Yidan Zhang Baosong Yang Fuli Feng 41 0 0 08 May 2025
Representation Learning on a Random Lattice Aryeh Brill OOD FAtt AI4CE 73 0 0 28 Apr 2025
Learning Multi-Level Features with Matryoshka Sparse Autoencoders Bart Bussmann Noa Nabeshima Adam Karvonen Neel Nanda 59 1 0 21 Mar 2025
SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability Adam Karvonen Can Rager Johnny Lin Curt Tigges Joseph Isaac Bloom ... Kola Ayonrinde Matthew Wearden Arthur Conmy Samuel Marks Neel Nanda MU 64 8 0 12 Mar 2025
Do Sparse Autoencoders Generalize? A Case Study of Answerability Lovis Heindrich Philip Torr Fazl Barez Veronika Thost 79 1 0 27 Feb 2025
Interpreting CLIP with Hierarchical Sparse Autoencoders Vladimir Zaigrajew Hubert Baniecki P. Biecek 56 0 0 27 Feb 2025
Jacobian Sparse Autoencoders: Sparsify Computations, Not Just Activations Lucy Farnik Tim Lawson Conor Houghton Laurence Aitchison 61 0 0 25 Feb 2025
Are Sparse Autoencoders Useful? A Case Study in Sparse Probing Subhash Kantamneni Joshua Engels Senthooran Rajamanoharan Max Tegmark Neel Nanda 61 6 0 23 Feb 2025
Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignment Harrish Thasarathan Julian Forsyth Thomas Fel M. Kowal Konstantinos G. Derpanis 111 7 0 06 Feb 2025
Steering Language Model Refusal with Sparse Autoencoders Kyle O'Brien David Majercak Xavier Fernandes Richard Edgar Jingya Chen Harsha Nori Dean Carignan Eric Horvitz Forough Poursabzi-Sangde LLMSV 67 10 0 18 Nov 2024
Sparse Autoencoders Reveal Universal Feature Spaces Across Large Language Models Michael Lan Philip Torr Austin Meek Ashkan Khakzar David M. Krueger Fazl Barez 43 11 0 09 Oct 2024
An X-Ray Is Worth 15 Features: Sparse Autoencoders for Interpretable Radiology Report Generation Ahmed Abdulaal Hugo Fry Nina Montaña-Brown Ayodeji Ijishakin Jack Gao Stephanie L. Hyland Daniel C. Alexander Daniel Coelho De Castro MedIm 39 8 0 04 Oct 2024
A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models Daking Rai Yilun Zhou Shi Feng Abulhair Saparov Ziyu Yao 82 19 0 02 Jul 2024