Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions

23 February 2024

Papers citing "Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions"

3 / 3 papers shown

Title
Quantifying Feature Space Universality Across Large Language Models via Sparse Autoencoders Michael Lan Philip Torr Austin Meek Ashkan Khakzar David M. Krueger Fazl Barez 43 11 0 09 Oct 2024
In-context Learning and Induction Heads Catherine Olsson Nelson Elhage Neel Nanda Nicholas Joseph Nova Dassarma ... Tom B. Brown Jack Clark Jared Kaplan Sam McCandlish C. Olah 252 474 0 24 Sep 2022
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 282 2,000 0 31 Dec 2020