Transformers represent belief state geometry in their residual stream

24 May 2024

Papers citing "Transformers represent belief state geometry in their residual stream"

7 / 7 papers shown

Title
Learning curves theory for hierarchically compositional data with power-law distributed features Francesco Cagnetta Hyunmo Kang M. Wyart 36 0 0 11 May 2025
Scaling Laws and Representation Learning in Simple Hierarchical Languages: Transformers vs. Convolutional Architectures Francesco Cagnetta Alessandro Favero Antonio Sclocchi M. Wyart 26 0 0 11 May 2025
Evaluating Explanations: An Explanatory Virtues Framework for Mechanistic Interpretability -- The Strange Science Part I.ii Kola Ayonrinde Louis Jaburi XAI 83 1 0 02 May 2025
Modes of Sequence Models and Learning Coefficients Zhongtian Chen Daniel Murfet 87 1 0 25 Apr 2025
Constrained belief updates explain geometric structures in transformer representations Mateusz Piotrowski P. Riechers Daniel Filan A. Shai 76 0 0 04 Feb 2025
ICLR: In-Context Learning of Representations Core Francisco Park Andrew Lee Ekdeep Singh Lubana Yongyi Yang Maya Okawa Kento Nishi Martin Wattenberg Hidenori Tanaka AIFin 118 3 0 29 Dec 2024
Programming Refusal with Conditional Activation Steering Bruce W. Lee Inkit Padhi K. Ramamurthy Erik Miehling Pierre L. Dognin Manish Nagireddy Amit Dhurandhar LLMSV 105 13 0 06 Sep 2024