How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model

30 April 2023

Papers citing "How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model"

34 / 34 papers shown

Title
Detection and Mitigation of Hallucination in Large Reasoning Models: A Mechanistic Perspective Zhongxiang Sun Qipeng Wang Haoyu Wang Xiao Zhang Jun Xu HILM LRM 12 0 0 19 May 2025
Distilled Circuits: A Mechanistic Study of Internal Restructuring in Knowledge Distillation Reilly Haskins Benjamin Adams 16 0 0 16 May 2025
Rethinking Circuit Completeness in Language Models: AND, OR, and ADDER Gates Hang Chen Jiaying Zhu Xinyu Yang Wenya Wang LRM 19 0 0 15 May 2025
Understanding In-context Learning of Addition via Activation Subspaces Xinyan Hu Kayo Yin Michael I. Jordan Jacob Steinhardt Lijie Chen 56 0 0 08 May 2025
Quiet Feature Learning in Algorithmic Tasks Prudhviraj Naidu Zixian Wang Leon Bergen R. Paturi VLM 60 0 0 06 May 2025
Towards Understanding the Nature of Attention with Low-Rank Sparse Decomposition Zhengfu He Rongxiang Weng Rui Lin Xuyang Ge Wentao Shu Qiong Tang Jiaming Zhang Xipeng Qiu 70 0 0 29 Apr 2025
Investigating Compositional Reasoning in Time Series Foundation Models Willa Potosnak Cristian Challu Mononito Goswami Kin G. Olivares Michał Wiliński Nina Żukowska Artur Dubrawski ReLM AI4TS LRM 53 0 0 09 Feb 2025
What is a Number, That a Large Language Model May Know It? Raja Marjieh Veniamin Veselovsky Thomas Griffiths Ilia Sucholutsky 239 2 0 03 Feb 2025
Unraveling Arithmetic in Large Language Models: The Role of Algebraic Structures Fu-Chieh Chang Pei-Yuan Wu Pei-Yuan Wu LRM 114 1 0 25 Nov 2024
Understanding Multimodal LLMs: the Mechanistic Interpretability of Llava in Visual Question Answering Zeping Yu Sophia Ananiadou 217 0 0 17 Nov 2024
JailbreakLens: Interpreting Jailbreak Mechanism in the Lens of Representation and Circuit Zeqing He Peng Kuang Zhixuan Chu Huiyu Xu Rui Zheng Kui Ren Chun Chen 62 5 0 17 Nov 2024
Arithmetic Without Algorithms: Language Models Solve Math With a Bag of Heuristics Yaniv Nikankin Anja Reusch Aaron Mueller Yonatan Belinkov AIFin LRM 41 25 0 28 Oct 2024
Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language Models Qitan Lv Jie Wang Hanzhu Chen Bin Li Yongdong Zhang Feng Wu HILM 33 3 0 19 Oct 2024
ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretability Zhongxiang Sun Xiaoxue Zang Kai Zheng Yang Song Jun Xu Xiao Zhang Weijie Yu Yang Song Han Li 57 7 0 15 Oct 2024
Unlearning-based Neural Interpretations Ching Lam Choi Alexandre Duplessis Serge Belongie FAtt 54 0 0 10 Oct 2024
Round and Round We Go! What makes Rotary Positional Encodings useful? Federico Barbero Alex Vitvitskyi Christos Perivolaropoulos Razvan Pascanu Petar Velickovic 85 19 0 08 Oct 2024
Circuit Compositions: Exploring Modular Structures in Transformer-Based Language Models Philipp Mondorf Sondre Wold Yun Xue 41 0 0 02 Oct 2024
From Yes-Men to Truth-Tellers: Addressing Sycophancy in Large Language Models with Pinpoint Tuning Wei Chen Zhen Huang Liang Xie Binbin Lin Houqiang Li ... Deng Cai Yonggang Zhang Wenxiao Wang Xu Shen Jieping Ye 57 6 0 03 Sep 2024
Transformer Circuit Faithfulness Metrics are not Robust Joseph Miller Bilal Chughtai William Saunders 58 7 0 11 Jul 2024
A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models Daking Rai Yilun Zhou Shi Feng Abulhair Saparov Ziyu Yao 85 22 0 02 Jul 2024
Finding Transformer Circuits with Edge Pruning Adithya Bhaskar Alexander Wettig Dan Friedman Danqi Chen 68 17 0 24 Jun 2024
Talking Heads: Understanding Inter-layer Communication in Transformer Language Models Jack Merullo Carsten Eickhoff Ellie Pavlick 58 13 0 13 Jun 2024
Eigenpruning: an Interpretability-Inspired PEFT Method Tomás Vergara-Browne Álvaro Soto A. Aizawa 39 1 0 04 Apr 2024
Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models Samuel Marks Can Rager Eric J. Michaud Yonatan Belinkov David Bau Aaron Mueller 48 122 0 28 Mar 2024
Opening the AI black box: program synthesis via mechanistic interpretability Eric J. Michaud Isaac Liao Vedang Lad Ziming Liu Anish Mudide Chloe Loughridge Zifan Carl Guo Tara Rezaei Kheirkhah Mateja Vukelić Max Tegmark 25 12 0 07 Feb 2024
Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models Asma Ghandeharioun Avi Caciularu Adam Pearce Lucas Dixon Mor Geva 39 90 0 11 Jan 2024
Positional Description Matters for Transformers Arithmetic Ruoqi Shen Sébastien Bubeck Ronen Eldan Yin Tat Lee Yuanzhi Li Yi Zhang 42 38 0 22 Nov 2023
Uncovering Intermediate Variables in Transformers using Circuit Probing Michael A. Lepori Thomas Serre Ellie Pavlick 78 7 0 07 Nov 2023
Identifying and Adapting Transformer-Components Responsible for Gender Bias in an English Language Model Abhijith Chintam Rahel Beloch Willem H. Zuidema Michael Hanna Oskar van der Wal 28 16 0 19 Oct 2023
Interpretable Diffusion via Information Decomposition Xianghao Kong Ollie Liu Han Li Dani Yogatama Greg Ver Steeg 26 21 0 12 Oct 2023
Language Models Represent Space and Time Wes Gurnee Max Tegmark 52 142 0 03 Oct 2023
Towards Best Practices of Activation Patching in Language Models: Metrics and Methods Fred Zhang Neel Nanda LLMSV 38 101 0 27 Sep 2023
Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small Kevin Wang Alexandre Variengien Arthur Conmy Buck Shlegeris Jacob Steinhardt 215 515 0 01 Nov 2022
Probing Classifiers: Promises, Shortcomings, and Advances Yonatan Belinkov 229 409 0 24 Feb 2021