Iteration Head: A Mechanistic Study of Chain-of-Thought

Iteration Head: A Mechanistic Study of Chain-of-Thought

4 June 2024

Vivien A. Cabannes

Francois Charton

Papers citing "Iteration Head: A Mechanistic Study of Chain-of-Thought"

13 / 13 papers shown

Title
Rethinking Agent Design: From Top-Down Workflows to Bottom-Up Skill Evolution Jiawei Du Jinlong Wu Yuzheng Chen Yucheng Hu Bing Li Joey Tianyi Zhou 104 0 0 23 May 2025
Lower Bounds for Chain-of-Thought Reasoning in Hard-Attention Transformers Alireza Amiri Xinting Huang Mark Rofin Michael Hahn LRM 434 1 0 04 Feb 2025
Understanding and Mitigating Gender Bias in LLMs via Interpretable Neuron Editing Zeping Yu Sophia Ananiadou KELM 82 3 0 24 Jan 2025
The mechanistic basis of data dependence and abrupt learning in an in-context classification task Gautam Reddy 61 59 0 03 Dec 2023
Think before you speak: Training Language Models With Pause Tokens Sachin Goyal Ziwei Ji A. S. Rawat A. Menon Sanjiv Kumar Vaishnavh Nagarajan LRM 69 114 0 03 Oct 2023
RWKV: Reinventing RNNs for the Transformer Era Bo Peng Eric Alcaide Quentin G. Anthony Alon Albalak Samuel Arcadinho ... Qihang Zhao P. Zhou Qinghua Zhou Jian Zhu Rui-Jie Zhu 165 585 0 22 May 2023
In-context Learning and Induction Heads Catherine Olsson Nelson Elhage Neel Nanda Nicholas Joseph Nova Dassarma ... Tom B. Brown Jack Clark Jared Kaplan Sam McCandlish C. Olah 296 494 0 24 Sep 2022
What Can Transformers Learn In-Context? A Case Study of Simple Function Classes Shivam Garg Dimitris Tsipras Percy Liang Gregory Valiant 95 479 0 01 Aug 2022
Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit Boaz Barak Benjamin L. Edelman Surbhi Goel Sham Kakade Eran Malach Cyril Zhang 62 128 0 18 Jul 2022
Show Your Work: Scratchpads for Intermediate Computation with Language Models Maxwell Nye Anders Andreassen Guy Gur-Ari Henryk Michalewski Jacob Austin ... Aitor Lewkowycz Maarten Bosma D. Luan Charles Sutton Augustus Odena ReLM LRM 127 728 0 30 Nov 2021
An Explanation of In-context Learning as Implicit Bayesian Inference Sang Michael Xie Aditi Raghunathan Percy Liang Tengyu Ma ReLM BDL VPVLM LRM 139 728 0 03 Nov 2021
Understanding the Role of Individual Units in a Deep Neural Network David Bau Jun-Yan Zhu Hendrik Strobelt Àgata Lapedriza Bolei Zhou Antonio Torralba GAN 42 446 0 10 Sep 2020
Theoretical Limitations of Self-Attention in Neural Sequence Models Michael Hahn 44 267 0 16 Jun 2019