v1v2v3 (latest)

Reasoning Circuits in Language Models: A Mechanistic Interpretation of Syllogistic Inference

16 August 2024

Papers citing "Reasoning Circuits in Language Models: A Mechanistic Interpretation of Syllogistic Inference"

32 / 32 papers shown

Title
Signatures of human-like processing in Transformer forward passes Jennifer Hu Michael A. Lepori Michael Franke AI4CE 429 0 0 18 Apr 2025
Quantifying Logical Consistency in Transformers via Query-Key Alignment Eduard Tulchinskii Anastasia Voznyuk Laida Kushnareva Andrei Andriiainen Irina Piontkovskaya Evgeny Burnaev Serguei Barannikov LRM 143 0 0 24 Feb 2025
Understanding Synthetic Context Extension via Retrieval Heads Xinyu Zhao Fangcong Yin Greg Durrett 133 2 0 29 Oct 2024
Attention Heads of Large Language Models: A Survey Zifan Zheng Yezhaohui Wang Yuxin Huang Shichao Song Mingchuan Yang Bo Tang Feiyu Xiong Zhiyu Li LRM 114 28 0 05 Sep 2024
Dual Process Learning: Controlling Use of In-Context vs. In-Weights Strategies with Weight Forgetting Suraj Anand Michael A. Lepori Jack Merullo Ellie Pavlick CLL 105 8 0 28 May 2024
How does GPT-2 Predict Acronyms? Extracting and Understanding a Circuit via Mechanistic Interpretability Jorge García-Carrasco Alejandro Maté Juan Trujillo 58 10 0 07 May 2024
How to use and interpret activation patching Stefan Heimersheim Neel Nanda 80 48 0 23 Apr 2024
Estimating the Causal Effects of Natural Logic Features in Transformer-Based NLI Models Julia Rozanova Marco Valentino André Freitas CML 61 1 0 03 Apr 2024
A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepedia Giovanni Monea Maxime Peyrard Martin Josifoski Vishrav Chaudhary Jason Eisner Emre Kiciman Hamid Palangi Barun Patra Robert West KELM 137 15 0 04 Dec 2023
Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks Zhaofeng Wu Linlu Qiu Alexis Ross Ekin Akyürek Boyuan Chen Bailin Wang Najoung Kim Jacob Andreas Yoon Kim LRM ReLM 170 223 0 05 Jul 2023
Evaluating Large Language Models with NeuBAROCO: Syllogistic Reasoning Ability and Human-like Biases Risako Ando Takanobu Morishita Hirohiko Abe K. Mineshima Mitsuhiro Okada LRM ELM 95 13 0 21 Jun 2023
Dissecting Recall of Factual Associations in Auto-Regressive Language Models Mor Geva Jasmijn Bastings Katja Filippova Amir Globerson KELM 274 324 0 28 Apr 2023
Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling Stella Biderman Hailey Schoelkopf Quentin G. Anthony Herbie Bradley Kyle O'Brien ... USVSN Sai Prashanth Edward Raff Aviya Skowron Lintang Sutawika Oskar van der Wal 107 1,307 0 03 Apr 2023
Progress measures for grokking via mechanistic interpretability Neel Nanda Lawrence Chan Tom Lieberum Jess Smith Jacob Steinhardt 90 450 0 12 Jan 2023
Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small Kevin Wang Alexandre Variengien Arthur Conmy Buck Shlegeris Jacob Steinhardt 316 563 0 01 Nov 2022
Understanding Transformer Memorization Recall Through Idioms Adi Haviv Ido Cohen Jacob Gidron R. Schuster Yoav Goldberg Mor Geva 96 53 0 07 Oct 2022
In-context Learning and Induction Heads Catherine Olsson Nelson Elhage Neel Nanda Nicholas Joseph Nova Dassarma ... Tom B. Brown Jack Clark Jared Kaplan Sam McCandlish C. Olah 323 528 0 24 Sep 2022
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 535 4,077 0 24 May 2022
Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space Mor Geva Avi Caciularu Ke Wang Yoav Goldberg KELM 130 387 0 28 Mar 2022
Locating and Editing Factual Associations in GPT Kevin Meng David Bau A. Andonian Yonatan Belinkov KELM 251 1,389 0 10 Feb 2022
Scaling Language Models: Methods, Analysis & Insights from Training Gopher Jack W. Rae Sebastian Borgeaud Trevor Cai Katie Millican Jordan Hoffmann ... Jeff Stanway L. Bennett Demis Hassabis Koray Kavukcuoglu G. Irving 138 1,325 0 08 Dec 2021
BERTnesia: Investigating the capture and forgetting of knowledge in BERT Jonas Wallat Jaspreet Singh Avishek Anand CLL KELM 127 60 0 05 Jun 2021
Critical Thinking for Language Models Gregor Betz Christian Voigt Kyle Richardson SyDa ReLM LRM AI4CE 103 35 0 15 Sep 2020
Are Pretrained Language Models Symbolic Reasoners Over Knowledge? Nora Kassner Benno Kroje Hinrich Schütze ReLM LRM 29 5 0 18 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 901 42,463 0 28 May 2020
GenericsKB: A Knowledge Base of Generic Statements Sumithra Bhakthavatsalam Chloe Anastasiades Peter Clark 45 70 0 02 May 2020
Transformers as Soft Reasoners over Language Peter Clark Oyvind Tafjord Kyle Richardson ReLM OffRL LRM 109 361 0 14 Feb 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 651 4,921 0 23 Jan 2020
oLMpics -- On what Language Model Pre-training Captures Alon Talmor Yanai Elazar Yoav Goldberg Jonathan Berant LRM 103 304 0 31 Dec 2019
Does Learning Require Memorization? A Short Tale about a Long Tail Vitaly Feldman TDI 142 502 0 12 Jun 2019
Similarity of Neural Network Representations Revisited Simon Kornblith Mohammad Norouzi Honglak Lee Geoffrey E. Hinton 143 1,435 0 01 May 2019
Convergent Learning: Do different neural networks learn the same representations? Yixuan Li J. Yosinski Jeff Clune Hod Lipson John E. Hopcroft SSL 98 371 0 24 Nov 2015