A Logic for Expressing Log-Precision Transformers

A Logic for Expressing Log-Precision Transformers

6 October 2022

William Merrill

Ashish Sabharwal

Papers citing "A Logic for Expressing Log-Precision Transformers"

15 / 15 papers shown

Title
Tracr-Injection: Distilling Algorithms into Pre-trained Language Models Tomás Vergara-Browne Álvaro Soto 17 0 0 15 May 2025
Language Models, Graph Searching, and Supervision Adulteration: When More Supervision is Less and How to Make More More Arvid Frydenlund LRM 55 0 0 13 Mar 2025
A Little Depth Goes a Long Way: The Expressive Power of Log-Depth Transformers William Merrill Ashish Sabharwal 55 5 0 05 Mar 2025
Finite State Automata Inside Transformers with Chain-of-Thought: A Mechanistic Study on State Tracking Yifan Zhang Wenyu Du Dongming Jin Jie Fu Zhi Jin LRM 53 0 0 27 Feb 2025
Lower Bounds for Chain-of-Thought Reasoning in Hard-Attention Transformers Alireza Amiri Xinting Huang Mark Rofin Michael Hahn LRM 213 0 0 04 Feb 2025
Circuit Complexity Bounds for Visual Autoregressive Model Yekun Ke Xiaoyu Li Yingyu Liang Zhenmei Shi Zhao Song 45 5 0 08 Jan 2025
ENTP: Encoder-only Next Token Prediction Ethan Ewer Daewon Chae Thomas Zeng Jinkyu Kim Kangwook Lee 38 3 0 02 Oct 2024
Representing Rule-based Chatbots with Transformers Dan Friedman Abhishek Panigrahi Danqi Chen 71 1 0 15 Jul 2024
Separations in the Representational Capabilities of Transformers and Recurrent Architectures S. Bhattamishra Michael Hahn Phil Blunsom Varun Kanade GNN 44 9 0 13 Jun 2024
The Illusion of State in State-Space Models William Merrill Jackson Petty Ashish Sabharwal 54 44 0 12 Apr 2024
Counting Like Transformers: Compiling Temporal Counting Logic Into Softmax Transformers Andy Yang David Chiang 41 8 0 05 Apr 2024
Transformers as Transducers Lena Strobl Dana Angluin David Chiang Jonathan Rawski Ashish Sabharwal 31 5 0 02 Apr 2024
Chain of Thought Empowers Transformers to Solve Inherently Serial Problems Zhiyuan Li Hong Liu Denny Zhou Tengyu Ma LRM AI4CE 30 101 0 20 Feb 2024
On The Expressivity of Recurrent Neural Cascades Nadezda A. Knorozova Alessandro Ronca 23 1 0 14 Dec 2023
Faith and Fate: Limits of Transformers on Compositionality Nouha Dziri Ximing Lu Melanie Sclar Xiang Lorraine Li Liwei Jian ... Sean Welleck Xiang Ren Allyson Ettinger Zaïd Harchaoui Yejin Choi ReLM LRM 30 334 0 29 May 2023