Formal Language Recognition by Hard Attention Transformers: Perspectives from Circuit Complexity

13 April 2022

Papers citing "Formal Language Recognition by Hard Attention Transformers: Perspectives from Circuit Complexity"

14 / 14 papers shown

Title
Exact Expressive Power of Transformers with Padding William Merrill Ashish Sabharwal 77 0 0 25 May 2025
Lower Bounds for Chain-of-Thought Reasoning in Hard-Attention Transformers Alireza Amiri Xinting Huang Mark Rofin Michael Hahn LRM 489 1 0 04 Feb 2025
Fundamental Limitations on Subquadratic Alternatives to Transformers Josh Alman Hantao Yu 72 3 0 05 Oct 2024
Representing Rule-based Chatbots with Transformers Dan Friedman Abhishek Panigrahi Danqi Chen 117 1 0 15 Jul 2024
Logicbreaks: A Framework for Understanding Subversion of Rule-based Inference Anton Xue Avishree Khare Rajeev Alur Surbhi Goel Eric Wong 99 2 0 21 Jun 2024
On the Representational Capacity of Neural Language Models with Chain-of-Thought Reasoning Franz Nowak Anej Svete Alexandra Butoi Ryan Cotterell ReLM LRM 80 15 0 20 Jun 2024
Investigating Recurrent Transformers with Dynamic Halt Jishnu Ray Chowdhury Cornelia Caragea 101 1 0 01 Feb 2024
Saturated Transformers are Constant-Depth Threshold Circuits William Merrill Ashish Sabharwal Noah A. Smith 78 105 0 30 Jun 2021
Self-Attention Networks Can Process Bounded Hierarchical Languages Shunyu Yao Binghui Peng Christos H. Papadimitriou Karthik Narasimhan 58 82 0 24 May 2021
Are Transformers universal approximators of sequence-to-sequence functions? Chulhee Yun Srinadh Bhojanapalli A. S. Rawat Sashank J. Reddi Sanjiv Kumar 110 355 0 20 Dec 2019
Theoretical Limitations of Self-Attention in Neural Sequence Models Michael Hahn 68 271 0 16 Jun 2019
On the Turing Completeness of Modern Neural Network Architectures Jorge A. Pérez Javier Marinkovic Pablo Barceló BDL 65 145 0 10 Jan 2019
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 701 131,652 0 12 Jun 2017
Effective Approaches to Attention-based Neural Machine Translation Thang Luong Hieu H. Pham Christopher D. Manning 382 7,962 0 17 Aug 2015