On the Computational Power of Transformers and its Implications in Sequence Modeling

16 June 2020

Papers citing "On the Computational Power of Transformers and its Implications in Sequence Modeling"

19 / 19 papers shown

Title
Can Large Language Models Learn Formal Logic? A Data-Driven Training and Evaluation Framework Yuan Xia Akanksha Atrey Fadoua Khmaissia Kedar S. Namjoshi LRM ELM 45 0 0 28 Apr 2025
Lower Bounds for Chain-of-Thought Reasoning in Hard-Attention Transformers Alireza Amiri Xinting Huang Mark Rofin Michael Hahn LRM 180 0 0 04 Feb 2025
Training Neural Networks as Recognizers of Formal Languages Alexandra Butoi Ghazal Khalighinejad Anej Svete Josef Valvoda Ryan Cotterell Brian DuSell NAI 44 2 0 11 Nov 2024
Representing Rule-based Chatbots with Transformers Dan Friedman Abhishek Panigrahi Danqi Chen 66 1 0 15 Jul 2024
On the Representational Capacity of Neural Language Models with Chain-of-Thought Reasoning Franz Nowak Anej Svete Alexandra Butoi Ryan Cotterell ReLM LRM 54 12 0 20 Jun 2024
Separations in the Representational Capabilities of Transformers and Recurrent Architectures S. Bhattamishra Michael Hahn Phil Blunsom Varun Kanade GNN 44 9 0 13 Jun 2024
Limits of Deep Learning: Sequence Modeling through the Lens of Complexity Theory Nikola Zubić Federico Soldá Aurelio Sulser Davide Scaramuzza LRM BDL 52 5 0 26 May 2024
Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining Licong Lin Yu Bai Song Mei OffRL 32 43 0 12 Oct 2023
Transformers in Reinforcement Learning: A Survey Pranav Agarwal A. Rahman P. St-Charles Simon J. D. Prince Samira Ebrahimi Kahou OffRL 24 18 0 12 Jul 2023
A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity Hongkang Li Hao Wu Sijia Liu Pin-Yu Chen ViT MLT 37 57 0 12 Feb 2023
Memory Augmented Large Language Models are Computationally Universal Dale Schuurmans 35 45 0 10 Jan 2023
Recurrent Convolutional Neural Networks Learn Succinct Learning Algorithms Surbhi Goel Sham Kakade Adam Tauman Kalai Cyril Zhang 34 1 0 01 Sep 2022
What Can Transformers Learn In-Context? A Case Study of Simple Function Classes Shivam Garg Dimitris Tsipras Percy Liang Gregory Valiant 26 449 0 01 Aug 2022
Pure Transformers are Powerful Graph Learners Jinwoo Kim Tien Dat Nguyen Seonwoo Min Sungjun Cho Moontae Lee Honglak Lee Seunghoon Hong 43 189 0 06 Jul 2022
Overcoming a Theoretical Limitation of Self-Attention David Chiang Peter A. Cholak 30 78 0 24 Feb 2022
Inductive Biases and Variable Creation in Self-Attention Mechanisms Benjamin L. Edelman Surbhi Goel Sham Kakade Cyril Zhang 27 116 0 19 Oct 2021
On the Expressive Power of Self-Attention Matrices Valerii Likhosherstov K. Choromanski Adrian Weller 37 34 0 07 Jun 2021
Representing Numbers in NLP: a Survey and a Vision Avijit Thawani Jay Pujara Pedro A. Szekely Filip Ilievski 32 114 0 24 Mar 2021
OpenNMT: Open-Source Toolkit for Neural Machine Translation Guillaume Klein Yoon Kim Yuntian Deng Jean Senellart Alexander M. Rush 271 1,896 0 10 Jan 2017