On the Ability and Limitations of Transformers to Recognize Formal Languages

23 September 2020

Papers citing "On the Ability and Limitations of Transformers to Recognize Formal Languages"

9 / 9 papers shown

Title
Choppy: Cut Transformer For Ranked List Truncation Dara Bahri Yi Tay Che Zheng Donald Metzler Andrew Tomkins 24 29 0 26 Apr 2020
TraDE: Transformers for Density Estimation Rasool Fakoor Pratik Chaudhari Jonas W. Mueller Alex Smola 52 30 0 06 Apr 2020
Learning Deterministic Weighted Automata with Queries and Counterexamples Gail Weiss Yoav Goldberg Eran Yahav TPM 48 45 0 30 Oct 2019
Transformer Dissection: A Unified Understanding of Transformer's Attention via the Lens of Kernel Yao-Hung Hubert Tsai Shaojie Bai M. Yamada Louis-Philippe Morency Ruslan Salakhutdinov 87 251 0 30 Aug 2019
Theoretical Limitations of Self-Attention in Neural Sequence Models Michael Hahn 39 266 0 16 Jun 2019
Assessing the Ability of Self-Attention Networks to Learn Word Order Baosong Yang Longyue Wang Derek F. Wong Lidia S. Chao Zhaopeng Tu 23 32 0 03 Jun 2019
Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned Elena Voita David Talbot F. Moiseev Rico Sennrich Ivan Titov 59 1,120 0 23 May 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 107 3,707 0 09 Jan 2019
On Evaluating the Generalization of LSTM Models in Formal Languages Mirac Suzgun Yonatan Belinkov Stuart M. Shieber AI4CE 18 41 0 02 Nov 2018