A Formal Hierarchy of RNN Architectures

18 April 2020

Papers citing "A Formal Hierarchy of RNN Architectures"

22 / 22 papers shown

Title
NoPE: The Counting Power of Transformers with No Positional Encodings Chris Köcher Alexander Kozachinskiy Anthony Widjaja Lin Marco Sälzer Georg Zetzsche 12 0 0 16 May 2025
Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues Riccardo Grazzi Julien N. Siems Jörg Franke Arber Zela Frank Hutter Massimiliano Pontil 96 11 0 19 Nov 2024
Training Neural Networks as Recognizers of Formal Languages Alexandra Butoi Ghazal Khalighinejad Anej Svete Josef Valvoda Ryan Cotterell Brian DuSell NAI 44 2 0 11 Nov 2024
On the Representational Capacity of Neural Language Models with Chain-of-Thought Reasoning Franz Nowak Anej Svete Alexandra Butoi Ryan Cotterell ReLM LRM 54 13 0 20 Jun 2024
A Tensor Decomposition Perspective on Second-order RNNs M. Lizaire Michael Rizvi-Martel Marawan Gamal Abdel Hameed Guillaume Rabusseau 55 0 0 07 Jun 2024
What Languages are Easy to Language-Model? A Perspective from Learning Probabilistic Regular Languages Nadav Borenstein Anej Svete R. Chan Josef Valvoda Franz Nowak Isabelle Augenstein Eleanor Chodroff Ryan Cotterell 42 12 0 06 Jun 2024
On The Expressivity of Recurrent Neural Cascades Nadezda A. Knorozova Alessandro Ronca 23 1 0 14 Dec 2023
Recurrent Neural Language Models as Probabilistic Finite-state Automata Anej Svete Ryan Cotterell 42 2 0 08 Oct 2023
DDNAS: Discretized Differentiable Neural Architecture Search for Text Classification Kuan-Yu Chen Cheng Li Kuo-Jung Lee 28 1 0 12 Jul 2023
Empirical Analysis of the Inductive Bias of Recurrent Neural Networks by Discrete Fourier Transform of Output Sequences Taiga Ishii Ryo Ueda Yusuke Miyao 24 0 0 16 May 2023
Modelling Concurrency Bugs Using Machine Learning Teodor Rares Begu 18 0 0 08 May 2023
Theoretical Conditions and Empirical Failure of Bracket Counting on Long Sequences with Linear Recurrent Networks Nadine El-Naggar Pranava Madhyastha Tillman Weyde 22 1 0 07 Apr 2023
Exploring the Long-Term Generalization of Counting Behavior in RNNs Nadine El-Naggar Pranava Madhyastha Tillman Weyde 21 5 0 29 Nov 2022
Simplicity Bias in Transformers and their Ability to Learn Sparse Boolean Functions S. Bhattamishra Arkil Patel Varun Kanade Phil Blunsom 22 46 0 22 Nov 2022
Memory-Augmented Graph Neural Networks: A Brain-Inspired Review Guixiang Ma Vy A. Vo Ted Willke Nesreen Ahmed 40 1 0 22 Sep 2022
Neural Networks and the Chomsky Hierarchy Grégoire Delétang Anian Ruoss Jordi Grau-Moya Tim Genewein L. Wenliang ... Chris Cundy Marcus Hutter Shane Legg Joel Veness Pedro A. Ortega UQCV 109 133 0 05 Jul 2022
Extracting Finite Automata from RNNs Using State Merging William Merrill Nikolaos Tsilivis 22 14 0 28 Jan 2022
Thinking Like Transformers Gail Weiss Yoav Goldberg Eran Yahav AI4CE 35 128 0 13 Jun 2021
Formal Language Theory Meets Modern NLP William Merrill AI4CE NAI 21 12 0 19 Feb 2021
Distillation of Weighted Automata from Recurrent Neural Networks using a Spectral Approach Rémi Eyraud Stéphane Ayache 24 16 0 28 Sep 2020
On the Computational Power of Transformers and its Implications in Sequence Modeling S. Bhattamishra Arkil Patel Navin Goyal 33 66 0 16 Jun 2020
Convolutional Neural Networks for Sentence Classification Yoon Kim AILaw VLM 312 13,377 0 25 Aug 2014