On the Practical Computational Power of Finite Precision RNNs for Language Recognition

13 May 2018

Papers citing "On the Practical Computational Power of Finite Precision RNNs for Language Recognition"

50 / 56 papers shown

Title
NoPE: The Counting Power of Transformers with No Positional Encodings Chris Köcher Alexander Kozachinskiy Anthony Widjaja Lin Marco Sälzer Georg Zetzsche 12 0 0 16 May 2025
Training Neural Networks as Recognizers of Formal Languages Alexandra Butoi Ghazal Khalighinejad Anej Svete Josef Valvoda Ryan Cotterell Brian DuSell NAI 44 2 0 11 Nov 2024
Can Transformers Learn $n$ -gram Language Models? Anej Svete Nadav Borenstein M. Zhou Isabelle Augenstein Ryan Cotterell 47 7 0 03 Oct 2024
On the Representational Capacity of Neural Language Models with Chain-of-Thought Reasoning Franz Nowak Anej Svete Alexandra Butoi Ryan Cotterell ReLM LRM 54 13 0 20 Jun 2024
A Tensor Decomposition Perspective on Second-order RNNs M. Lizaire Michael Rizvi-Martel Marawan Gamal Abdel Hameed Guillaume Rabusseau 55 0 0 07 Jun 2024
What Languages are Easy to Language-Model? A Perspective from Learning Probabilistic Regular Languages Nadav Borenstein Anej Svete R. Chan Josef Valvoda Franz Nowak Isabelle Augenstein Eleanor Chodroff Ryan Cotterell 42 12 0 06 Jun 2024
Limits of Deep Learning: Sequence Modeling through the Lens of Complexity Theory Nikola Zubić Federico Soldá Aurelio Sulser Davide Scaramuzza LRM BDL 52 5 0 26 May 2024
Inducing Systematicity in Transformers by Attending to Structurally Quantized Embeddings Yichen Jiang Xiang Zhou Mohit Bansal 35 1 0 09 Feb 2024
Positional Encoding Helps Recurrent Neural Networks Handle a Large Vocabulary Takashi Morita 24 3 0 31 Jan 2024
Recurrent Neural Language Models as Probabilistic Finite-state Automata Anej Svete Ryan Cotterell 42 2 0 08 Oct 2023
Human Behavioral Benchmarking: Numeric Magnitude Comparison Effects in Large Language Models Raj Sanjay Shah Vijay Marupudi Reba Koenen Khushi Bhardwaj Sashank Varma 27 6 0 18 May 2023
Empirical Analysis of the Inductive Bias of Recurrent Neural Networks by Discrete Fourier Transform of Output Sequences Taiga Ishii Ryo Ueda Yusuke Miyao 24 0 0 16 May 2023
Theoretical Conditions and Empirical Failure of Bracket Counting on Long Sequences with Linear Recurrent Networks Nadine El-Naggar Pranava Madhyastha Tillman Weyde 22 1 0 07 Apr 2023
Exposing the Functionalities of Neurons for Gated Recurrent Unit Based Sequence-to-Sequence Model Yi-Ting Lee Da-Yi Wu Chih-Chun Yang Shou-De Lin MILM 24 0 0 27 Mar 2023
Ordered Memory Baselines Daniel Borisov Matthew D’Iorio Jeffrey Hyacinthe 13 0 0 08 Feb 2023
Memory Augmented Large Language Models are Computationally Universal Dale Schuurmans 35 45 0 10 Jan 2023
Exploring the Long-Term Generalization of Counting Behavior in RNNs Nadine El-Naggar Pranava Madhyastha Tillman Weyde 21 5 0 29 Nov 2022
Simplicity Bias in Transformers and their Ability to Learn Sparse Boolean Functions S. Bhattamishra Arkil Patel Varun Kanade Phil Blunsom 22 45 0 22 Nov 2022
A Short Survey of Systematic Generalization Yuanpeng Li AI4CE 43 1 0 22 Nov 2022
Benchmarking Compositionality with Formal Languages Josef Valvoda Naomi Saphra Jonathan Rawski Adina Williams Ryan Cotterell NAI CoGe 38 8 0 17 Aug 2022
Neural Networks and the Chomsky Hierarchy Grégoire Delétang Anian Ruoss Jordi Grau-Moya Tim Genewein L. Wenliang ... Chris Cundy Marcus Hutter Shane Legg Joel Veness Pedro A. Ortega UQCV 107 131 0 05 Jul 2022
Extracting Finite Automata from RNNs Using State Merging William Merrill Nikolaos Tsilivis 22 14 0 28 Jan 2022
Minimum Description Length Recurrent Neural Networks Nur Lan Michal Geyer Emmanuel Chemla Roni Katzir 21 13 0 31 Oct 2021
A Survey on Deep Learning for Skeleton-Based Human Animation Lucas Mourot Ludovic Hoyet F. Clerc Francois Schnitzler Pierre Hellier 3DH 34 77 0 13 Oct 2021
Inducing Transformer's Compositional Generalization Ability via Auxiliary Sequence Prediction Tasks Yichen Jiang Joey Tianyi Zhou 100 25 0 30 Sep 2021
Deep Sequence Modeling: Development and Applications in Asset Pricing Lingbo Cong Ke Tang Jingyuan Wang Yang Zhang 24 15 0 20 Aug 2021
Thinking Like Transformers Gail Weiss Yoav Goldberg Eran Yahav AI4CE 35 128 0 13 Jun 2021
Going Beyond Linear Transformers with Recurrent Fast Weight Programmers Kazuki Irie Imanol Schlag Róbert Csordás Jürgen Schmidhuber 33 57 0 11 Jun 2021
The Limitations of Limited Context for Constituency Parsing Yuchen Li Andrej Risteski 26 4 0 03 Jun 2021
How Attentive are Graph Attention Networks? Shaked Brody Uri Alon Eran Yahav GNN 60 1,019 0 30 May 2021
Formal Language Theory Meets Modern NLP William Merrill AI4CE NAI 21 12 0 19 Feb 2021
Exploring Transitivity in Neural NLI Models through Veridicality Hitomi Yanaka K. Mineshima Kentaro Inui 25 23 0 26 Jan 2021
Can RNNs learn Recursive Nested Subject-Verb Agreements? Yair Lakretz T. Desbordes J. King Benoît Crabbé Maxime Oquab S. Dehaene 160 19 0 06 Jan 2021
Advances in Electron Microscopy with Deep Learning Jeffrey M. Ede 35 2 0 04 Jan 2021
Language Inference with Multi-head Automata through Reinforcement Learning Alper Şekerci Özlem Salehi AI4CE 19 0 0 20 Oct 2020
Learning to Recombine and Resample Data for Compositional Generalization Ekin Akyürek Afra Feyza Akyürek Jacob Andreas 29 79 0 08 Oct 2020
How LSTM Encodes Syntax: Exploring Context Vectors and Semi-Quantization on Natural Text Chihiro Shibata Kei Uchiumi D. Mochihashi 19 7 0 01 Oct 2020
Distillation of Weighted Automata from Recurrent Neural Networks using a Spectral Approach Rémi Eyraud Stéphane Ayache 24 16 0 28 Sep 2020
Review: Deep Learning in Electron Microscopy Jeffrey M. Ede 36 79 0 17 Sep 2020
On the Computational Power of Transformers and its Implications in Sequence Modeling S. Bhattamishra Arkil Patel Navin Goyal 33 65 0 16 Jun 2020
Adaptive Partial Scanning Transmission Electron Microscopy with Reinforcement Learning Jeffrey M. Ede 24 12 0 06 Apr 2020
It's Not What Machines Can Learn, It's What We Cannot Teach Gal Yehuda Moshe Gabel Assaf Schuster FaML 14 37 0 21 Feb 2020
A Data Efficient End-To-End Spoken Language Understanding Architecture Marco Dinarelli Nikita Kapoor Bassam Jabaian Laurent Besacier 3DV 17 20 0 14 Feb 2020
Does syntax need to grow on trees? Sources of hierarchical inductive bias in sequence-to-sequence networks R. Thomas McCoy Robert Frank Tal Linzen 25 106 0 10 Jan 2020
Location Attention for Extrapolation to Longer Sequences Yann Dubois Gautier Dagan Dieuwke Hupkes Elia Bruni 23 40 0 10 Nov 2019
Memory-Augmented Recurrent Neural Networks Can Learn Generalized Dyck Languages Mirac Suzgun Sebastian Gehrmann Yonatan Belinkov Stuart M. Shieber 29 50 0 08 Nov 2019
Sequential Neural Networks as Automata William Merrill 23 74 0 04 Jun 2019
Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain) Mariya Toneva Leila Wehbe MILM AI4CE 42 220 0 28 May 2019
Weighted Automata Extraction from Recurrent Neural Networks via Regression on State Spaces Takamasa Okudono Masaki Waga Taro Sekiyama I. Hasuo 11 37 0 05 Apr 2019
GuacaMol: Benchmarking Models for De Novo Molecular Design Nathan Brown Marco Fiscato Marwin H. S. Segler Alain C. Vaucher ELM 44 693 0 22 Nov 2018