Extracting Finite Automata from RNNs Using State Merging

v1v2v3 (latest)

Extracting Finite Automata from RNNs Using State Merging

28 January 2022

William Merrill

Nikolaos Tsilivis

ArXiv (abs)PDF HTML

Papers citing "Extracting Finite Automata from RNNs Using State Merging"

13 / 13 papers shown

Title
On the Representational Capacity of Neural Language Models with Chain-of-Thought Reasoning Franz Nowak Anej Svete Alexandra Butoi Ryan Cotterell ReLM LRM 101 17 0 20 Jun 2024
What Languages are Easy to Language-Model? A Perspective from Learning Probabilistic Regular Languages Nadav Borenstein Anej Svete R. Chan Josef Valvoda Franz Nowak Isabelle Augenstein Eleanor Chodroff Ryan Cotterell 87 13 0 06 Jun 2024
Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets Alethea Power Yuri Burda Harrison Edwards Igor Babuschkin Vedant Misra 91 365 0 06 Jan 2022
Extracting Weighted Automata for Approximate Minimization in Language Modelling Clara Lacroce Prakash Panangaden Guillaume Rabusseau 67 7 0 05 Jun 2021
Effects of Parameter Norm Growth During Transformer Training: Inductive Bias from Gradient Descent William Merrill Vivek Ramanujan Yoav Goldberg Roy Schwartz Noah A. Smith AI4CE 58 36 0 19 Oct 2020
A Formal Hierarchy of RNN Architectures William Merrill Gail Weiss Yoav Goldberg Roy Schwartz Noah A. Smith Eran Yahav 85 82 0 18 Apr 2020
Sequential Neural Networks as Automata William Merrill 73 78 0 04 Jun 2019
On the Practical Computational Power of Finite Precision RNNs for Language Recognition Gail Weiss Yoav Goldberg Eran Yahav 87 267 0 13 May 2018
Extracting Automata from Recurrent Neural Networks Using Queries and Counterexamples Gail Weiss Yoav Goldberg Eran Yahav 67 187 0 27 Nov 2017
An Empirical Evaluation of Rule Extraction from Recurrent Neural Networks Qinglong Wang Kaixuan Zhang Alexander Ororbia Masashi Sugiyama Xue Liu C. Lee Giles 67 61 0 29 Sep 2017
Opening the Black Box of Deep Neural Networks via Information Ravid Shwartz-Ziv Naftali Tishby AI4CE 115 1,419 0 02 Mar 2017
Visualizing and Understanding Recurrent Networks A. Karpathy Justin Johnson Li Fei-Fei HAI 132 1,102 0 05 Jun 2015
On the Properties of Neural Machine Translation: Encoder-Decoder Approaches Kyunghyun Cho B. V. Merrienboer Dzmitry Bahdanau Yoshua Bengio AI4CE AIMat 268 6,791 0 03 Sep 2014

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.