Fading memory as inductive bias in residual recurrent networks

v1v2 (latest)

Fading memory as inductive bias in residual recurrent networks

27 July 2023

Felix Effenberger

ArXiv (abs)PDF HTML

Papers citing "Fading memory as inductive bias in residual recurrent networks"

15 / 15 papers shown

Title
Online learning of long-range dependencies Nicolas Zucchet Robert Meier Simon Schug Asier Mujika João Sacramento CLL 62 20 0 25 May 2023
RWKV: Reinventing RNNs for the Transformer Era Bo Peng Eric Alcaide Quentin G. Anthony Alon Albalak Samuel Arcadinho ... Qihang Zhao P. Zhou Qinghua Zhou Jian Zhu Rui-Jie Zhu 216 597 0 22 May 2023
Resurrecting Recurrent Neural Networks for Long Sequences Antonio Orvieto Samuel L. Smith Albert Gu Anushan Fernando Çağlar Gülçehre Razvan Pascanu Soham De 326 295 0 11 Mar 2023
Long Range Arena: A Benchmark for Efficient Transformers Yi Tay Mostafa Dehghani Samira Abnar Songlin Yang Dara Bahri Philip Pham J. Rao Liu Yang Sebastian Ruder Donald Metzler 147 720 0 08 Nov 2020
Coupled Oscillatory Recurrent Neural Network (coRNN): An accurate and (gradient) stable architecture for learning long time dependencies T. Konstantin Rusch Siddhartha Mishra 117 93 0 02 Oct 2020
Efficient Transformers: A Survey Yi Tay Mostafa Dehghani Dara Bahri Donald Metzler VLM 156 1,123 0 14 Sep 2020
On Second Order Behaviour in Augmented Neural ODEs Alexander Norcliffe Cristian Bodnar Ben Day Nikola Simidjievski Pietro Lio 70 93 0 12 Jun 2020
Non-normal Recurrent Neural Network (nnRNN): learning long time dependencies while improving expressivity with transient dynamics Giancarlo Kerg Kyle Goyette M. P. Touzel Gauthier Gidel Eugene Vorontsov Yoshua Bengio Guillaume Lajoie 65 58 0 28 May 2019
AntisymmetricRNN: A Dynamical System View on Recurrent Neural Networks B. Chang Minmin Chen E. Haber Ed H. Chi PINN GNN 107 204 0 26 Feb 2019
On Lazy Training in Differentiable Programming Lénaïc Chizat Edouard Oyallon Francis R. Bach 111 835 0 19 Dec 2018
Neural Ordinary Differential Equations T. Chen Yulia Rubanova J. Bettencourt David Duvenaud AI4CE 417 5,111 0 19 Jun 2018
Relational inductive biases, deep learning, and graph networks Peter W. Battaglia Jessica B. Hamrick V. Bapst Alvaro Sanchez-Gonzalez V. Zambaldi ... Pushmeet Kohli M. Botvinick Oriol Vinyals Yujia Li Razvan Pascanu AI4CE NAI 761 3,121 0 04 Jun 2018
On orthogonality and learning recurrent networks with long term dependencies Eugene Vorontsov C. Trabelsi Samuel Kadoury C. Pal ODL 84 241 0 31 Jan 2017
Unitary Evolution Recurrent Neural Networks Martín Arjovsky Amar Shah Yoshua Bengio ODL 75 770 0 20 Nov 2015
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation Kyunghyun Cho B. V. Merrienboer Çağlar Gülçehre Dzmitry Bahdanau Fethi Bougares Holger Schwenk Yoshua Bengio AIMat 1.0K 23,354 0 03 Jun 2014

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.