Persistent learning signals and working memory without continuous attractors

24 August 2023

Papers citing "Persistent learning signals and working memory without continuous attractors"

29 / 29 papers shown

Title
Back to the Continuous Attractor Ábel Ságodi Guillermo Martín-Sánchez Piotr Sokól Il Memming Park 71 3 0 31 Jul 2024
Resurrecting Recurrent Neural Networks for Long Sequences Antonio Orvieto Samuel L. Smith Albert Gu Anushan Fernando Çağlar Gülçehre Razvan Pascanu Soham De 326 295 0 11 Mar 2023
On the difficulty of learning chaotic dynamics with RNNs Jonas M. Mikhaeil Zahra Monfared Daniel Durstewitz 123 58 0 14 Oct 2021
UnICORNN: A recurrent model for learning very long time dependencies T. Konstantin Rusch Siddhartha Mishra 68 61 0 09 Mar 2021
On the Origin of Implicit Regularization in Stochastic Gradient Descent Samuel L. Smith Benoit Dherin David Barrett Soham De MLT 40 203 0 28 Jan 2021
Event-Based Backpropagation can compute Exact Gradients for Spiking Neural Networks Timo C. Wunderlich Christian Pehle 76 120 0 17 Sep 2020
On Lyapunov Exponents for RNNs: Understanding Information Propagation Using Dynamical Systems Tools Ryan H. Vogt M. P. Touzel Eli Shlizerman Guillaume Lajoie 73 42 0 25 Jun 2020
Lipschitz Recurrent Neural Networks N. Benjamin Erichson Omri Azencot A. Queiruga Liam Hodgkinson Michael W. Mahoney 67 111 0 22 Jun 2020
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 514 42,449 0 03 Dec 2019
A Unified Framework of Online Learning Algorithms for Training Recurrent Neural Networks O. Marschall Kyunghyun Cho Cristina Savin FedML 71 73 0 05 Jul 2019
Gated recurrent units viewed through the lens of continuous time dynamical systems I. Jordan Piotr A. Sokól Il Memming Park 47 58 0 03 Jun 2019
ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEs A. Gholami Kurt Keutzer George Biros 100 170 0 27 Feb 2019
AntisymmetricRNN: A Dynamical System View on Recurrent Neural Networks B. Chang Minmin Chen E. Haber Ed H. Chi PINN GNN 107 204 0 26 Feb 2019
Robust computation with rhythmic spike patterns E. P. Frady Friedrich T. Sommer 52 78 0 23 Jan 2019
Information Geometry of Orthogonal Initializations and Training Piotr A. Sokól Il-Su Park AI4CE 117 17 0 09 Oct 2018
Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables Signal Propagation in Recurrent Neural Networks Minmin Chen Jeffrey Pennington S. Schoenholz SyDa AI4CE 55 116 0 14 Jun 2018
How to Start Training: The Effect of Initialization and Architecture Boris Hanin David Rolnick 71 255 0 05 Mar 2018
Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice Jeffrey Pennington S. Schoenholz Surya Ganguli ODL 43 252 0 13 Nov 2017
Gradient Descent for Spiking Neural Networks Dongsung Huh T. Sejnowski 61 257 0 14 Jun 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 704 131,652 0 12 Jun 2017
Stable Architectures for Deep Neural Networks E. Haber Lars Ruthotto 147 729 0 09 May 2017
On orthogonality and learning recurrent networks with long term dependencies Eugene Vorontsov C. Trabelsi Samuel Kadoury C. Pal ODL 84 241 0 31 Jan 2017
Efficient Orthogonal Parametrisation of Recurrent Neural Networks Using Householder Reflections Zakaria Mhammedi Andrew D. Hellicar Ashfaqur Rahman James Bailey 87 129 0 01 Dec 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 194,020 0 10 Dec 2015
Unitary Evolution Recurrent Neural Networks Martín Arjovsky Amar Shah Yoshua Bengio ODL 75 770 0 20 Nov 2015
Training Very Deep Networks R. Srivastava Klaus Greff Jürgen Schmidhuber 161 1,682 0 22 Jul 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 463 43,305 0 11 Feb 2015
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks Andrew M. Saxe James L. McClelland Surya Ganguli ODL 173 1,849 0 20 Dec 2013
On the difficulty of training Recurrent Neural Networks Razvan Pascanu Tomas Mikolov Yoshua Bengio ODL 196 5,353 0 21 Nov 2012