v1v2 (latest)

Non-normal Recurrent Neural Network (nnRNN): learning long time dependencies while improving expressivity with transient dynamics

28 May 2019

Papers citing "Non-normal Recurrent Neural Network (nnRNN): learning long time dependencies while improving expressivity with transient dynamics"

28 / 28 papers shown

Title
Gated Recurrent Neural Networks with Weighted Time-Delay Feedback N. Benjamin Erichson Soon Hoe Lim Michael W. Mahoney 76 6 0 01 Dec 2022
Forecasting Sequential Data using Consistent Koopman Autoencoders Omri Azencot N. Benjamin Erichson Vanessa Lin Michael W. Mahoney AI4TS AI4CE 195 152 0 04 Mar 2020
Improved memory in recurrent neural networks with sequential non-normal dynamics A. Orhan Xaq Pitkow 66 14 0 31 May 2019
AntisymmetricRNN: A Dynamical System View on Recurrent Neural Networks B. Chang Minmin Chen E. Haber Ed H. Chi PINN GNN 117 207 0 26 Feb 2019
Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Group Mario Lezcano Casado David Martínez-Rubio 50 200 0 24 Jan 2019
Towards Non-saturating Recurrent Units for Modelling Long-term Dependencies A. Chandar Chinnadhurai Sankar Eugene Vorontsov Samira Ebrahimi Kahou Yoshua Bengio 93 56 0 22 Jan 2019
Complex Unitary Recurrent Neural Networks using Scaled Cayley Transform K. D. G. Maduranga Kyle E. Helfrich Qiang Ye 64 32 0 09 Nov 2018
h-detach: Modifying the LSTM Gradient Towards Better Optimization Devansh Arpit Bhargav Kanuparthi Giancarlo Kerg Nan Rosemary Ke Ioannis Mitliagkas Yoshua Bengio 103 32 0 06 Oct 2018
Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables Signal Propagation in Recurrent Neural Networks Minmin Chen Jeffrey Pennington S. Schoenholz SyDa AI4CE 67 117 0 14 Jun 2018
Quaternion Recurrent Neural Networks Titouan Parcollet Mirco Ravanelli Mohamed Morchid G. Linarès C. Trabelsi R. Mori Yoshua Bengio 46 132 0 12 Jun 2018
Stabilizing Gradients for Deep Neural Networks via Efficient SVD Parameterization Jiong Zhang Qi Lei Inderjit S. Dhillon 55 112 0 25 Mar 2018
Can recurrent neural networks warp time? Corentin Tallec Yann Ollivier CLL AI4CE 138 136 0 23 Mar 2018
An Analysis of Neural Language Modeling at Multiple Scales Stephen Merity N. Keskar R. Socher 74 171 0 22 Mar 2018
Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN Shuai Li W. Li Chris Cook Ce Zhu Yanbo Gao 95 731 0 13 Mar 2018
Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice Jeffrey Pennington S. Schoenholz Surya Ganguli ODL 53 254 0 13 Nov 2017
Orthogonal Recurrent Neural Networks with Scaled Cayley Transform Kyle E. Helfrich Devin Willmott Qiang Ye 101 131 0 29 Jul 2017
Gated Orthogonal Recurrent Units: On Learning to Forget Li Jing Çağlar Gülçehre J. Peurifoy Yichen Shen Max Tegmark Marin Soljacic Yoshua Bengio 88 126 0 08 Jun 2017
On orthogonality and learning recurrent networks with long term dependencies Eugene Vorontsov C. Trabelsi Samuel Kadoury C. Pal ODL 115 242 0 31 Jan 2017
Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs Li Jing Yichen Shen T. Dubček J. Peurifoy S. Skirlo Yann LeCun Max Tegmark Marin Soljacic 95 178 0 15 Dec 2016
Efficient Orthogonal Parametrisation of Recurrent Neural Networks Using Householder Reflections Zakaria Mhammedi Andrew D. Hellicar Ashfaqur Rahman James Bailey 101 129 0 01 Dec 2016
Full-Capacity Unitary Recurrent Neural Networks Scott Wisdom Thomas Powers J. Hershey Jonathan Le Roux L. Atlas 87 294 0 31 Oct 2016
On the Expressive Power of Deep Neural Networks M. Raghu Ben Poole Jon M. Kleinberg Surya Ganguli Jascha Narain Sohl-Dickstein 84 791 0 16 Jun 2016
Recurrent Orthogonal Networks and Long-Memory Tasks Mikael Henaff Arthur Szlam Yann LeCun 83 133 0 22 Feb 2016
Unitary Evolution Recurrent Neural Networks Martín Arjovsky Amar Shah Yoshua Bengio ODL 86 771 0 20 Nov 2015
A Simple Way to Initialize Recurrent Networks of Rectified Linear Units Quoc V. Le Navdeep Jaitly Geoffrey E. Hinton ODL 105 722 0 03 Apr 2015
Deep Fried Convnets Zichao Yang Marcin Moczulski Misha Denil Nando de Freitas Alex Smola Le Song Ziyu Wang 134 267 0 22 Dec 2014
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks Andrew M. Saxe James L. McClelland Surya Ganguli ODL 200 1,852 0 20 Dec 2013
On the difficulty of training Recurrent Neural Networks Razvan Pascanu Tomas Mikolov Yoshua Bengio ODL 218 5,361 0 21 Nov 2012