Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs

15 December 2016

Papers citing "Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs"

50 / 103 papers shown

Title
An Ode to an ODE K. Choromanski Jared Davis Valerii Likhosherstov Xingyou Song Jean-Jacques E. Slotine Jacob Varley Honglak Lee Adrian Weller Vikas Sindhwani 28 30 0 19 Jun 2020
The Recurrent Neural Tangent Kernel Sina Alemohammad Zichao Wang Randall Balestriero Richard Baraniuk AAML 6 77 0 18 Jun 2020
Physical reservoir computing -- An introductory perspective Kohei Nakajima 49 308 0 03 May 2020
Multi-Decoder RNN Autoencoder Based on Variational Bayes Method Daisuke Kaji Kazuho Watanabe Masahiro Kobayashi BDL DRL 16 3 0 29 Apr 2020
CWY Parametrization: a Solution for Parallelized Optimization of Orthogonal and Stiefel Matrices Valerii Likhosherstov Jared Davis K. Choromanski Adrian Weller 20 4 0 18 Apr 2020
Orthogonal Over-Parameterized Training Weiyang Liu Rongmei Lin Zhen Liu James M. Rehg Liam Paull Li Xiong Le Song Adrian Weller 32 41 0 09 Apr 2020
Stochastic Flows and Geometric Optimization on the Orthogonal Group K. Choromanski David Cheikhi Jared Davis Valerii Likhosherstov Achille Nazaret ... Yuan Gao Aldo Pacchiano Tamás Sarlós Adrian Weller Vikas Sindhwani AI4CE 17 7 0 30 Mar 2020
Depth Enables Long-Term Memory for Recurrent Neural Networks A. Ziv 16 0 0 23 Mar 2020
Affinity guided Geometric Semi-Supervised Metric Learning U. Dutta Mehrtash Harandi C. Sekhar 19 2 0 27 Feb 2020
Design of optical neural networks with component imprecisions Michael Y.-S. Fang S. Manipatruni Casimir Wierzynski A. Khosrowshahi M. DeWeese 33 128 0 13 Dec 2019
Volume-preserving Neural Networks Gordon MacDonald Andrew Godbout Bryn Gillcash Stephanie Cairns 15 3 0 21 Nov 2019
Eigenvalue Normalized Recurrent Neural Networks for Short Term Memory Kyle E. Helfrich Q. Ye 12 6 0 18 Nov 2019
Input-Output Equivalence of Unitary and Contractive RNNs M. Emami Mojtaba Sahraee-Ardakan S. Rangan A. Fletcher 14 1 0 30 Oct 2019
Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamics Pantelis R. Vlachas Jaideep Pathak Brian R. Hunt T. Sapsis M. Girvan Edward Ott Petros Koumoutsakos AI4TS 19 384 0 09 Oct 2019
Trivializations for Gradient-Based Optimization on Manifolds Mario Lezcano Casado 11 122 0 20 Sep 2019
RNNs Evolving on an Equilibrium Manifold: A Panacea for Vanishing and Exploding Gradients? Anil Kag Ziming Zhang Venkatesh Saligrama 18 8 0 22 Aug 2019
Parameterized quantum circuits as machine learning models Marcello Benedetti Erika Lloyd Stefan H. Sack Mattia Fiorentini 27 869 0 18 Jun 2019
Compressing RNNs for IoT devices by 15-38x using Kronecker Products Urmish Thakker Jesse G. Beu Dibakar Gope Chu Zhou Igor Fedorov Ganesh S. Dasika Matthew Mattina 27 36 0 07 Jun 2019
Butterfly Transform: An Efficient FFT Based Neural Architecture Design Keivan Alizadeh-Vahid Anish K. Prabhu Ali Farhadi Mohammad Rastegari 32 50 0 05 Jun 2019
Complex Evolution Recurrent Neural Networks (ceRNNs) Izhak Shafran Tom Bagby RJ Skerry-Ryan 17 8 0 05 Jun 2019
Non-normal Recurrent Neural Network (nnRNN): learning long time dependencies while improving expressivity with transient dynamics Giancarlo Kerg Kyle Goyette M. P. Touzel Gauthier Gidel Eugene Vorontsov Yoshua Bengio Guillaume Lajoie 13 58 0 28 May 2019
Quantifying Long Range Dependence in Language and User Behavior to improve RNNs Francois Belletti Minmin Chen Ed H. Chi AI4TS 8 21 0 23 May 2019
Adaptively Truncating Backpropagation Through Time to Control Gradient Bias Christopher Aicher N. Foti E. Fox MQ 22 32 0 17 May 2019
Wave Physics as an Analog Recurrent Neural Network Tyler W. Hughes Ian A. D. Williamson M. Minkov S. Fan 19 276 0 29 Apr 2019
Low-Depth Optical Neural Networks Xiao-Ming Zhang M. Yung 25 8 0 03 Apr 2019
Learning Fast Algorithms for Linear Transforms Using Butterfly Factorizations Tri Dao Albert Gu Matthew Eichhorn Atri Rudra Christopher Ré 11 102 0 14 Mar 2019
Equilibrated Recurrent Neural Network: Neuronal Time-Delayed Self-Feedback Improves Accuracy and Stability Ziming Zhang Anil Kag Alan Sullivan Venkatesh Saligrama 23 5 0 02 Mar 2019
State-Regularized Recurrent Neural Networks Cheng Wang Mathias Niepert 18 39 0 25 Jan 2019
Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Group Mario Lezcano Casado David Martínez-Rubio 27 194 0 24 Jan 2019
Towards Non-saturating Recurrent Units for Modelling Long-term Dependencies A. Chandar Chinnadhurai Sankar Eugene Vorontsov Samira Ebrahimi Kahou Yoshua Bengio 21 56 0 22 Jan 2019
FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Network Aditya Kusupati Manish Singh Kush S. Bhatia A. Kumar Prateek Jain Manik Varma 21 189 0 08 Jan 2019
Reversible Recurrent Neural Networks M. Mackay Paul Vicol Jimmy Ba Roger C. Grosse 6 52 0 25 Oct 2018
h-detach: Modifying the LSTM Gradient Towards Better Optimization Devansh Arpit Bhargav Kanuparthi Giancarlo Kerg Nan Rosemary Ke Ioannis Mitliagkas Yoshua Bengio 23 32 0 06 Oct 2018
Single-Microphone Speech Enhancement and Separation Using Deep Learning Morten Kolbaek 17 7 0 31 Aug 2018
Matrix optimization on universal unitary photonic devices Sunil Pai Ben Bartlett O. Solgaard D. Miller 14 107 0 02 Aug 2018
IGLOO: Slicing the Features Space to Represent Sequences Vsevolod Sourkov VLM 18 5 0 09 Jul 2018
Complex Gated Recurrent Neural Networks Moritz Wolter Angela Yao AI4CE 21 64 0 21 Jun 2018
Detecting Cyberattacks in Industrial Control Systems Using Convolutional Neural Networks Moshe Kravchik A. Shabtai 23 273 0 21 Jun 2018
Stable Recurrent Models John Miller Moritz Hardt 11 116 0 25 May 2018
How to Start Training: The Effect of Initialization and Architecture Boris Hanin David Rolnick 8 254 0 05 Mar 2018
An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling Shaojie Bai J. Zico Kolter V. Koltun DRL 42 4,715 0 04 Mar 2018
Overcoming the vanishing gradient problem in plain recurrent networks Yuhuang Hu Adrian E. G. Huber Jithendar Anumula Shih-Chii Liu GNN 37 103 0 18 Jan 2018
Neural Networks in Adversarial Setting and Ill-Conditioned Weight Space M. Singh Abhishek Sinha Balaji Krishnamurthy AAML 25 6 0 03 Jan 2018
Rotational Unit of Memory Rumen Dangovski L. Jing Marin Soljacic 11 7 0 26 Oct 2017
Training Feedforward Neural Networks with Standard Logistic Activations is Feasible Emanuele Sansone F. D. De Natale 24 4 0 03 Oct 2017
Regularizing and Optimizing LSTM Language Models Stephen Merity N. Keskar R. Socher 60 1,091 0 07 Aug 2017
Revisiting Activation Regularization for Language RNNs Stephen Merity Bryan McCann R. Socher 33 44 0 03 Aug 2017
Orthogonal Recurrent Neural Networks with Scaled Cayley Transform Kyle E. Helfrich Devin Willmott Q. Ye 45 128 0 29 Jul 2017
Gated Orthogonal Recurrent Units: On Learning to Forget Li Jing Çağlar Gülçehre J. Peurifoy Yichen Shen Max Tegmark Marin Soljacic Yoshua Bengio 35 126 0 08 Jun 2017
Kronecker Recurrent Units C. Jose Moustapha Cissé F. Fleuret ODL 24 45 0 29 May 2017