A Simple Way to Initialize Recurrent Networks of Rectified Linear Units

3 April 2015

Papers citing "A Simple Way to Initialize Recurrent Networks of Rectified Linear Units"

50 / 125 papers shown

Title
Evaluating complexity and resilience trade-offs in emerging memory inference machines C. Bennett Ryan Dellana T. Xiao Ben Feinberg S. Agarwal S. Cardwell M. Marinella William M. Severa Brad Aimone 16 2 0 25 Feb 2020
Contracting Implicit Recurrent Neural Networks: Stable Models with Improved Trainability Max Revay I. Manchester 14 43 0 22 Dec 2019
On Generalization Bounds of a Family of Recurrent Neural Networks Minshuo Chen Xingguo Li T. Zhao 19 70 0 28 Oct 2019
HUBERT Untangles BERT to Improve Transfer across NLP Tasks M. Moradshahi Hamid Palangi M. Lam P. Smolensky Jianfeng Gao 26 16 0 25 Oct 2019
Generating Accurate Pseudo-labels in Semi-Supervised Learning and Avoiding Overconfident Predictions via Hermite Polynomial Activations Vishnu Suresh Lokhande Songwong Tasneeyapant Abhay Venkatesh Sathya Ravi Vikas Singh 18 29 0 12 Sep 2019
An Adaptive Stochastic Nesterov Accelerated Quasi Newton Method for Training RNNs S. Indrapriyadarsini Shahrzad Mahboubi H. Ninomiya H. Asai ODL 9 3 0 09 Sep 2019
R-Transformer: Recurrent Neural Network Enhanced Transformer Z. Wang Yao Ma Zitao Liu Jiliang Tang ViT 16 105 0 12 Jul 2019
SHE: A Fast and Accurate Deep Neural Network for Encrypted Data Qian Lou Lei Jiang 15 120 0 01 Jun 2019
Learning to Adaptively Scale Recurrent Neural Networks Hao Hu Liqiang Wang Guo-Jun Qi AI4CE 17 9 0 15 Feb 2019
Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Group Mario Lezcano Casado David Martínez-Rubio 27 194 0 24 Jan 2019
Towards Non-saturating Recurrent Units for Modelling Long-term Dependencies A. Chandar Chinnadhurai Sankar Eugene Vorontsov Samira Ebrahimi Kahou Yoshua Bengio 21 56 0 22 Jan 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 38 3,674 0 09 Jan 2019
The PyTorch-Kaldi Speech Recognition Toolkit Mirco Ravanelli Titouan Parcollet Yoshua Bengio VLM OffRL 14 225 0 19 Nov 2018
Learning to Skip Ineffectual Recurrent Computations in LSTMs A. Ardakani Zhengyun Ji W. Gross 11 16 0 09 Nov 2018
SNIP: Single-shot Network Pruning based on Connection Sensitivity Namhoon Lee Thalaiyasingam Ajanthan Philip Torr VLM 16 1,171 0 04 Oct 2018
Multimodal Language Analysis with Recurrent Multistage Fusion Paul Pu Liang Liu Ziyin Amir Zadeh Louis-Philippe Morency 30 198 0 12 Aug 2018
3D Depthwise Convolution: Reducing Model Parameters in 3D Vision Tasks Rongtian Ye Fangyu Liu Liqiang Zhang MDE 16 46 0 05 Aug 2018
Financial Trading as a Game: A Deep Reinforcement Learning Approach Chien-Yi Huang AIFin 29 72 0 08 Jul 2018
Beyond Backprop: Online Alternating Minimization with Auxiliary Variables A. Choromańska Benjamin Cowen Sadhana Kumaravel Ronny Luss Mattia Rigotti ... Brian Kingsbury Paolo Diachille V. Gurev Ravi Tejwani Djallel Bouneffouf 16 52 0 24 Jun 2018
Persistent Hidden States and Nonlinear Transformation for Long Short-Term Memory Heeyoul Choi 24 12 0 22 Jun 2018
Detecting Cyberattacks in Industrial Control Systems Using Convolutional Neural Networks Moshe Kravchik A. Shabtai 23 273 0 21 Jun 2018
On the Practical Computational Power of Finite Precision RNNs for Language Recognition Gail Weiss Yoav Goldberg Eran Yahav 15 260 0 13 May 2018
How Robust are Deep Neural Networks? B. Sengupta Karl J. Friston OOD 25 31 0 30 Apr 2018
Deep Facial Expression Recognition: A Survey Shan Li Weihong Deng 151 1,280 0 23 Apr 2018
Regularizing RNNs for Caption Generation by Reconstructing The Past with The Present Xinpeng Chen Lin Ma Wenhao Jiang Jian Yao Wei Liu 17 92 0 30 Mar 2018
Can recurrent neural networks warp time? Corentin Tallec Yann Ollivier CLL AI4CE 17 135 0 23 Mar 2018
An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling Shaojie Bai J. Zico Kolter V. Koltun DRL 42 4,715 0 04 Mar 2018
Learning Longer-term Dependencies in RNNs with Auxiliary Losses Trieu H. Trinh Andrew M. Dai Thang Luong Quoc V. Le 32 179 0 01 Mar 2018
Recent Advances in Recurrent Neural Networks Hojjat Salehinejad Sharan Sankar Joseph Barfett E. Colak S. Valaee AI4TS 30 573 0 29 Dec 2017
Dilated Recurrent Neural Networks Shiyu Chang Yang Zhang Wei Han Mo Yu Xiaoxiao Guo Wei Tan Xiaodong Cui Michael Witbrock M. Hasegawa-Johnson Thomas S. Huang 41 298 0 05 Oct 2017
Improving speech recognition by revising gated recurrent units Mirco Ravanelli Philemon Brakel M. Omologo Yoshua Bengio 19 52 0 29 Sep 2017
Skip RNN: Learning to Skip State Updates in Recurrent Neural Networks Victor Campos Brendan Jou Xavier Giró-i-Nieto Jordi Torres Shih-Fu Chang 21 217 0 22 Aug 2017
Revisiting Activation Regularization for Language RNNs Stephen Merity Bryan McCann R. Socher 33 44 0 03 Aug 2017
Orthogonal Recurrent Neural Networks with Scaled Cayley Transform Kyle E. Helfrich Devin Willmott Q. Ye 45 128 0 29 Jul 2017
Gated Orthogonal Recurrent Units: On Learning to Forget Li Jing Çağlar Gülçehre J. Peurifoy Yichen Shen Max Tegmark Marin Soljacic Yoshua Bengio 35 126 0 08 Jun 2017
Kronecker Recurrent Units C. Jose Moustapha Cissé F. Fleuret ODL 24 45 0 29 May 2017
Compressing Recurrent Neural Network with Tensor Train Andros Tjandra S. Sakti Satoshi Nakamura 25 109 0 23 May 2017
The Statistical Recurrent Unit Junier B. Oliva Barnabás Póczós J. Schneider 18 50 0 01 Mar 2017
Fast and Accurate Entity Recognition with Iterated Dilated Convolutions Emma Strubell Pat Verga David Belanger Andrew McCallum 30 391 0 07 Feb 2017
On orthogonality and learning recurrent networks with long term dependencies Eugene Vorontsov C. Trabelsi Samuel Kadoury C. Pal ODL 36 238 0 31 Jan 2017
Gate-Variants of Gated Recurrent Unit (GRU) Neural Networks Rahul Dey F. Salem 11 1,361 0 20 Jan 2017
Towards End-to-End Speech Recognition with Deep Convolutional Neural Networks Wenjie Qu Mohammad Pezeshki Philemon Brakel Saizheng Zhang Yoshua Bengio Aaron Courville 24 366 0 10 Jan 2017
A Basic Recurrent Neural Network Model F. Salem 22 16 0 29 Dec 2016
An Empirical Study of Language CNN for Image Captioning Jiuxiang Gu G. Wang Jianfei Cai Tsuhan Chen 31 132 0 21 Dec 2016
Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs Li Jing Yichen Shen T. Dubček J. Peurifoy S. Skirlo Yann LeCun Max Tegmark Marin Soljacic 29 176 0 15 Dec 2016
DizzyRNN: Reparameterizing Recurrent Neural Networks for Norm-Preserving Backpropagation Victor D. Dorobantu Per Andre Stromhaug Jess Renteria 24 25 0 13 Dec 2016
Efficient Orthogonal Parametrisation of Recurrent Neural Networks Using Householder Reflections Zakaria Mhammedi Andrew D. Hellicar Ashfaqur Rahman James Bailey 24 129 0 01 Dec 2016
Capacity and Trainability in Recurrent Neural Networks Jasmine Collins Jascha Narain Sohl-Dickstein David Sussillo 35 203 0 29 Nov 2016
Scalable Bayesian Learning of Recurrent Neural Networks for Language Modeling Zhe Gan Chunyuan Li Changyou Chen Yunchen Pu Qinliang Su Lawrence Carin BDL UQCV 53 41 0 23 Nov 2016
Deep Recurrent Neural Network for Mobile Human Activity Recognition with High Throughput Masaya Inoue Sozo Inoue T. Nishida HAI BDL 19 247 0 11 Nov 2016