A Simple Way to Initialize Recurrent Networks of Rectified Linear Units

3 April 2015

Papers citing "A Simple Way to Initialize Recurrent Networks of Rectified Linear Units"

50 / 125 papers shown

Title
STEP: A Unified Spiking Transformer Evaluation Platform for Fair and Reproducible Benchmarking Sicheng Shen Dongcheng Zhao Linghao Feng Zeyang Yue Jindong Li Tenglong Li Guobin Shen Yi Zeng 24 0 0 16 May 2025
Unconditional stability of a recurrent neural circuit implementing divisive normalization Shivang Rawat David J. Heeger Stefano Martiniani 29 0 0 27 Sep 2024
Oja's plasticity rule overcomes several challenges of training neural networks under biological constraints Navid Shervani-Tabar Marzieh Alireza Mirhoseini Robert Rosenbaum AAML AI4CE 45 0 0 15 Aug 2024
Deep State Space Recurrent Neural Networks for Time Series Forecasting Hugo Inzirillo AI4TS 31 6 0 21 Jul 2024
Geometric sparsification in recurrent neural networks Wyatt Mackey Ioannis Schizas Jared Deighton David L. Boothe, Jr. Vasileios Maroulas 33 0 0 10 Jun 2024
The Ghanaian NLP Landscape: A First Look Sheriff Issaka Zhaoyi Zhang Mihir Heda Keyi Wang Yinka Ajibola Ryan DeMar Xuefeng Du 46 1 0 10 May 2024
Graph Expansion in Pruned Recurrent Neural Network Layers Preserve Performance Suryam Arnav Kalra Arindam Biswas Pabitra Mitra Biswajit Basu GNN 46 0 0 17 Mar 2024
Input Convex Lipschitz RNN: A Fast and Robust Approach for Engineering Tasks Zihao Wang Zhe Wu 25 3 0 15 Jan 2024
Fading memory as inductive bias in residual recurrent networks I. Dubinin Felix Effenberger 43 4 0 27 Jul 2023
Long Short-term Memory with Two-Compartment Spiking Neuron Shimin Zhang Qu Yang Chenxiang Ma Jibin Wu Haizhou Li Kay Chen Tan 33 7 0 14 Jul 2023
SMPConv: Self-moving Point Representations for Continuous Convolution Sanghyeon Kim Eunbyung Park 3DPC 42 13 0 05 Apr 2023
Convolutional unitary or orthogonal recurrent neural networks M. Magnasco 16 2 0 14 Feb 2023
A Natural Bias for Language Generation Models Clara Meister Wojciech Stokowiec Tiago Pimentel Lei Yu Laura Rimell A. Kuncoro MILM 33 6 0 19 Dec 2022
Gated Recurrent Neural Networks with Weighted Time-Delay Feedback N. Benjamin Erichson S. H. Lim Michael W. Mahoney 26 6 0 01 Dec 2022
Exploring the Long-Term Generalization of Counting Behavior in RNNs Nadine El-Naggar Pranava Madhyastha Tillman Weyde 15 5 0 29 Nov 2022
Lempel-Ziv Networks Rebecca Saul Mohammad Mahmudul Alam John Hurwitz Edward Raff Tim Oates James Holt 26 2 0 23 Nov 2022
Random orthogonal additive filters: a solution to the vanishing/exploding gradient of deep neural networks Andrea Ceni ODL 25 3 0 03 Oct 2022
An Embarrassingly Simple Approach for Intellectual Property Rights Protection on Recurrent Neural Networks Zhi Qin Tan H. P. Wong Chee Seng Chan 22 1 0 03 Oct 2022
Efficient LSTM Training with Eligibility Traces Mitchell L. Hoyer Shahram Eivazi S. Otte 9 1 0 30 Sep 2022
Towards Large-Scale Small Object Detection: Survey and Benchmarks Gong Cheng Xiang Yuan Xiwen Yao Ke Yan Qinghua Zeng Xingxing Xie Junwei Han ObjD 30 307 0 28 Jul 2022
RF-Next: Efficient Receptive Field Search for Convolutional Neural Networks Shanghua Gao Zhong-Yu Li Qi Han Ming-Ming Cheng Liang Wang 32 34 0 14 Jun 2022
Towards a General Purpose CNN for Long Range Dependencies in $N$ D David W. Romero David M. Knigge Albert Gu Erik J. Bekkers E. Gavves Jakub M. Tomczak Mark Hoogendoorn 24 19 0 07 Jun 2022
Feedback Gradient Descent: Efficient and Stable Optimization with Orthogonality for DNNs Fanchen Bu D. Chang 28 6 0 12 May 2022
Path Development Network with Finite-dimensional Lie Group Representation Han Lou Siran Li Hao Ni 18 7 0 02 Apr 2022
NeuroView-RNN: It's About Time C. Barberan Sina Alemohammad Naiming Liu Randall Balestriero Richard G. Baraniuk AI4TS HAI 41 2 0 23 Feb 2022
Semantic-based Data Augmentation for Math Word Problems Ai Li Jiaqing Liang Yanghua Xiao AAML 24 7 0 07 Jan 2022
Classification of Long Sequential Data using Circular Dilated Convolutional Neural Networks Lei Cheng Ruslan Khalitov Tong Yu Zhirong Yang 25 32 0 06 Jan 2022
Target Propagation via Regularized Inversion Vincent Roulet Zaïd Harchaoui BDL AAML 27 4 0 02 Dec 2021
Gradients are Not All You Need Luke Metz C. Freeman S. Schoenholz Tal Kachman 30 93 0 10 Nov 2021
Cortico-cerebellar networks as decoupling neural interfaces J. Pemberton E. Boven Richard Apps Rui Ponte Costa 33 6 0 21 Oct 2021
FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizes David W. Romero Robert-Jan Bruintjes Jakub M. Tomczak Erik J. Bekkers Mark Hoogendoorn Jan van Gemert 80 82 0 15 Oct 2021
Orthogonal Graph Neural Networks Kai Guo Kaixiong Zhou Xia Hu Yu Li Yi Chang Xin Wang 43 34 0 23 Sep 2021
Oscillatory Fourier Neural Network: A Compact and Efficient Architecture for Sequential Processing Bing Han Cheng Wang Kaushik Roy 34 7 0 14 Sep 2021
CAN3D: Fast 3D Medical Image Segmentation via Compact Context Aggregation Wei Dai B. Woo Siyu Liu Matthew Marques Craig B. Engstrom P. Greer Stuart Crozier Jason Dowling Shekhar S. Chandra 33 14 0 12 Sep 2021
Acceleration Method for Learning Fine-Layered Optical Neural Networks K. Aoyama H. Sawada 27 1 0 01 Sep 2021
Dirichlet Energy Constrained Learning for Deep Graph Neural Networks Kaixiong Zhou Xiao Shi Huang Daochen Zha Rui Chen Li Li Soo-Hyun Choi Xia Hu GNN AI4CE 33 113 0 06 Jul 2021
Recurrent Neural Network from Adder's Perspective: Carry-lookahead RNN Haowei Jiang Fei-wei Qin Jin Cao Yong Peng Yanli Shao LRM ODL 16 42 0 22 Jun 2021
RNNs of RNNs: Recursive Construction of Stable Assemblies of Recurrent Neural Networks L. Kozachkov Michaela Ennis Jean-Jacques E. Slotine 22 18 0 16 Jun 2021
Spectral Pruning for Recurrent Neural Networks Takashi Furuya Kazuma Suetake K. Taniguchi Hiroyuki Kusumoto Ryuji Saiin Tomohiro Daimon 27 4 0 23 May 2021
Continual Learning for Recurrent Neural Networks: an Empirical Evaluation Andrea Cossu Antonio Carta Vincenzo Lomonaco D. Bacciu CLL 36 107 0 12 Mar 2021
Advances in Electron Microscopy with Deep Learning Jeffrey M. Ede 35 2 0 04 Jan 2021
Nanopore Base Calling on the Edge Peter Perešíni V. Boža Broňa Brejová T. Vinař 19 38 0 09 Nov 2020
FedSL: Federated Split Learning on Distributed Sequential Data in Recurrent Neural Networks Ali Abedi Shehroz S. Khan FedML 39 53 0 06 Nov 2020
Revisiting Batch Normalization for Training Low-latency Deep Spiking Neural Networks from Scratch Youngeun Kim Priyadarshini Panda 30 171 0 05 Oct 2020
Coupled Oscillatory Recurrent Neural Network (coRNN): An accurate and (gradient) stable architecture for learning long time dependencies T. Konstantin Rusch Siddhartha Mishra 24 89 0 02 Oct 2020
Review: Deep Learning in Electron Microscopy Jeffrey M. Ede 34 79 0 17 Sep 2020
HiPPO: Recurrent Memory with Optimal Polynomial Projections Albert Gu Tri Dao Stefano Ermon Atri Rudra Christopher Ré 30 489 0 17 Aug 2020
Deep Isometric Learning for Visual Recognition Haozhi Qi Chong You Xinyu Wang Yi Ma Jitendra Malik VLM 30 53 0 30 Jun 2020
Lipschitz Recurrent Neural Networks N. Benjamin Erichson Omri Azencot A. Queiruga Liam Hodgkinson Michael W. Mahoney 30 107 0 22 Jun 2020
Effective and Efficient Computation with Multiple-timescale Spiking Recurrent Neural Networks Bojian Yin Federico Corradi Sander M. Bohté 20 99 0 24 May 2020