v1v2 (latest)

Fast-Slow Recurrent Neural Networks

24 May 2017

Papers citing "Fast-Slow Recurrent Neural Networks"

49 / 49 papers shown

Title
Contextually Guided Transformers via Low-Rank Adaptation A. Zhmoginov Jihwan Lee Max Vladymyrov Mark Sandler OffRL 57 0 0 06 Jun 2025
A Quantitative Review on Language Model Efficiency Research Meng Jiang Hy Dang Lingbo Tong 76 0 0 28 May 2023
Long-horizon video prediction using a dynamic latent hierarchy Alexey Zakharov Qinghai Guo Zafeirios Fountas 77 4 0 29 Dec 2022
Efficient Transformers with Dynamic Token Pooling Piotr Nawrot J. Chorowski Adrian Lañcucki Edoardo Ponti 83 46 0 17 Nov 2022
Variational Predictive Routing with Nested Subjective Timescales Alexey Zakharov Qinghai Guo Zafeirios Fountas BDL AI4TS 65 9 0 21 Oct 2021
Clockwork Variational Autoencoders Vaibhav Saxena Jimmy Ba Danijar Hafner VGen DRL 90 49 0 18 Feb 2021
A New Training Pipeline for an Improved Neural Transducer Albert Zeyer André Merboldt Ralf Schluter Hermann Ney AI4TS MedIm 75 52 0 19 May 2020
Deep Learning for Source Code Modeling and Generation: Models, Applications and Challenges T. H. Le Hao Chen Muhammad Ali Babar VLM 147 155 0 13 Feb 2020
Explicit Sparse Transformer: Concentrated Attention Through Explicit Selection Guangxiang Zhao Junyang Lin Zhiyuan Zhang Xuancheng Ren Qi Su Xu Sun 79 113 0 25 Dec 2019
Eigenvalue Normalized Recurrent Neural Networks for Short Term Memory Kyle E. Helfrich Qiang Ye 44 6 0 18 Nov 2019
Multi-Zone Unit for Recurrent Neural Networks Fandong Meng Jinchao Zhang Yang Liu Jie Zhou AI4CE 57 2 0 17 Nov 2019
Deep Independently Recurrent Neural Network (IndRNN) Shuai Li Wanqing Li Chris Cook Yanbo Gao 66 51 0 11 Oct 2019
Meta-Learning with Warped Gradient Descent Sebastian Flennerhag Andrei A. Rusu Razvan Pascanu Francesco Visin Hujun Yin R. Hadsell 110 210 0 30 Aug 2019
Augmenting Self-attention with Persistent Memory Sainbayar Sukhbaatar Edouard Grave Guillaume Lample Hervé Jégou Armand Joulin RALM KELM 77 139 0 02 Jul 2019
Dynamic Evaluation of Transformer Language Models Ben Krause Emmanuel Kahembwe Iain Murray Steve Renals 100 43 0 17 Apr 2019
DA-LSTM: A Long Short-Term Memory with Depth Adaptive to Non-uniform Information Flow in Sequential Data Yifeng Zhang Ka-Ho Chow Shueng-Han Gary Chan AI4TS 67 2 0 18 Jan 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 275 3,756 0 09 Jan 2019
Long Short-Term Memory with Dynamic Skip Connections Tao Gui Qi Zhang Lujun Zhao Y. Lin Minlong Peng Jingjing Gong Xuanjing Huang 85 27 0 09 Nov 2018
Benchmarking Deep Sequential Models on Volatility Predictions for Financial Time Series Qiang Zhang Kyle Birkeland Yaodong Yang Yixiao Liu 64 9 0 08 Nov 2018
Trellis Networks for Sequence Modeling Shaojie Bai J. Zico Kolter V. Koltun 86 146 0 15 Oct 2018
Improved Language Modeling by Decoding the Past Siddhartha Brahma BDL AI4TS 42 6 0 14 Aug 2018
Character-Level Language Modeling with Deeper Self-Attention Rami Al-Rfou Dokook Choe Noah Constant Mandy Guo Llion Jones 159 393 0 09 Aug 2018
An Analysis of Neural Language Modeling at Multiple Scales Stephen Merity N. Keskar R. Socher 79 171 0 22 Mar 2018
Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN Shuai Li W. Li Chris Cook Ce Zhu Yanbo Gao 98 732 0 13 Mar 2018
Wider and Deeper, Cheaper and Faster: Tensorized LSTMs for Sequence Learning Zhen He Shaobing Gao Liang Xiao Daxue Liu Hangen He David Barber AIMat 92 65 0 05 Nov 2017
Language Modeling with Highway LSTM Gakuto Kurata Bhuvana Ramabhadran G. Saon A. Sethy AI4TS 73 38 0 19 Sep 2017
Simple Recurrent Units for Highly Parallelizable Recurrence Tao Lei Yu Zhang Sida I. Wang Huijing Dai Yoav Artzi LRM 165 277 0 08 Sep 2017
Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs Li Jing Yichen Shen T. Dubček J. Peurifoy S. Skirlo Yann LeCun Max Tegmark Marin Soljacic 99 178 0 15 Dec 2016
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 563 5,392 0 05 Nov 2016
Neural Machine Translation in Linear Time Nal Kalchbrenner L. Espeholt Karen Simonyan Aaron van den Oord Alex Graves Koray Kavukcuoglu AIMat 124 553 0 31 Oct 2016
Surprisal-Driven Zoneout K. Rocki Tomasz Kornuta Tegan Maharaj 101 8 0 24 Oct 2016
HyperNetworks David R Ha Andrew M. Dai Quoc V. Le 182 1,637 0 27 Sep 2016
Multiplicative LSTM for sequence modelling Ben Krause Liang Lu Iain Murray Steve Renals 118 208 0 26 Sep 2016
Hierarchical Multiscale Recurrent Neural Networks Junyoung Chung Sungjin Ahn Yoshua Bengio BDL 130 538 0 06 Sep 2016
Layer Normalization Jimmy Lei Ba J. Kiros Geoffrey E. Hinton 444 10,560 0 21 Jul 2016
Recurrent Highway Networks J. Zilly R. Srivastava Jan Koutník Jürgen Schmidhuber 126 417 0 12 Jul 2016
Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations David M. Krueger Tegan Maharaj János Kramár Mohammad Pezeshki Nicolas Ballas Nan Rosemary Ke Anirudh Goyal Yoshua Bengio Aaron Courville C. Pal 115 318 0 03 Jun 2016
Adaptive Computation Time for Recurrent Neural Networks Alex Graves 131 552 0 29 Mar 2016
End-to-End Attention-based Large Vocabulary Speech Recognition Dzmitry Bahdanau J. Chorowski Dmitriy Serdyuk Philemon Brakel Yoshua Bengio 132 1,152 0 18 Aug 2015
A Hierarchical Recurrent Encoder-Decoder For Generative Context-Aware Query Suggestion Alessandro Sordoni Yoshua Bengio H. Vahabi Christina Lioma J. Simonsen J. Nie 3DV 95 552 0 08 Jul 2015
Highway Networks R. Srivastava Klaus Greff Jürgen Schmidhuber 187 1,771 0 03 May 2015
Gated Feedback Recurrent Neural Networks Junyoung Chung Çağlar Gülçehre Kyunghyun Cho Yoshua Bengio 88 831 0 09 Feb 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 2.4K 150,597 0 22 Dec 2014
Neural Turing Machines Alex Graves Greg Wayne Ivo Danihelka 120 2,334 0 20 Oct 2014
Memory Networks Jason Weston S. Chopra Antoine Bordes GNN KELM 324 1,711 0 15 Oct 2014
Recurrent Neural Network Regularization Wojciech Zaremba Ilya Sutskever Oriol Vinyals ODL 182 2,782 0 08 Sep 2014
A Clockwork RNN Jan Koutník Klaus Greff Faustino J. Gomez Jürgen Schmidhuber 92 503 0 14 Feb 2014
How to Construct Deep Recurrent Neural Networks Razvan Pascanu Çağlar Gülçehre Kyunghyun Cho Yoshua Bengio 232 1,011 0 20 Dec 2013
Generating Sequences With Recurrent Neural Networks Alex Graves GAN 279 4,045 0 04 Aug 2013