Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations

3 June 2016

Aaron Courville

Papers citing "Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations"

50 / 69 papers shown

Title
DropDim: A Regularization Method for Transformer Networks Hao Zhang Dan Qu Kejia Shao Xu Yang 28 12 0 20 Apr 2023
Efficient Transformers with Dynamic Token Pooling Piotr Nawrot J. Chorowski Adrian Lañcucki Edoardo Ponti 22 42 0 17 Nov 2022
Learning Sequence Representations by Non-local Recurrent Neural Memory Wenjie Pei Xin Feng Canmiao Fu Qi Cao Guangming Lu Yu-Wing Tai AI4TS 27 1 0 20 Jul 2022
RF-Next: Efficient Receptive Field Search for Convolutional Neural Networks Shanghua Gao Zhong-Yu Li Qi Han Ming-Ming Cheng Liang Wang 32 34 0 14 Jun 2022
Temporal Latent Bottleneck: Synthesis of Fast and Slow Processing Mechanisms in Sequence Learning Aniket Didolkar Kshitij Gupta Anirudh Goyal Nitesh B. Gundavarapu Alex Lamb Nan Rosemary Ke Yoshua Bengio AI4CE 118 17 0 30 May 2022
A Survey on Dropout Methods and Experimental Verification in Recommendation Yong Li Weizhi Ma C. L. Philip Chen Hao Fei Yiqun Liu Shaoping Ma Yue Yang 33 9 0 05 Apr 2022
Prosodic Clustering for Phoneme-level Prosody Control in End-to-End Speech Synthesis Alexandra Vioni Myrsini Christidou Nikolaos Ellinas G. Vamvoukakis Panos Kakoulidis Taehoon Kim June Sig Sung Hyoungmin Park Aimilios Chalamandaris Pirros Tsiakoulis 16 11 0 19 Nov 2021
Preventing posterior collapse in variational autoencoders for text generation via decoder regularization Alban Petit Caio Corro DRL 21 3 0 28 Oct 2021
Translatotron 2: High-quality direct speech-to-speech translation with voice preservation Ye Jia Michelle Tadmor Ramanovich Tal Remez Roi Pomerantz 26 67 0 19 Jul 2021
Discrete-Valued Neural Communication Dianbo Liu DianboLiu Alex Lamb Kenji Kawaguchi Anirudh Goyal Chen Sun Michael C. Mozer Yoshua Bengio 26 50 0 06 Jul 2021
Recurrent Neural Network from Adder's Perspective: Carry-lookahead RNN Haowei Jiang Fei-wei Qin Jin Cao Yong Peng Yanli Shao LRM ODL 16 42 0 22 Jun 2021
WaveGrad 2: Iterative Refinement for Text-to-Speech Synthesis Nanxin Chen Yu Zhang Heiga Zen Ron J. Weiss Mohammad Norouzi Najim Dehak William Chan DiffM 23 88 0 17 Jun 2021
On the limit of English conversational speech recognition Zoltán Tüske G. Saon Brian Kingsbury 22 50 0 03 May 2021
Comparing the Benefit of Synthetic Training Data for Various Automatic Speech Recognition Architectures Nick Rossenbach Mohammad Zeineldeen Benedikt Hilmes Ralf Schluter Hermann Ney 30 12 0 12 Apr 2021
UniDrop: A Simple yet Effective Technique to Improve Transformer without Extra Cost Zhen Wu Lijun Wu Qi Meng Yingce Xia Shufang Xie Tao Qin Xinyu Dai Tie-Yan Liu 18 22 0 11 Apr 2021
LocalDrop: A Hybrid Regularization for Deep Neural Networks Ziqing Lu Chang Xu Bo Du Takashi Ishida Lefei Zhang Masashi Sugiyama 33 14 0 01 Mar 2021
Sequential Deep Learning for Credit Risk Monitoring with Tabular Financial Data Jillian M. Clements Di Xu N. Yousefi Dmitry Efimov 28 41 0 30 Dec 2020
AISHELL-3: A Multi-speaker Mandarin TTS Corpus and the Baselines Yao Shi Hui Bu Xin Xu Shaojing Zhang Ming Li 30 219 0 22 Oct 2020
Replacing Human Audio with Synthetic Audio for On-device Unspoken Punctuation Prediction Daria Soboleva Ondrej Skopek Márius vSajgalík Victor Cuarbune Felix Weissenberger ... B. Prisacari Daniel Valcarce Justin Lu Rohit Prabhavalkar Balint Miklos 32 9 0 20 Oct 2020
HiPPO: Recurrent Memory with Optimal Polynomial Projections Albert Gu Tri Dao Stefano Ermon Atri Rudra Christopher Ré 45 489 0 17 Aug 2020
A Transfer Learning End-to-End ArabicText-To-Speech (TTS) Deep Architecture Fady K. Fahmy M. Khalil Hazem M. Abbas 41 20 0 22 Jul 2020
PatchUp: A Feature-Space Block-Level Regularization Technique for Convolutional Neural Networks Mojtaba Faramarzi Mohammad Amini Akilesh Badrinaaraayanan Vikas Verma A. Chandar AAML 34 31 0 14 Jun 2020
Investigation of learning abilities on linguistic features in sequence-to-sequence text-to-speech synthesis Yusuke Yasuda Xin Wang Junichi Yamagishi AI4TS 19 31 0 20 May 2020
Temporal Convolutional Attention-based Network For Sequence Modeling Hongyan Hao Yan Wang Siqiao Xue Yudi Xia Jian Zhao S. Furao 30 41 0 28 Feb 2020
Single headed attention based sequence-to-sequence model for state-of-the-art results on Switchboard Zoltán Tüske G. Saon Kartik Audhkhasi Brian Kingsbury BDL 23 68 0 20 Jan 2020
ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training Weizhen Qi Yu Yan Yeyun Gong Dayiheng Liu Nan Duan Jiusheng Chen Ruofei Zhang Ming Zhou AI4TS 27 446 0 13 Jan 2020
Modelling EHR timeseries by restricting feature interaction Kun Zhang Yuan Xue Gerardo Flores A. Rajkomar Claire Cui Andrew M. Dai 17 5 0 14 Nov 2019
Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamics Pantelis R. Vlachas Jaideep Pathak Brian R. Hunt T. Sapsis M. Girvan Edward Ott Petros Koumoutsakos AI4TS 19 384 0 09 Oct 2019
High-Dimensional Multivariate Forecasting with Low-Rank Gaussian Copula Processes David Salinas Michael Bohlke-Schneider Laurent Callot Roberto Medico Jan Gasthaus AI4TS 30 224 0 07 Oct 2019
Initial investigation of an encoder-decoder end-to-end TTS framework using marginalization of monotonic hard latent alignments Yusuke Yasuda Xin Wang Junichi Yamagishi 21 8 0 30 Aug 2019
Non-local Recurrent Neural Memory for Supervised Sequence Modeling Canmiao Fu Wenjie Pei Qiong Cao Chaopeng Zhang Yong Zhao Xiaoyong Shen Yu-Wing Tai 21 11 0 26 Aug 2019
DropAttention: A Regularization Method for Fully-Connected Self-Attention Networks Zehui Lin Pengfei Liu Luyao Huang Junkun Chen Xipeng Qiu Xuanjing Huang 3DPC 16 44 0 25 Jul 2019
R-Transformer: Recurrent Neural Network Enhanced Transformer Z. Wang Yao Ma Zitao Liu Jiliang Tang ViT 16 105 0 12 Jul 2019
Survey of Dropout Methods for Deep Neural Networks Alex Labach Hojjat Salehinejad S. Valaee 27 149 0 25 Apr 2019
Direct speech-to-speech translation with a sequence-to-sequence model Ye Jia Ron J. Weiss Fadi Biadsy Wolfgang Macherey Melvin Johnson Z. Chen Yonghui Wu 21 222 0 12 Apr 2019
Augment your batch: better training with larger batches Elad Hoffer Tal Ben-Nun Itay Hubara Niv Giladi Torsten Hoefler Daniel Soudry ODL 30 72 0 27 Jan 2019
Stroke-based sketched symbol reconstruction and segmentation Kurmanbek Kaiyrbekov M. Sezgin 22 19 0 10 Jan 2019
Investigation of enhanced Tacotron text-to-speech synthesis systems with self-attention for pitch accent language Yusuke Yasuda Xin Wang Shinji Takaki Junichi Yamagishi 22 86 0 29 Oct 2018
Sequence-to-Sequence Acoustic Modeling for Voice Conversion Jing-Xuan Zhang Zhenhua Ling Li-Juan Liu Yuan Jiang Lirong Dai 11 129 0 16 Oct 2018
Semi-Supervised Training for Improving Data Efficiency in End-to-End Speech Synthesis Yu-An Chung Yuxuan Wang Wei-Ning Hsu Yu Zhang RJ Skerry-Ryan 22 117 0 30 Aug 2018
Grow and Prune Compact, Fast, and Accurate LSTMs Xiaoliang Dai Hongxu Yin N. Jha VLM SyDa 31 90 0 30 May 2018
Token-level and sequence-level loss smoothing for RNN language models Maha Elbayad Laurent Besacier Jakob Verbeek 22 19 0 14 May 2018
Noisin: Unbiased Regularization for Recurrent Neural Networks Adji Bousso Dieng Rajesh Ranganath Jaan Altosaar David M. Blei 22 22 0 03 May 2018
Regularizing RNNs for Caption Generation by Reconstructing The Past with The Present Xinpeng Chen Lin Ma Wenhao Jiang Jian Yao Wei Liu 17 92 0 30 Mar 2018
Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis Yuxuan Wang Daisy Stanton Yu Zhang RJ Skerry-Ryan Eric Battenberg Joel Shor Y. Xiao Fei Ren Ye Jia Rif A. Saurous 21 815 0 23 Mar 2018
An Analysis of Neural Language Modeling at Multiple Scales Stephen Merity N. Keskar R. Socher 24 170 0 22 Mar 2018
Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches Yeming Wen Paul Vicol Jimmy Ba Dustin Tran Roger C. Grosse BDL 22 307 0 12 Mar 2018
An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling Shaojie Bai J. Zico Kolter V. Koltun DRL 42 4,724 0 04 Mar 2018
Scalable and accurate deep learning for electronic health records A. Rajkomar Eyal Oren Kai Chen Andrew M. Dai Nissan Hajaj ... A. Butte M. Howell Claire Cui Greg S. Corrado Jeffrey Dean OOD BDL 44 2,110 0 24 Jan 2018
Recent Advances in Recurrent Neural Networks Hojjat Salehinejad Sharan Sankar Joseph Barfett E. Colak S. Valaee AI4TS 30 573 0 29 Dec 2017