Breaking the Softmax Bottleneck: A High-Rank RNN Language Model

10 November 2017

Papers citing "Breaking the Softmax Bottleneck: A High-Rank RNN Language Model"

29 / 79 papers shown

Title
A Lightweight Recurrent Network for Sequence Modeling Biao Zhang Rico Sennrich 27 7 0 30 May 2019
Gmail Smart Compose: Real-Time Assisted Writing Mengzhao Chen Benjamin Lee G. Bansal Yuan Cao Shuyuan Zhang ... Yinan Wang Andrew M. Dai Z. Chen Timothy Sohn Yonghui Wu 16 203 0 17 May 2019
Language Modeling with Deep Transformers Kazuki Irie Albert Zeyer Ralf Schluter Hermann Ney KELM 41 172 0 10 May 2019
Language Models with Transformers Chenguang Wang Mu Li Alex Smola 15 120 0 20 Apr 2019
Knowledge Distillation For Recurrent Neural Network Language Modeling With Trust Regularization Yangyang Shi M. Hwang X. Lei Haoyu Sheng 31 25 0 08 Apr 2019
Calibration of Encoder Decoder Models for Neural Machine Translation Aviral Kumar Sunita Sarawagi 24 98 0 03 Mar 2019
Evaluating the Search Phase of Neural Architecture Search Kaicheng Yu C. Sciuto Martin Jaggi C. Musat Mathieu Salzmann 20 342 0 21 Feb 2019
Breaking the Softmax Bottleneck via Learnable Monotonic Pointwise Non-linearities O. Ganea Sylvain Gelly Gary Bécigneul Aliaksei Severyn 26 18 0 21 Feb 2019
Random Search and Reproducibility for Neural Architecture Search Liam Li Ameet Talwalkar OOD 33 717 0 20 Feb 2019
Error-Correcting Neural Sequence Prediction James OÑeill Danushka Bollegala 23 1 0 21 Jan 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 38 3,674 0 09 Jan 2019
Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks Songlin Yang Shawn Tan Alessandro Sordoni Aaron Courville 32 323 0 22 Oct 2018
Sequence to Sequence Mixture Model for Diverse Machine Translation Xuanli He Gholamreza Haffari Mohammad Norouzi 20 57 0 17 Oct 2018
Trellis Networks for Sequence Modeling Shaojie Bai J. Zico Kolter V. Koltun 25 145 0 15 Oct 2018
Understanding Recurrent Neural Architectures by Analyzing and Synthesizing Long Distance Dependencies in Benchmark Sequential Datasets Abhijit Mahalunkar John D. Kelleher 24 8 0 06 Oct 2018
Noise Contrastive Estimation and Negative Sampling for Conditional Models: Consistency and Statistical Efficiency Zhuang Ma Michael Collins 14 142 0 06 Sep 2018
Direct Output Connection for a High-Rank Language Model Sho Takase Jun Suzuki Masaaki Nagata 18 36 0 30 Aug 2018
Learning Neural Templates for Text Generation Sam Wiseman Stuart M. Shieber Alexander M. Rush 40 200 0 30 Aug 2018
Pyramidal Recurrent Unit for Language Modeling Sachin Mehta Rik Koncel-Kedziorski Mohammad Rastegari Hannaneh Hajishirzi 21 10 0 27 Aug 2018
Navigating with Graph Representations for Fast and Scalable Decoding of Neural Language Models Minjia Zhang Xiaodong Liu Wenhan Wang Jianfeng Gao Yuxiong He 23 30 0 11 Jun 2018
Relational recurrent neural networks Adam Santoro Ryan Faulkner David Raposo Jack W. Rae Mike Chrzanowski T. Weber Daan Wierstra Oriol Vinyals Razvan Pascanu Timothy Lillicrap GNN 30 209 0 05 Jun 2018
Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context Urvashi Khandelwal He He Peng Qi Dan Jurafsky RALM 16 293 0 12 May 2018
Noisin: Unbiased Regularization for Recurrent Neural Networks Adji Bousso Dieng Rajesh Ranganath Jaan Altosaar David M. Blei 22 22 0 03 May 2018
Fast Parametric Learning with Activation Memorization Jack W. Rae Chris Dyer Peter Dayan Timothy Lillicrap KELM 41 46 0 27 Mar 2018
An Analysis of Neural Language Modeling at Multiple Scales Stephen Merity N. Keskar R. Socher 24 170 0 22 Mar 2018
An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling Shaojie Bai J. Zico Kolter V. Koltun DRL 42 4,724 0 04 Mar 2018
Gradual Learning of Recurrent Neural Networks Ziv Aharoni Gal Rattner Haim Permuter AI4CE 27 4 0 29 Aug 2017
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 271 5,327 0 05 Nov 2016
Generalizing and Hybridizing Count-based and Neural Language Models Graham Neubig Chris Dyer 64 31 0 01 Jun 2016