Representation Degeneration Problem in Training Natural Language Generation Models

28 July 2019

Xu Tan

Papers citing "Representation Degeneration Problem in Training Natural Language Generation Models"

29 / 29 papers shown

Title
Accurate and Efficient Multivariate Time Series Forecasting via Offline Clustering Yiming Niu Jinliang Deng Lulu Zhang Zimu Zhou Yongxin Tong AI4TS 163 0 0 09 May 2025
Implicit Geometry of Next-token Prediction: From Language Sparsity Patterns to Model Representations Yize Zhao Tina Behnia V. Vakilian Christos Thrampoulidis 187 10 0 20 Feb 2025
DEUCE: Dual-diversity Enhancement and Uncertainty-awareness for Cold-start Active Learning Jiaxin Guo Cheng Chen Shuzhen Li Tianze Zhang 127 0 0 01 Feb 2025
PowerNorm: Rethinking Batch Normalization in Transformers Sheng Shen Z. Yao A. Gholami Michael W. Mahoney Kurt Keutzer BDL 89 16 0 17 Mar 2020
FRAGE: Frequency-Agnostic Word Representation Chengyue Gong Di He Xu Tan Tao Qin Liwei Wang Tie-Yan Liu OOD 85 144 0 18 Sep 2018
Tensor2Tensor for Neural Machine Translation Ashish Vaswani Samy Bengio E. Brevdo François Chollet Aidan Gomez ... Nal Kalchbrenner Niki Parmar Ryan Sepassi Noam M. Shazeer Jakob Uszkoreit 98 530 0 16 Mar 2018
Breaking the Softmax Bottleneck: A High-Rank RNN Language Model Zhilin Yang Zihang Dai Ruslan Salakhutdinov William W. Cohen BDL 87 373 0 10 Nov 2017
Regularizing and Optimizing LSTM Language Models Stephen Merity N. Keskar R. Socher 178 1,096 0 07 Aug 2017
Learned in Translation: Contextualized Word Vectors Bryan McCann James Bradbury Caiming Xiong R. Socher 126 910 0 01 Aug 2017
Dual Supervised Learning Yingce Xia Tao Qin Wei-neng Chen Jiang Bian Nenghai Yu Tie-Yan Liu SSL 134 143 0 03 Jul 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 814 132,725 0 12 Jun 2017
Convolutional Sequence to Sequence Learning Jonas Gehring Michael Auli David Grangier Denis Yarats Yann N. Dauphin AIMat 174 3,292 0 08 May 2017
Stochastic Gradient Descent as Approximate Bayesian Inference Stephan Mandt Matthew D. Hoffman David M. Blei BDL 71 599 0 13 Apr 2017
Massive Exploration of Neural Machine Translation Architectures D. Britz Anna Goldie Minh-Thang Luong Quoc V. Le 76 519 0 11 Mar 2017
All-but-the-Top: Simple and Effective Postprocessing for Word Representations Jiaqi Mu S. Bhat Pramod Viswanath 97 311 0 05 Feb 2017
Language Modeling with Gated Convolutional Networks Yann N. Dauphin Angela Fan Michael Auli David Grangier 247 2,410 0 23 Dec 2016
Improving Neural Language Models with a Continuous Cache Edouard Grave Armand Joulin Nicolas Usunier KELM 69 302 0 13 Dec 2016
Large-Margin Softmax Loss for Convolutional Neural Networks Weiyang Liu Yandong Wen Zhiding Yu Meng Yang CVBM 84 1,457 0 07 Dec 2016
Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling Hakan Inan Khashayar Khosravi R. Socher 128 385 0 04 Nov 2016
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Zhiwen Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 918 6,799 0 26 Sep 2016
Pointer Sentinel Mixture Models Stephen Merity Caiming Xiong James Bradbury R. Socher RALM 352 2,900 0 26 Sep 2016
Using the Output Embedding to Improve Language Models Ofir Press Lior Wolf 104 736 0 20 Aug 2016
Layer Normalization Jimmy Lei Ba J. Kiros Geoffrey E. Hinton 437 10,548 0 21 Jul 2016
Exploring the Limits of Language Modeling Rafal Jozefowicz Oriol Vinyals M. Schuster Noam M. Shazeer Yonghui Wu 206 1,145 0 07 Feb 2016
Neural Machine Translation of Rare Words with Subword Units Rico Sennrich Barry Haddow Alexandra Birch 244 7,765 0 31 Aug 2015
Character-Aware Neural Language Models Yoon Kim Yacine Jernite David Sontag Alexander M. Rush 113 1,671 0 26 Aug 2015
Neural Responding Machine for Short-Text Conversation Lifeng Shang Zhengdong Lu Hang Li 121 1,146 0 09 Mar 2015
Neural Machine Translation by Jointly Learning to Align and Translate Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio AIMat 587 27,345 0 01 Sep 2014
Distributed Representations of Words and Phrases and their Compositionality Tomas Mikolov Ilya Sutskever Kai Chen G. Corrado J. Dean NAI OCL 406 33,573 0 16 Oct 2013