Pretrained Language Models for Text Generation: A Survey

14 January 2022

Papers citing "Pretrained Language Models for Text Generation: A Survey"

37 / 137 papers shown

Title
DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation Yizhe Zhang Siqi Sun Michel Galley Yen-Chun Chen Chris Brockett Xiang Gao Jianfeng Gao Jingjing Liu W. Dolan VLM 169 1,523 0 01 Nov 2019
Unsupervised pre-training for sequence to sequence speech recognition Zhiyun Fan Shiyu Zhou Bo Xu SSL AI4TS 35 16 0 28 Oct 2019
Reducing Transformer Depth on Demand with Structured Dropout Angela Fan Edouard Grave Armand Joulin 117 592 0 25 Sep 2019
Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Models Cheolhyoung Lee Kyunghyun Cho Wanmo Kang MoE 271 208 0 25 Sep 2019
Cross-Lingual Natural Language Generation via Pre-Training Zewen Chi Li Dong Furu Wei Wenhui Wang Xian-Ling Mao Heyan Huang 76 137 0 23 Sep 2019
TinyBERT: Distilling BERT for Natural Language Understanding Xiaoqi Jiao Yichun Yin Lifeng Shang Xin Jiang Xiao Chen Linlin Li F. Wang Qun Liu VLM 100 1,860 0 23 Sep 2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism Mohammad Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 324 1,899 0 17 Sep 2019
Explicit Cross-lingual Pre-training for Unsupervised Machine Translation Shuo Ren Yu-Huan Wu Shujie Liu M. Zhou Shuai Ma 58 44 0 31 Aug 2019
Towards Making the Most of BERT in Neural Machine Translation Jiacheng Yang Mingxuan Wang Hao Zhou Chengqi Zhao Yong Yu Weinan Zhang Lei Li CLL 46 158 0 15 Aug 2019
Hello, It's GPT-2 -- How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems Paweł Budzianowski Ivan Vulić 62 310 0 12 Jul 2019
XLNet: Generalized Autoregressive Pretraining for Language Understanding Zhilin Yang Zihang Dai Yiming Yang J. Carbonell Ruslan Salakhutdinov Quoc V. Le AI4CE 230 8,426 0 19 Jun 2019
Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology Ran Zmigrod Sabrina J. Mielke Hanna M. Wallach Ryan Cotterell 61 282 0 11 Jun 2019
Generating Long and Informative Reviews with Aspect-Aware Coarse-to-Fine Decoding Junyi Li Wayne Xin Zhao Ji-Rong Wen Yang Song 57 37 0 11 Jun 2019
How multilingual is Multilingual BERT? Telmo Pires Eva Schlinger Dan Garrette LRM VLM 148 1,409 0 04 Jun 2019
ERNIE: Enhanced Language Representation with Informative Entities Zhengyan Zhang Xu Han Zhiyuan Liu Xin Jiang Maosong Sun Qun Liu 95 1,397 0 17 May 2019
HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization Xingxing Zhang Furu Wei M. Zhou 77 379 0 16 May 2019
Challenges in Building Intelligent Open-domain Dialog Systems Minlie Huang Xiaoyan Zhu Jianfeng Gao VLM 88 316 0 13 May 2019
Unified Language Model Pre-training for Natural Language Understanding and Generation Li Dong Nan Yang Wenhui Wang Furu Wei Xiaodong Liu Yu Wang Jianfeng Gao M. Zhou H. Hon ELM AI4CE 220 1,555 0 08 May 2019
MASS: Masked Sequence to Sequence Pre-training for Language Generation Kaitao Song Xu Tan Tao Qin Jianfeng Lu Tie-Yan Liu 110 965 0 07 May 2019
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems Alex Jinpeng Wang Yada Pruksachatkun Nikita Nangia Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 256 2,312 0 02 May 2019
Few-Shot NLG with Pre-Trained Language Model Zhiyu Zoey Chen H. Eavani Wenhu Chen Yinyin Liu William Yang Wang LMTD 63 142 0 21 Apr 2019
fairseq: A Fast, Extensible Toolkit for Sequence Modeling Myle Ott Sergey Edunov Alexei Baevski Angela Fan Sam Gross Nathan Ng David Grangier Michael Auli VLM FaML 103 3,149 0 01 Apr 2019
Learning Gender-Neutral Word Embeddings Jieyu Zhao Yichao Zhou Zeyu Li Wei Wang Kai-Wei Chang FaML 94 412 0 29 Aug 2018
Correcting Length Bias in Neural Machine Translation Kenton W. Murray David Chiang AIMat 67 157 0 29 Aug 2018
Human vs Automatic Metrics: on the Importance of Correlation Design Anastasia Shimorina HAI 34 14 0 29 May 2018
A Call for Clarity in Reporting BLEU Scores Matt Post 142 2,985 0 23 Apr 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,154 0 20 Apr 2018
Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation Matt Post David Vilar 68 314 0 18 Apr 2018
Texygen: A Benchmarking Platform for Text Generation Models Yaoming Zhu Sidi Lu Lei Zheng Jiaxian Guo Weinan Zhang Jun Wang Yong Yu 94 684 0 06 Feb 2018
Non-Autoregressive Neural Machine Translation Jiatao Gu James Bradbury Caiming Xiong Victor O.K. Li R. Socher 97 795 0 07 Nov 2017
A Deep Reinforced Model for Abstractive Summarization Romain Paulus Caiming Xiong R. Socher AI4TS 197 1,557 0 11 May 2017
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension Mandar Joshi Eunsol Choi Daniel S. Weld Luke Zettlemoyer RALM 204 2,646 0 09 May 2017
Get To The Point: Summarization with Pointer-Generator Networks A. See Peter J. Liu Christopher D. Manning 3DPC 293 4,019 0 14 Apr 2017
Overcoming catastrophic forgetting in neural networks J. Kirkpatrick Razvan Pascanu Neil C. Rabinowitz J. Veness Guillaume Desjardins ... A. Grabska-Barwinska Demis Hassabis Claudia Clopath D. Kumaran R. Hadsell CLL 354 7,504 0 02 Dec 2016
Microsoft COCO Captions: Data Collection and Evaluation Server Xinlei Chen Hao Fang Nayeon Lee Ramakrishna Vedantam Saurabh Gupta Piotr Dollar C. L. Zitnick 211 2,475 0 01 Apr 2015
Sequence to Sequence Learning with Neural Networks Ilya Sutskever Oriol Vinyals Quoc V. Le AIMat 434 20,553 0 10 Sep 2014
Neural Machine Translation by Jointly Learning to Align and Translate Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio AIMat 549 27,300 0 01 Sep 2014