Pretrained Language Models for Text Generation: A Survey

14 January 2022

Papers citing "Pretrained Language Models for Text Generation: A Survey"

50 / 137 papers shown

Title
Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP Timo Schick Sahana Udupa Hinrich Schütze 306 385 0 28 Feb 2021
VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioning Jun Chen Han Guo Kai Yi Boyang Albert Li Mohamed Elhoseiny VLM 85 226 0 20 Feb 2021
Meta-Transfer Learning for Low-Resource Abstractive Summarization Yi-Syuan Chen Hong-Han Shuai CLL OffRL 87 39 0 18 Feb 2021
TextBox: A Unified, Modularized, and Extensible Framework for Text Generation Junyi Li Tianyi Tang Gaole He Jinhao Jiang Xiaoxuan Hu Puzhao Xie Zhipeng Chen Zhuohao Yu Wayne Xin Zhao Ji-Rong Wen 64 25 0 06 Jan 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 223 4,254 0 01 Jan 2021
TAP: Text-Aware Pre-training for Text-VQA and Text-Caption Zhengyuan Yang Yijuan Lu Jianfeng Wang Xi Yin D. Florêncio Lijuan Wang Cha Zhang Lei Zhang Jiebo Luo VLM 79 144 0 08 Dec 2020
DialogBERT: Discourse-Aware Response Generation via Learning to Recover and Rank Utterances X. Gu Kang Min Yoo Jung-Woo Ha 47 73 0 03 Dec 2020
CPM: A Large-scale Generative Chinese Pre-trained Language Model Zhengyan Zhang Xu Han Hao Zhou Pei Ke Yuxian Gu ... Wentao Han Jie Tang Juan-Zi Li Xiaoyan Zhu Maosong Sun 61 117 0 01 Dec 2020
Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning Beliz Gunel Jingfei Du Alexis Conneau Ves Stoyanov 60 505 0 03 Nov 2020
VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generation Fuli Luo Wei Wang Jiahao Liu Yijia Liu Bin Bi Songfang Huang Fei Huang Luo Si 66 51 0 30 Oct 2020
Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentation Alexander R. Fabbri Simeng Han Haoyuan Li Haoran Li Marjan Ghazvininejad Shafiq Joty Dragomir R. Radev Yashar Mehdad 177 97 0 24 Oct 2020
Pre-training Text-to-Text Transformers for Concept-centric Common Sense Wangchunshu Zhou Dong-Ho Lee Ravi Kiran Selvam Seyeon Lee Bill Yuchen Lin Xiang Ren LRM VLM 50 72 0 24 Oct 2020
Consistency and Coherency Enhanced Story Generation Wei Wang Piji Li Haitao Zheng 47 11 0 17 Oct 2020
GSum: A General Framework for Guided Neural Abstractive Summarization Zi-Yi Dou Pengfei Liu Hiroaki Hayashi Zhengbao Jiang Graham Neubig 60 258 0 15 Oct 2020
Measuring and Reducing Gendered Correlations in Pre-trained Models Kellie Webster Xuezhi Wang Ian Tenney Alex Beutel Emily Pitler Ellie Pavlick Jilin Chen Ed Chi Slav Petrov FaML 72 258 0 12 Oct 2020
Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information Zehui Lin Xiao Pan Mingxuan Wang Xipeng Qiu Jiangtao Feng Hao Zhou Lei Li 55 127 0 07 Oct 2020
StyleDGPT: Stylized Response Generation with Pre-trained Language Models Ze Yang Wei Wu Can Xu Xinnian Liang Jiaqi Bai Liran Wang Wei Wang Zhoujun Li VLM 92 25 0 06 Oct 2020
MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Models Peng Xu M. Patwary Mohammad Shoeybi Raul Puri Pascale Fung Anima Anandkumar Bryan Catanzaro 62 128 0 02 Oct 2020
KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoning Ye Liu Yao Wan Lifang He Hao Peng Philip S. Yu 81 189 0 26 Sep 2020
ConvBERT: Improving BERT with Span-based Dynamic Convolution Zihang Jiang Weihao Yu Daquan Zhou Yunpeng Chen Jiashi Feng Shuicheng Yan 72 159 0 06 Aug 2020
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 540 2,081 0 28 Jul 2020
Investigating Pretrained Language Models for Graph-to-Text Generation Leonardo F. R. Ribeiro Martin Schmitt Hinrich Schütze Iryna Gurevych 44 218 0 16 Jul 2020
PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning Siqi Bao H. He Fan Wang Hua Wu Haifeng Wang Wenquan Wu Zhen Guo Zhibin Liu Xinchao Xu 78 138 0 30 Jun 2020
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding Dmitry Lepikhin HyoukJoong Lee Yuanzhong Xu Dehao Chen Orhan Firat Yanping Huang M. Krikun Noam M. Shazeer Zhiwen Chen MoE 89 1,162 0 30 Jun 2020
Evaluation of Text Generation: A Survey Asli Celikyilmaz Elizabeth Clark Jianfeng Gao ELM LM&MA 104 385 0 26 Jun 2020
ColdGANs: Taming Language GANs with Cautious Sampling Strategies Thomas Scialom Paul-Alexis Dray Sylvain Lamprier Benjamin Piwowarski Jacopo Staiano GAN SyDa 52 18 0 08 Jun 2020
DeBERTa: Decoding-enhanced BERT with Disentangled Attention Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen AAML 138 2,731 0 05 Jun 2020
GOBO: Quantizing Attention-Based NLP Models for Low Latency and Energy Efficient Inference Ali Hadi Zadeh Isak Edo Omar Mohamed Awad Andreas Moshovos MQ 57 188 0 08 May 2020
PlotMachines: Outline-Conditioned Generation with Dynamic Plot State Tracking Hannah Rashkin Asli Celikyilmaz Yejin Choi Jianfeng Gao 54 153 0 30 Apr 2020
Recipes for Adapting Pre-trained Monolingual and Multilingual Models to Machine Translation Asa Cooper Stickland Xian Li Marjan Ghazvininejad 68 46 0 30 Apr 2020
Logic2Text: High-Fidelity Natural Language Generation from Logical Forms Zhiyu Zoey Chen Wenhu Chen Hanwen Zha Xiyou Zhou Yunkai Zhang Sairam Sundaresan William Yang Wang NAI 58 66 0 30 Apr 2020
SongNet: Rigid Formats Controlled Text Generation Piji Li Haisong Zhang Xiaojiang Liu Shuming Shi 99 54 0 17 Apr 2020
Training with Quantization Noise for Extreme Model Compression Angela Fan Pierre Stock Benjamin Graham Edouard Grave Remi Gribonval Hervé Jégou Armand Joulin MQ 90 245 0 15 Apr 2020
DynaBERT: Dynamic BERT with Adaptive Width and Depth Lu Hou Zhiqi Huang Lifeng Shang Xin Jiang Xiao Chen Qun Liu MQ 79 322 0 08 Apr 2020
Have Your Text and Use It Too! End-to-End Neural Data-to-Text Generation with Semantic Fidelity Hamza Harkous Isabel Groves Amir Saffari 63 89 0 08 Apr 2020
Reference Language based Unsupervised Neural Machine Translation Z. Li Hai Zhao Rui Wang Masao Utiyama Eiichiro Sumita 36 27 0 05 Apr 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 356 1,484 0 18 Mar 2020
Hybrid Generative-Retrieval Transformers for Dialogue Domain Adaptation Igor Shalyminov Alessandro Sordoni Adam Atkinson Hannes Schulz VLM 43 13 0 03 Mar 2020
XGPT: Cross-modal Generative Pre-Training for Image Captioning Qiaolin Xia Haoyang Huang Nan Duan Dongdong Zhang Lei Ji Zhifang Sui Edward Cui Taroon Bharti Xin Liu Ming Zhou MLLM VLM 70 75 0 03 Mar 2020
Few-shot Natural Language Generation for Task-Oriented Dialog Baolin Peng Chenguang Zhu Chunyuan Li Xiujun Li Jinchao Li Michael Zeng Jianfeng Gao 75 201 0 27 Feb 2020
CodeBERT: A Pre-Trained Model for Programming and Natural Languages Zhangyin Feng Daya Guo Duyu Tang Nan Duan Xiaocheng Feng ... Linjun Shou Bing Qin Ting Liu Daxin Jiang Ming Zhou 160 2,633 0 19 Feb 2020
Incorporating BERT into Neural Machine Translation Jinhua Zhu Yingce Xia Lijun Wu Di He Tao Qin Wen-gang Zhou Houqiang Li Tie-Yan Liu FedML AIMat 42 359 0 17 Feb 2020
A Multilingual View of Unsupervised Machine Translation Xavier Garcia Pierre Foret Thibault Sellam Ankur P. Parikh 79 37 0 07 Feb 2020
Towards a Human-like Open-Domain Chatbot Daniel De Freitas Minh-Thang Luong David R. So Jamie Hall Noah Fiedel ... Zi Yang Apoorv Kulshreshtha Gaurav Nemade Yifeng Lu Quoc V. Le 99 935 0 27 Jan 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 599 4,801 0 23 Jan 2020
A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation Jian Guan Fei Huang Zhihao Zhao Xiaoyan Zhu Minlie Huang LRM SyDa 54 247 0 15 Jan 2020
ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training Weizhen Qi Yu Yan Yeyun Gong Dayiheng Liu Nan Duan Jiusheng Chen Ruofei Zhang Ming Zhou AI4TS 69 450 0 13 Jan 2020
PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization Jingqing Zhang Yao-Min Zhao Mohammad Saleh Peter J. Liu RALM 3DGS 273 2,048 0 18 Dec 2019
Plug and Play Language Models: A Simple Approach to Controlled Text Generation Sumanth Dathathri Andrea Madotto Janice Lan Jane Hung Eric Frank Piero Molino J. Yosinski Rosanne Liu KELM 127 969 0 04 Dec 2019
How Can We Know What Language Models Know? Zhengbao Jiang Frank F. Xu Jun Araki Graham Neubig KELM 130 1,405 0 28 Nov 2019