MASS: Masked Sequence to Sequence Pre-training for Language Generation

7 May 2019

Xu Tan

Papers citing "MASS: Masked Sequence to Sequence Pre-training for Language Generation"

50 / 218 papers shown

Title
Multimodal Pretraining for Dense Video Captioning Gabriel Huang Bo Pang Zhenhai Zhu Clara E. Rivera Radu Soricut 21 81 0 10 Nov 2020
Emergent Communication Pretraining for Few-Shot Machine Translation Yaoyiran Li E. Ponti Ivan Vulić Anna Korhonen 25 19 0 02 Nov 2020
The LMU Munich System for the WMT 2020 Unsupervised Machine Translation Shared Task Alexandra Chronopoulou Dario Stojanovski Viktor Hangya Alexander Fraser 37 5 0 25 Oct 2020
Unsupervised Neural Machine Translation for Low-Resource Domains via Meta-Learning Cheonbok Park Yunwon Tae Taehee Kim Soyoung Yang Mohammad Azam Khan Lucy Park Jaegul Choo 102 17 0 18 Oct 2020
Unsupervised Bitext Mining and Translation via Self-trained Contextual Embeddings Phillip Keung Julian Salazar Y. Lu Noah A. Smith SSL 27 25 0 15 Oct 2020
Plug and Play Autoencoders for Conditional Text Generation Florian Mai Nikolaos Pappas Ivan Montero Noah A. Smith U. Washington 25 36 0 06 Oct 2020
Improving AMR Parsing with Sequence-to-Sequence Pre-training Dong Xu Junhui Li Muhua Zhu Min Zhang Guodong Zhou AIMat 18 68 0 05 Oct 2020
On Losses for Modern Language Models Stephane Aroca-Ouellette Frank Rudzicz 19 33 0 04 Oct 2020
Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space Dayiheng Liu Yeyun Gong Jie Fu Yu Yan Jiusheng Chen Jiancheng Lv Nan Duan M. Zhou 15 37 0 04 Oct 2020
Examining the rhetorical capacities of neural language models Zining Zhu Chuer Pan Mohamed Abdalla Frank Rudzicz 33 10 0 01 Oct 2020
KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoning Ye Liu Yao Wan Lifang He Hao Peng Philip S. Yu 32 188 0 26 Sep 2020
Interest-Behaviour Multiplicative Network for Resource-limited Recommendation Qianliang Wu Tong Zhang Zhen Cui Jian Yang 19 1 0 24 Sep 2020
Harnessing Multilinguality in Unsupervised Machine Translation for Rare Languages Xavier Garcia Aditya Siddhant Orhan Firat Ankur P. Parikh 30 31 0 23 Sep 2020
Softmax Tempering for Training Neural Machine Translation Models Raj Dabre Atsushi Fujita 28 11 0 20 Sep 2020
Code-switching pre-training for neural machine translation Zhen Yang Bojie Hu Ambyera Han Shen Huang Qi Ju 27 71 0 17 Sep 2020
Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT Alexandra Chronopoulou Dario Stojanovski Alexander Fraser 18 33 0 16 Sep 2020
Learning to summarize from human feedback Nisan Stiennon Long Ouyang Jeff Wu Daniel M. Ziegler Ryan J. Lowe Chelsea Voss Alec Radford Dario Amodei Paul Christiano ALM 40 1,984 0 02 Sep 2020
Multilingual Translation with Extensible Multilingual Pretraining and Finetuning Y. Tang C. Tran Xian Li Peng-Jen Chen Naman Goyal Vishrav Chaudhary Jiatao Gu Angela Fan CLL 52 445 0 02 Aug 2020
CoreGen: Contextualized Code Representation Learning for Commit Message Generation L. Nie Cuiyun Gao Zhicong Zhong Wai Lam Yang Liu Zenglin Xu 21 46 0 14 Jul 2020
Cross-lingual Retrieval for Iterative Self-Supervised Training C. Tran Y. Tang Xian Li Jiatao Gu RALM 28 72 0 16 Jun 2020
Unsupervised Translation of Programming Languages Marie-Anne Lachaux Baptiste Roziere L. Chanussot Guillaume Lample 45 409 0 05 Jun 2020
Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing Zihang Dai Guokun Lai Yiming Yang Quoc V. Le 48 229 0 05 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 71 40,200 0 28 May 2020
Stronger Baselines for Grammatical Error Correction Using Pretrained Encoder-Decoder Model Satoru Katsumata Mamoru Komachi 30 53 0 24 May 2020
A Generative Approach to Titling and Clustering Wikipedia Sections Anjalie Field S. Rothe Simon Baumgartner Cong Yu Abe Ittycheriah 26 4 0 22 May 2020
Unsupervised Multimodal Neural Machine Translation with Pseudo Visual Pivoting Po-Yao (Bernie) Huang Junjie Hu Xiaojun Chang Alexander G. Hauptmann 36 50 0 06 May 2020
Improving Truthfulness of Headline Generation Kazuki Matsumaru Sho Takase Naoaki Okazaki HILM 6 49 0 02 May 2020
Recipes for Adapting Pre-trained Monolingual and Multilingual Models to Machine Translation Asa Cooper Stickland Xian Li Marjan Ghazvininejad 36 44 0 30 Apr 2020
Conditional Augmentation for Aspect Term Extraction via Masked Sequence-to-Sequence Generation Kun Li Chengbo Chen Xiaojun Quan Qing Ling Yan Song 35 95 0 30 Apr 2020
Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning Alexandre Tamborrino Nicola Pellicanò B. Pannier Pascal Voitot Louise Naudin LRM 19 62 0 29 Apr 2020
QURIOUS: Question Generation Pretraining for Text Generation Shashi Narayan Gonçalo Simães Ji Ma Hannah Craighead Ryan T. McDonald 37 15 0 23 Apr 2020
When and Why is Unsupervised Neural Machine Translation Useless? Yunsu Kim Miguel Graça Hermann Ney SSL 25 70 0 22 Apr 2020
Asking and Answering Questions to Evaluate the Factual Consistency of Summaries Alex Jinpeng Wang Kyunghyun Cho M. Lewis HILM 15 472 0 08 Apr 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 243 1,452 0 18 Mar 2020
A Survey on Contextual Embeddings Qi Liu Matt J. Kusner Phil Blunsom 225 146 0 16 Mar 2020
XGPT: Cross-modal Generative Pre-Training for Image Captioning Qiaolin Xia Haoyang Huang Nan Duan Dongdong Zhang Lei Ji Zhifang Sui Edward Cui Taroon Bharti Xin Liu Ming Zhou MLLM VLM 25 74 0 03 Mar 2020
Do all Roads Lead to Rome? Understanding the Role of Initialization in Iterative Back-Translation Mikel Artetxe Gorka Labaka Noe Casas Eneko Agirre LRM 29 5 0 28 Feb 2020
MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers Wenhui Wang Furu Wei Li Dong Hangbo Bao Nan Yang Ming Zhou VLM 47 1,203 0 25 Feb 2020
Generating Representative Headlines for News Stories Xiaotao Gu Yuning Mao Jiawei Han Jialu Liu Hongkun Yu You Wu Cong Yu Daniel Finnie Jiaqi Zhai Nicholas Zukoski 30 70 0 26 Jan 2020
Multilingual Denoising Pre-training for Neural Machine Translation Yinhan Liu Jiatao Gu Naman Goyal Xian Li Sergey Edunov Marjan Ghazvininejad M. Lewis Luke Zettlemoyer AI4CE AIMat 49 1,769 0 22 Jan 2020
ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training Weizhen Qi Yu Yan Yeyun Gong Dayiheng Liu Nan Duan Jiusheng Chen Ruofei Zhang Ming Zhou AI4TS 27 446 0 13 Jan 2020
A Study of Multilingual Neural Machine Translation Xu Tan Yichong Leng Jiale Chen Yi Ren Tao Qin Tie-Yan Liu 24 8 0 25 Dec 2019
PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization Jingqing Zhang Yao-Min Zhao Mohammad Saleh Peter J. Liu RALM 3DGS 45 2,018 0 18 Dec 2019
Fine-Tuning by Curriculum Learning for Non-Autoregressive Neural Machine Translation Junliang Guo Xu Tan Linli Xu Tao Qin Enhong Chen Tie-Yan Liu 14 85 0 20 Nov 2019
Weakly-Supervised Video Moment Retrieval via Semantic Completion Network Zhijie Lin Zhou Zhao Zhu Zhang Qi. Wang Huasheng Liu 22 149 0 19 Nov 2019
Generating Persona Consistent Dialogues by Exploiting Natural Language Inference Haoyu Song Weinan Zhang Jingwen Hu Ting Liu 27 73 0 14 Nov 2019
Microsoft Research Asia's Systems for WMT19 Yingce Xia Xu Tan Fei Tian Fei Gao Weicong Chen ... Yiren Wang Lijun Wu Jinhua Zhu Tao Qin Tie-Yan Liu VLM 24 26 0 07 Nov 2019
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension M. Lewis Yinhan Liu Naman Goyal Marjan Ghazvininejad Abdel-rahman Mohamed Omer Levy Veselin Stoyanov Luke Zettlemoyer AIMat VLM 41 10,609 0 29 Oct 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 118 19,493 0 23 Oct 2019
A Mutual Information Maximization Perspective of Language Representation Learning Lingpeng Kong Cyprien de Masson dÁutume Wang Ling Lei Yu Zihang Dai Dani Yogatama SSL 226 165 0 18 Oct 2019