MASS: Masked Sequence to Sequence Pre-training for Language Generation

7 May 2019

Xu Tan

Papers citing "MASS: Masked Sequence to Sequence Pre-training for Language Generation"

50 / 218 papers shown

Title
Enhanced Seq2Seq Autoencoder via Contrastive Learning for Abstractive Text Summarization Chujie Zheng Kunpeng Zhang Harry J. Wang Ling Fan Zhe Wang 25 6 0 26 Aug 2021
Alleviating Exposure Bias via Contrastive Learning for Abstractive Text Summarization Shichao Sun Wenjie Li 19 26 0 26 Aug 2021
Auxiliary Task Update Decomposition: The Good, The Bad and The Neutral Lucio Dery Yann N. Dauphin David Grangier MoMe 18 29 0 25 Aug 2021
Regularizing Transformers With Deep Probabilistic Layers Aurora Cobo Aguilera Pablo Martínez Olmos Antonio Artés-Rodríguez Fernando Pérez-Cruz 41 7 0 23 Aug 2021
MvSR-NAT: Multi-view Subset Regularization for Non-Autoregressive Machine Translation Pan Xie Zexian Li Xiaohui Hu 34 11 0 19 Aug 2021
PARADISE: Exploiting Parallel Data for Multilingual Sequence-to-Sequence Pretraining Machel Reid Mikel Artetxe VLM 50 26 0 04 Aug 2021
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing Pengfei Liu Weizhe Yuan Jinlan Fu Zhengbao Jiang Hiroaki Hayashi Graham Neubig VLM SyDa 55 3,838 0 28 Jul 2021
Back-Translated Task Adaptive Pretraining: Improving Accuracy and Robustness on Text Classification Junghoon Lee Jounghee Kim Pilsung Kang VLM 13 5 0 22 Jul 2021
A Survey on Low-Resource Neural Machine Translation Rui Wang Xu Tan Renqian Luo Tao Qin Tie-Yan Liu 3DV 33 58 0 09 Jul 2021
DeepRapper: Neural Rap Generation with Rhyme and Rhythm Modeling Lanqing Xue Kaitao Song Duocai Wu Xu Tan N. Zhang Tao Qin Weiqiang Zhang Tie-Yan Liu 37 37 0 05 Jul 2021
ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information Zijun Sun Xiaoya Li Xiaofei Sun Yuxian Meng Xiang Ao Qing He Fei Wu Jiwei Li SSeg 57 183 0 30 Jun 2021
The Values Encoded in Machine Learning Research Abeba Birhane Pratyusha Kalluri Dallas Card William Agnew Ravit Dotan Michelle Bao 41 274 0 29 Jun 2021
SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption Dara Bahri Heinrich Jiang Yi Tay Donald Metzler SSL 19 163 0 29 Jun 2021
Neural Machine Translation for Low-Resource Languages: A Survey Surangika Ranathunga E. Lee Marjana Prifti Skenduli Ravi Shekhar Mehreen Alam Rishemjit Kaur 38 236 0 29 Jun 2021
Pre-Trained Models: Past, Present and Future Xu Han Zhengyan Zhang Ning Ding Yuxian Gu Xiao Liu ... Jie Tang Ji-Rong Wen Jinhui Yuan Wayne Xin Zhao Jun Zhu AIFin MQ AI4MH 58 815 0 14 Jun 2021
MusicBERT: Symbolic Music Understanding with Large-Scale Pre-Training Mingliang Zeng Xu Tan Rui Wang Zeqian Ju Tao Qin Tie-Yan Liu 19 128 0 10 Jun 2021
Crosslingual Embeddings are Essential in UNMT for Distant Languages: An English to IndoAryan Case Study Tamali Banerjee V. Rudra Murthy P. Bhattacharyya 35 9 0 09 Jun 2021
A Unified Generative Framework for Various NER Subtasks Hang Yan Tao Gui Junqi Dai Qipeng Guo Zheng-Wei Zhang Xipeng Qiu 34 288 0 02 Jun 2021
Focus Attention: Promoting Faithfulness and Diversity in Summarization Rahul Aralikatte Shashi Narayan Joshua Maynez S. Rothe Ryan T. McDonald 35 45 0 25 May 2021
Should We Trust This Summary? Bayesian Abstractive Summarization to The Rescue Alexios Gidiotis Grigorios Tsoumakas UQCV UD BDL 22 9 0 21 May 2021
Contrastive Learning for Many-to-many Multilingual Neural Machine Translation Xiao Pan Mingxuan Wang Liwei Wu Lei Li 18 200 0 20 May 2021
SeaD: End-to-end Text-to-SQL Generation with Schema-aware Denoising K. Xuan Yongbo Wang Yongliang Wang Zujie Wen Yang Dong VLM 38 52 0 17 May 2021
Can You Traducir This? Machine Translation for Code-Switched Input Jitao Xu François Yvon 20 30 0 11 May 2021
FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognition Yichong Leng Xu Tan Linchen Zhu Jin Xu Renqian Luo Linquan Liu Tao Qin Xiang-Yang Li Ed Lin Tie-Yan Liu KELM 24 63 0 09 May 2021
Extract, Denoise and Enforce: Evaluating and Improving Concept Preservation for Text-to-Text Generation Yuning Mao Wenchang Ma Deren Lei Jiawei Han Xiang Ren 29 4 0 18 Apr 2021
K-PLUG: Knowledge-injected Pre-trained Language Model for Natural Language Understanding and Generation in E-Commerce Song Xu Haoran Li Peng Yuan Yujia Wang Youzheng Wu Xiaodong He Ying Liu Bowen Zhou KELM 35 24 0 14 Apr 2021
A New Approach to Overgenerating and Scoring Abstractive Summaries Kaiqiang Song Bingqing Wang Z. Feng Fei Liu 22 17 0 05 Apr 2021
Inference Time Style Control for Summarization Shuyang Cao Lu Wang AI4TS 26 15 0 05 Apr 2021
Mask Attention Networks: Rethinking and Strengthen Transformer Zhihao Fan Yeyun Gong Dayiheng Liu Zhongyu Wei Siyuan Wang Jian Jiao Nan Duan Ruofei Zhang Xuanjing Huang 34 72 0 25 Mar 2021
The NLP Cookbook: Modern Recipes for Transformer based Deep Learning Architectures Sushant Singh A. Mahmood AI4TS 60 92 0 23 Mar 2021
Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglots Samson Tan Chenyu You AAML 29 35 0 17 Mar 2021
Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution Xavier Garcia Noah Constant Ankur P. Parikh Orhan Firat 48 42 0 11 Mar 2021
MalBERT: Using Transformers for Cybersecurity and Malicious Software Detection Abir Rahali M. Akhloufi 29 30 0 05 Mar 2021
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 299 1,084 0 17 Feb 2021
DOBF: A Deobfuscation Pre-Training Objective for Programming Languages Baptiste Roziere Marie-Anne Lachaux Marc Szafraniec Guillaume Lample AI4CE 52 137 0 15 Feb 2021
Proof Artifact Co-training for Theorem Proving with Language Models Jesse Michael Han Jason M. Rute Yuhuai Wu Edward W. Ayers Stanislas Polu AIMat 25 120 0 11 Feb 2021
Unifying Vision-and-Language Tasks via Text Generation Jaemin Cho Jie Lei Hao Tan Joey Tianyi Zhou MLLM 277 525 0 04 Feb 2021
Outline to Story: Fine-grained Controllable Story Generation from Cascaded Events Le Fang Tao Zeng Chao-Ning Liu Liefeng Bo Wen Dong Changyou Chen 35 12 0 04 Jan 2021
Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting Wangchunshu Zhou Tao Ge Canwen Xu Ke Xu Furu Wei LRM 16 15 0 02 Jan 2021
Intent Classification and Slot Filling for Privacy Policies Wasi Uddin Ahmad Jianfeng Chi Tu Le Thomas B. Norton Yuan Tian Kai-Wei Chang 13 23 0 01 Jan 2021
MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers Wenhui Wang Hangbo Bao Shaohan Huang Li Dong Furu Wei MQ 24 257 0 31 Dec 2020
XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders Shuming Ma Jian Yang Haoyang Huang Zewen Chi Li Dong ... Akiko Eriguchi Saksham Singhal Xia Song Arul Menezes Furu Wei LRM 26 33 0 31 Dec 2020
Neural Machine Translation: A Review of Methods, Resources, and Tools Zhixing Tan Shuo Wang Zonghan Yang Gang Chen Xuancheng Huang Maosong Sun Yang Liu 3DV AI4TS 19 105 0 31 Dec 2020
CLEAR: Contrastive Learning for Sentence Representation Zhuofeng Wu Sinong Wang Jiatao Gu Madian Khabsa Fei Sun Hao Ma SSL 33 319 0 31 Dec 2020
ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning Yujia Qin Yankai Lin Ryuichi Takanobu Zhiyuan Liu Peng Li Heng Ji Minlie Huang Maosong Sun Jie Zhou 55 125 0 30 Dec 2020
Graph-Evolving Meta-Learning for Low-Resource Medical Dialogue Generation Shuai Lin Pan Zhou Xiaodan Liang Jianheng Tang Ruihui Zhao Ziliang Chen Liang Lin MedIm 33 53 0 22 Dec 2020
LRC-BERT: Latent-representation Contrastive Knowledge Distillation for Natural Language Understanding Hao Fu Shaojun Zhou Qihong Yang Junjie Tang Guiquan Liu Kaikui Liu Xiaolong Li 52 57 0 14 Dec 2020
Contrastive Learning with Adversarial Perturbations for Conditional Text Generation Seanie Lee Dong Bok Lee Sung Ju Hwang 27 106 0 14 Dec 2020
GLGE: A New General Language Generation Evaluation Benchmark Dayiheng Liu Yu Yan Yeyun Gong Weizhen Qi Hang Zhang ... Jiancheng Lv Ruofei Zhang Winnie Wu Ming Zhou Nan Duan ELM 40 66 0 24 Nov 2020
Multilingual AMR-to-Text Generation Angela Fan Claire Gardent 17 32 0 10 Nov 2020