Exploration of Masked and Causal Language Modelling for Text Generation

Exploration of Masked and Causal Language Modelling for Text Generation

21 May 2024

Nicolo Micheletti

ArXiv (abs)PDF HTML

Papers citing "Exploration of Masked and Causal Language Modelling for Text Generation"

15 / 15 papers shown

Title
Generating Synthetic Free-text Medical Records with Low Re-identification Risk using Masked Language Modeling Samuel Belkadi Libo Ren Nicolo Micheletti Lifeng Han Goran Nenadic SyDa MedIm 77 0 0 15 Sep 2024
A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond Yisheng Xiao Lijun Wu Junliang Guo Juntao Li Hao Fei Tao Qin Tie-Yan Liu 3DV MedIm AI4CE 87 87 0 20 Apr 2022
ExBERT: An External Knowledge Enhanced BERT for Natural Language Inference Amit Gajbhiye Noura Al Moubayed S. Bradley 57 10 0 03 Aug 2021
A Survey on Data Augmentation for Text Classification Markus Bayer M. Kaufhold Christian A. Reuter 145 354 0 07 Jul 2021
SciFive: a text-to-text transformer model for biomedical literature Long Phan J. Anibal H. Tran Shaurya Chanana Erol Bahadroglu Alec Peltekian G. Altan-Bonnet MedIm 62 151 0 28 May 2021
A Survey of Data Augmentation Approaches for NLP Steven Y. Feng Varun Gangal Jason W. Wei Sarath Chandar Soroush Vosoughi Teruko Mitamura Eduard H. Hovy AIMat 108 828 0 07 May 2021
Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing Yu Gu Robert Tinn Hao Cheng Michael R. Lucas Naoto Usuyama Xiaodong Liu Tristan Naumann Jianfeng Gao Hoifung Poon LM&MA AI4CE 131 1,785 0 31 Jul 2020
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 582 2,105 0 28 Jul 2020
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 506 20,376 0 23 Oct 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 700 24,572 0 26 Jul 2019
Generating Long Sequences with Sparse Transformers R. Child Scott Gray Alec Radford Ilya Sutskever 136 1,919 0 23 Apr 2019
Data Augmentation for BERT Fine-Tuning in Open-Domain Question Answering Wei Yang Yuqing Xie Luchen Tan Kun Xiong Ming Li Jimmy J. Lin RALM OOD 65 64 0 14 Apr 2019
BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model Alex Jinpeng Wang Kyunghyun Cho VLM 112 358 0 11 Feb 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 263 3,747 0 09 Jan 2019
Neural Machine Translation by Jointly Learning to Align and Translate Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio AIMat 584 27,345 0 01 Sep 2014