Don't Stop Pretraining: Adapt Language Models to Domains and Tasks

23 April 2020

Kyle Lo

Papers citing "Don't Stop Pretraining: Adapt Language Models to Domains and Tasks"

22 / 522 papers shown

Title
Improving Low Compute Language Modeling with In-Domain Embedding Initialisation Charles F Welch Rada Mihalcea Jonathan K. Kummerfeld AI4CE 24 4 0 29 Sep 2020
DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue Shikib Mehri Mihail Eric Dilek Z. Hakkani-Tür ELM 26 136 0 28 Sep 2020
What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams Di Jin Eileen Pan Nassim Oufattole W. Weng Hanyi Fang Peter Szolovits FaML ELM LM&MA 31 709 0 28 Sep 2020
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models Samuel Gehman Suchin Gururangan Maarten Sap Yejin Choi Noah A. Smith 52 1,134 0 24 Sep 2020
A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support Ashish Sharma Adam S. Miner David C. Atkins Tim Althoff AI4MH 25 272 0 17 Sep 2020
Transformer Based Multi-Source Domain Adaptation Dustin Wright Isabelle Augenstein 32 53 0 16 Sep 2020
Learning an Effective Context-Response Matching Model with Self-Supervised Tasks for Retrieval-based Dialogues Ruijian Xu Chongyang Tao Daxin Jiang Xueliang Zhao Dongyan Zhao Rui Yan 29 70 0 14 Sep 2020
Improving Machine Reading Comprehension with Contextualized Commonsense Knowledge Kai Sun Dian Yu Jianshu Chen Dong Yu Claire Cardie 30 12 0 12 Sep 2020
Investigating Pretrained Language Models for Graph-to-Text Generation Leonardo F. R. Ribeiro Martin Schmitt Hinrich Schütze Iryna Gurevych 19 216 0 16 Jul 2020
Composed Fine-Tuning: Freezing Pre-Trained Denoising Autoencoders for Improved Generalization Sang Michael Xie Tengyu Ma Percy Liang 35 13 0 29 Jun 2020
CausaLM: Causal Model Explanation Through Counterfactual Language Models Amir Feder Nadav Oved Uri Shalit Roi Reichart CML LRM 44 157 0 27 May 2020
BERTweet: A pre-trained language model for English Tweets Dat Quoc Nguyen Thanh Tien Vu A. Nguyen VLM 34 901 0 20 May 2020
Evidence Inference 2.0: More Data, Better Models Jay DeYoung Eric P. Lehman Benjamin E. Nye Iain J. Marshall Byron C. Wallace 25 68 0 08 May 2020
Fact or Fiction: Verifying Scientific Claims David Wadden Shanchuan Lin Kyle Lo Lucy Lu Wang Madeleine van Zuylen Arman Cohan Hannaneh Hajishirzi HAI 36 430 0 30 Apr 2020
Generative Data Augmentation for Commonsense Reasoning Yiben Yang Chaitanya Malaviya Jared Fernandez Swabha Swayamdipta Ronan Le Bras Ji-ping Wang Chandra Bhagavatula Yejin Choi Doug Downey LRM 30 91 0 24 Apr 2020
Train No Evil: Selective Masking for Task-Guided Pre-Training Yuxian Gu Zhengyan Zhang Xiaozhi Wang Zhiyuan Liu Maosong Sun 32 59 0 21 Apr 2020
Unsupervised Domain Clusters in Pretrained Language Models Roee Aharoni Yoav Goldberg 37 244 0 05 Apr 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 246 1,454 0 18 Mar 2020
LAMBERT: Layout-Aware (Language) Modeling for information extraction Lukasz Garncarek Rafal Powalski Tomasz Stanislawek Bartosz Topolski Piotr Halama M. Turski Filip Graliñski 8 87 0 19 Feb 2020
Explaining Relationships Between Scientific Documents Kelvin Luu Xinyi Wu Rik Koncel-Kedziorski Kyle Lo Isabel Cachola Noah A. Smith 41 48 0 02 Feb 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 304 6,996 0 20 Apr 2018
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Zhehuai Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 718 6,750 0 26 Sep 2016