Don't Stop Pretraining: Adapt Language Models to Domains and Tasks

23 April 2020

Kyle Lo

Papers citing "Don't Stop Pretraining: Adapt Language Models to Domains and Tasks"

50 / 522 papers shown

Title
Knowledge-Augmented Language Models for Cause-Effect Relation Classification Pedram Hosseini David A. Broniatowski Mona T. Diab CML 23 18 0 16 Dec 2021
Learning Rich Representation of Keyphrases from Text Mayank Kulkarni Debanjan Mahata Ravneet Arora Rajarshi Bhowmik VLM 27 65 0 16 Dec 2021
GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval Kexin Wang Nandan Thakur Nils Reimers Iryna Gurevych VLM 24 149 0 14 Dec 2021
MoCA: Incorporating Multi-stage Domain Pretraining and Cross-guided Multimodal Attention for Textbook Question Answering Fangzhi Xu Qika Lin Jing Liu Lingling Zhang Tianzhe Zhao Qianyi Chai Yudai Pan 14 2 0 06 Dec 2021
MultiVerS: Improving scientific claim verification with weak supervision and full-document context David Wadden Bertie Vidgen Lucy Lu Wang Dirk Hovy J. Pierrehumbert Hannaneh Hajishirzi 31 153 0 02 Dec 2021
NER-BERT: A Pre-trained Model for Low-Resource Entity Tagging Zihan Liu Feijun Jiang Yuxiang Hu Chen Shi Pascale Fung 22 37 0 01 Dec 2021
Temporal Effects on Pre-trained Models for Language Processing Tasks Oshin Agarwal A. Nenkova VLM 28 53 0 24 Nov 2021
Merging Models with Fisher-Weighted Averaging Michael Matena Colin Raffel FedML MoMe 50 354 0 18 Nov 2021
Linking-Enhanced Pre-Training for Table Semantic Parsing Bowen Qin Lihan Wang Binyuan Hui Ruiying Geng Zhen Cao Min Yang Jian Sun Yongbin Li 35 1 0 18 Nov 2021
Joint Unsupervised and Supervised Training for Multilingual ASR Junwen Bai Bo-wen Li Yu Zhang Ankur Bapna Nikhil Siddhartha K. Sim Tara N. Sainath 32 58 0 15 Nov 2021
Scaling Law for Recommendation Models: Towards General-purpose User Representations Kyuyong Shin Hanock Kwak KyungHyun Kim Max Nihlén Ramström Jisu Jeong Jung-Woo Ha S. Kim ELM 36 38 0 15 Nov 2021
DEEP: DEnoising Entity Pre-training for Neural Machine Translation Junjie Hu Hiroaki Hayashi Kyunghyun Cho Graham Neubig AI4CE 27 21 0 14 Nov 2021
SocialBERT -- Transformers for Online SocialNetwork Language Modelling I. Karpov Nick Kartashev 30 3 0 13 Nov 2021
On Transferability of Prompt Tuning for Natural Language Processing Yusheng Su Xiaozhi Wang Yujia Qin Chi-Min Chan Yankai Lin ... Peng Li Juanzi Li Lei Hou Maosong Sun Jie Zhou AAML VLM 31 98 0 12 Nov 2021
Character-level HyperNetworks for Hate Speech Detection Tomer Wullach A. Adler Einat Minkov 24 12 0 11 Nov 2021
Recent Advances in Automated Question Answering In Biomedical Domain K. D. Baksi 28 0 0 10 Nov 2021
Learning to Generalize Compositionally by Transferring Across Semantic Parsing Tasks Wang Zhu Peter Shaw Tal Linzen Fei Sha 35 7 0 09 Nov 2021
Adapting to the Long Tail: A Meta-Analysis of Transfer Learning Research for Language Understanding Tasks Aakanksha Naik J. Lehman Carolyn Rose 46 7 0 02 Nov 2021
MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare Shaoxiong Ji Tianlin Zhang Luna Ansari Jie Fu Prayag Tiwari Min Zhang VLM AI4MH 33 223 0 29 Oct 2021
Can Character-based Language Models Improve Downstream Task Performance in Low-Resource and Noisy Language Scenarios? Arij Riabi Benoît Sagot Djamé Seddah 31 15 0 26 Oct 2021
ClimateBert: A Pretrained Language Model for Climate-Related Text Nicolas Webersinke Mathias Kraus Jiabo Huang Markus Leippold AI4CE 37 131 0 22 Oct 2021
Improved Multilingual Language Model Pretraining for Social Media Text via Translation Pair Prediction Shubhanshu Mishra A. Haghighi VLM 29 4 0 20 Oct 2021
Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora Xisen Jin Dejiao Zhang Henghui Zhu Wei Xiao Shang-Wen Li Xiaokai Wei Andrew O. Arnold Xiang Ren KELM CLL 31 112 0 16 Oct 2021
DS-TOD: Efficient Domain Specialization for Task Oriented Dialog Chia-Chien Hung Anne Lauscher Simone Paolo Ponzetto Goran Glavaš 39 31 0 15 Oct 2021
Building Chinese Biomedical Language Models via Multi-Level Text Discrimination Quan Wang Songtai Dai Benfeng Xu Yajuan Lyu Yong Zhu Hua Wu Haifeng Wang 71 14 0 14 Oct 2021
Exploring Wav2vec 2.0 fine-tuning for improved speech emotion recognition Li-Wei Chen Alexander I. Rudnicky VLM 27 122 0 12 Oct 2021
K-Wav2vec 2.0: Automatic Speech Recognition based on Joint Decoding of Graphemes and Syllables Jounghee Kim Pilsung Kang VLM 29 6 0 11 Oct 2021
Advances in Multi-turn Dialogue Comprehension: A Survey ZhuoSheng Zhang Hai Zhao 29 21 0 11 Oct 2021
The Inductive Bias of In-Context Learning: Rethinking Pretraining Example Design Yoav Levine Noam Wies Daniel Jannai D. Navon Yedid Hoshen Amnon Shashua AI4CE 35 36 0 09 Oct 2021
Improving Multi-Party Dialogue Discourse Parsing via Domain Integration Zhengyuan Liu Nancy F. Chen 32 33 0 09 Oct 2021
Machine Learning Featurizations for AI Hacking of Political Systems Nathan Sanders B. Schneier 20 2 0 08 Oct 2021
Towards Continual Knowledge Learning of Language Models Joel Jang Seonghyeon Ye Sohee Yang Joongbo Shin Janghoon Han Gyeonghun Kim Stanley Jungkyu Choi Minjoon Seo CLL KELM 233 151 0 07 Oct 2021
MatSciBERT: A Materials Domain Language Model for Text Mining and Information Extraction Tanishq Gupta Mohd Zaki N. M. A. Krishnan Mausam 49 178 0 30 Sep 2021
ReINTEL Challenge 2020: A Comparative Study of Hybrid Deep Neural Network for Reliable Intelligence Identification on Vietnamese SNSs Hoang Viet Trinh Tung Tien Bui Tam Minh Nguyen Huy Quang Dao Quang Huu Pham Ngoc N. Tran Ta Minh Thanh 24 1 0 27 Sep 2021
Rumour Detection via Zero-shot Cross-lingual Transfer Learning Lin Tian Xiuzhen Zhang Jey Han Lau 46 13 0 27 Sep 2021
DialogueCSE: Dialogue-based Contrastive Learning of Sentence Embeddings Che Liu Rui Wang Jinghua Liu Jian Sun Fei Huang Luo Si 43 40 0 26 Sep 2021
Caption Enriched Samples for Improving Hateful Memes Detection Efrat Blaier Itzik Malkiel Lior Wolf VLM 56 21 0 22 Sep 2021
ConvFiT: Conversational Fine-Tuning of Pretrained Language Models Ivan Vulić Pei-hao Su Sam Coope D. Gerz Paweł Budzianowski I. Casanueva Nikola Mrkvsić Tsung-Hsien Wen 27 36 0 21 Sep 2021
Improving Span Representation for Domain-adapted Coreference Resolution Nupoor Gandhi Anjalie Field Yulia Tsvetkov CLL 30 3 0 20 Sep 2021
Cross-lingual Transfer of Monolingual Models Evangelia Gogoulou Ariel Ekgren T. Isbister Magnus Sahlgren 31 18 0 15 Sep 2021
Automatically Exposing Problems with Neural Dialog Models Dian Yu Kenji Sagae 31 9 0 14 Sep 2021
Learning Bill Similarity with Annotated and Augmented Corpora of Bills Jiseon Kim Elden Griggs In Song Kim Alice Oh AILaw 20 5 0 14 Sep 2021
Different Strokes for Different Folks: Investigating Appropriate Further Pre-training Approaches for Diverse Dialogue Tasks Yao Qiu Jinchao Zhang Jie Zhou 16 5 0 14 Sep 2021
Task-adaptive Pre-training and Self-training are Complementary for Natural Language Understanding Shiyang Li Semih Yavuz Wenhu Chen Xifeng Yan 22 12 0 14 Sep 2021
STraTA: Self-Training with Task Augmentation for Better Few-shot Learning Tu Vu Minh-Thang Luong Quoc V. Le Grady Simon Mohit Iyyer 131 61 0 13 Sep 2021
IndoBERTweet: A Pretrained Language Model for Indonesian Twitter with Effective Domain-Specific Vocabulary Initialization Fajri Koto Jey Han Lau Timothy Baldwin VLM 55 82 0 10 Sep 2021
Identifying Morality Frames in Political Tweets using Relational Learning Shamik Roy Maria Leonor Pacheco Dan Goldwasser 37 41 0 09 Sep 2021
Avoiding Inference Heuristics in Few-shot Prompt-based Finetuning Prasetya Ajie Utama N. Moosavi Victor Sanh Iryna Gurevych AAML 63 35 0 09 Sep 2021
Enhancing Natural Language Representation with Large-Scale Out-of-Domain Commonsense Wanyun Cui Xingran Chen 22 6 0 06 Sep 2021
Task-Oriented Dialogue System as Natural Language Generation Weizhi Wang Zhirui Zhang Junliang Guo Yinpei Dai Boxing Chen Weihua Luo 36 32 0 31 Aug 2021