Don't Stop Pretraining: Adapt Language Models to Domains and Tasks

23 April 2020

Kyle Lo

Papers citing "Don't Stop Pretraining: Adapt Language Models to Domains and Tasks"

50 / 522 papers shown

Title
Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contexts Ashutosh Baheti Maarten Sap Alan Ritter Mark O. Riedl 21 84 0 26 Aug 2021
Data Augmentation for Low-Resource Named Entity Recognition Using Backtranslation Usama Yaseen Stefan Langer MedIm 21 15 0 26 Aug 2021
Auxiliary Task Update Decomposition: The Good, The Bad and The Neutral Lucio Dery Yann N. Dauphin David Grangier MoMe 21 29 0 25 Aug 2021
Bursting Scientific Filter Bubbles: Boosting Innovation via Novel Author Discovery Jason Portenoy Marissa Radensky Jevin D. West Eric Horvitz Daniel S. Weld Tom Hope 94 31 0 12 Aug 2021
Robust Transfer Learning with Pretrained Language Models through Adapters Wenjuan Han Bo Pang Ying Nian Wu 16 54 0 05 Aug 2021
More but Correct: Generating Diversified and Entity-revised Medical Response Bin Li Encheng Chen Hongrui Liu Yixuan Weng Bin Sun Shutao Li Yongping Bai Meiling Hu MedIm 19 11 0 03 Aug 2021
Target-Oriented Fine-tuning for Zero-Resource Named Entity Recognition Ying Zhang Fandong Meng Jinan Xu Jinan Xu Jie Zhou 30 10 0 22 Jul 2021
Back-Translated Task Adaptive Pretraining: Improving Accuracy and Robustness on Text Classification Junghoon Lee Jounghee Kim Pilsung Kang VLM 19 5 0 22 Jul 2021
Small-Text: Active Learning for Text Classification in Python Christopher Schröder Lydia Muller A. Niekler Martin Potthast CLIP VLM AI4CE 39 23 0 21 Jul 2021
Improved Text Classification via Contrastive Adversarial Training Lin Pan Chung-Wei Hang Avirup Sil Saloni Potdar AAML 28 86 0 21 Jul 2021
The Effectiveness of Intermediate-Task Training for Code-Switched Natural Language Understanding Archiki Prasad Mohammad Ali Rehan Shreyasi Pathak Preethi Jyothi 27 9 0 21 Jul 2021
Adaptive Transfer Learning on Graph Neural Networks Xueting Han Zhenhuan Huang Bang An Jing Bai 30 59 0 19 Jul 2021
A Theoretical Analysis of Fine-tuning with Linear Teachers Gal Shachaf Alon Brutzkus Amir Globerson 34 17 0 04 Jul 2021
Scientia Potentia Est -- On the Role of Knowledge in Computational Argumentation Anne Lauscher Henning Wachsmuth Iryna Gurevych Goran Glavaš 38 31 0 01 Jul 2021
Cross-Lingual Transfer Learning for Statistical Type Inference Zhiming Li Xiaofei Xie Haoliang Li Zhengzi Xu Yi Li Yang Liu 17 2 0 01 Jul 2021
Domain-Specific Pretraining for Vertical Search: Case Study on Biomedical Literature Yu-Chiang Frank Wang Jinchao Li Tristan Naumann Chenyan Xiong Hao Cheng ... Yang Qin Eric Horvitz Paul N. Bennett Jianfeng Gao Hoifung Poon OOD 33 13 0 25 Jun 2021
GAIA: A Transfer Learning System of Object Detection that Fits Your Needs Xingyuan Bu Junran Peng Junjie Yan Tieniu Tan Zhaoxiang Zhang ObjD VLM 31 53 0 21 Jun 2021
Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets Irene Solaiman Christy Dennison 30 222 0 18 Jun 2021
Specializing Multilingual Language Models: An Empirical Study Ethan C. Chau Noah A. Smith 27 27 0 16 Jun 2021
Named Entity Recognition with Small Strongly Labeled and Large Weakly Labeled Data Haoming Jiang Danqing Zhang Tianyu Cao Bing Yin T. Zhao NoLa 30 44 0 16 Jun 2021
CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark Ningyu Zhang Mosha Chen Zhen Bi Xiaozhuan Liang Lei Li ... Jun Yan Hongying Zan Kunli Zhang Buzhou Tang Qingcai Chen LM&MA ELM 34 179 0 15 Jun 2021
Pre-Trained Models: Past, Present and Future Xu Han Zhengyan Zhang Ning Ding Yuxian Gu Xiao Liu ... Jie Tang Ji-Rong Wen Jinhui Yuan Wayne Xin Zhao Jun Zhu AIFin MQ AI4MH 58 816 0 14 Jun 2021
CodemixedNLP: An Extensible and Open NLP Toolkit for Code-Mixing Sai Muralidhar Jayanthi Kavya Nerella Khyathi Raghavi Chandu A. Black MoE 36 8 0 10 Jun 2021
Linguistically Informed Masking for Representation Learning in the Patent Domain Sophia Althammer Mark Buckley Sebastian Hofstatter Allan Hanbury 45 11 0 10 Jun 2021
Signal Transformer: Complex-valued Attention and Meta-Learning for Signal Recognition Yihong Dong Ying Peng Muqiao Yang Songtao Lu Qingjiang Shi 46 9 0 05 Jun 2021
MPC-BERT: A Pre-Trained Language Model for Multi-Party Conversation Understanding Jia-Chen Gu Chongyang Tao Zhenhua Ling Can Xu Xiubo Geng Daxin Jiang 21 53 0 03 Jun 2021
DynaEval: Unifying Turn and Dialogue Level Evaluation Chen Zhang Yiming Chen L. F. D’Haro Yan Zhang Thomas Friedrichs Grandee Lee Haizhou Li 24 73 0 02 Jun 2021
Improving Formality Style Transfer with Context-Aware Rule Injection Zonghai Yao Hong-ye Yu 26 16 0 01 Jun 2021
CLEVE: Contrastive Pre-training for Event Extraction Ziqi Wang Xiaozhi Wang Xu Han Yankai Lin Lei Hou Zhiyuan Liu Peng Li Juan-Zi Li Jie Zhou 37 116 0 30 May 2021
Sentiment analysis in tweets: an assessment study from classical to modern text representation models Sérgio Barreto Ricardo Moura Jonnathan Carvalho A. Paes A. Plastino 23 14 0 29 May 2021
UCPhrase: Unsupervised Context-aware Quality Phrase Tagging Xiaotao Gu Zihan Wang Zhenyu Bi Yu Meng Liyuan Liu Jiawei Han Jingbo Shang 103 36 0 28 May 2021
CiteWorth: Cite-Worthiness Detection for Improved Scientific Document Understanding Dustin Wright Isabelle Augenstein 16 24 0 23 May 2021
Lawformer: A Pre-trained Language Model for Chinese Legal Long Documents Chaojun Xiao Xueyu Hu Zhiyuan Liu Cunchao Tu Maosong Sun AILaw ELM 48 229 0 09 May 2021
Unsupervised Sentiment Analysis by Transferring Multi-source Knowledge Yong Dai Jian-Dong Liu Jian Zhang H. Fu Zenglin Xu 30 12 0 09 May 2021
Empirical Evaluation of Pre-trained Transformers for Human-Level NLP: The Role of Sample Size and Dimensionality Adithya V Ganesan Matthew Matero Aravind Reddy Ravula Huy-Hien Vu H. Andrew Schwartz 30 35 0 07 May 2021
DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts Alisa Liu Maarten Sap Ximing Lu Swabha Swayamdipta Chandra Bhagavatula Noah A. Smith Yejin Choi MU 31 359 0 07 May 2021
SUPERB: Speech processing Universal PERformance Benchmark Shu-Wen Yang Po-Han Chi Yung-Sung Chuang Cheng-I Jeff Lai Kushal Lakhotia ... Shuyan Dong Shang-Wen Li Shinji Watanabe Abdel-rahman Mohamed Hung-yi Lee SSL 59 891 0 03 May 2021
Framing Unpacked: A Semi-Supervised Interpretable Multi-View Model of Media Frames Shima Khanehzar Trevor Cohn Gosia Mikołajczak A. Turpin Lea Frermann 22 11 0 22 Apr 2021
Reference-based Weak Supervision for Answer Sentence Selection using Web Data Vivek Krishnamurthy Thuy Vu Alessandro Moschitti 21 1 0 18 Apr 2021
On the Influence of Masking Policies in Intermediate Pre-training Qinyuan Ye Belinda Z. Li Sinong Wang Benjamin Bolte Hao Ma Wen-tau Yih Xiang Ren Madian Khabsa 13 12 0 18 Apr 2021
SciCo: Hierarchical Cross-Document Coreference for Scientific Concepts Arie Cattan Sophie Johnson Daniel S. Weld Ido Dagan Iz Beltagy Doug Downey Tom Hope 28 23 0 18 Apr 2021
Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems E. Razumovskaia Goran Glavaš Olga Majewska Edoardo Ponti Anna Korhonen Ivan Vulić 33 32 0 17 Apr 2021
Sequential Cross-Document Coreference Resolution Emily Allaway Shuai Wang Miguel Ballesteros 30 16 0 17 Apr 2021
On the Importance of Effectively Adapting Pretrained Language Models for Active Learning Katerina Margatina Loïc Barrault Nikolaos Aletras 27 36 0 16 Apr 2021
Capturing Row and Column Semantics in Transformer Based Question Answering over Tables Michael R. Glass Mustafa Canim A. Gliozzo Saneem A. Chemmengath Vishwajeet Kumar Rishav Chakravarti Avirup Sil FeiFei Pan Samarth Bharadwaj Nicolas Rodolfo Fauceglia LMTD 21 54 0 16 Apr 2021
AMMU : A Survey of Transformer-based Biomedical Pretrained Language Models Katikapalli Subramanyam Kalyan A. Rajasekharan S. Sangeetha LM&MA MedIm 28 164 0 16 Apr 2021
What to Pre-Train on? Efficient Intermediate Task Selection Clifton A. Poth Jonas Pfeiffer Andreas Rucklé Iryna Gurevych 24 95 0 16 Apr 2021
Probing Across Time: What Does RoBERTa Know and When? Leo Z. Liu Yizhong Wang Jungo Kasai Hannaneh Hajishirzi Noah A. Smith KELM 16 80 0 16 Apr 2021
Towards Robust Neural Retrieval Models with Synthetic Pre-Training Revanth Reddy Gangi Reddy Vikas Yadav Md Arafat Sultan M. Franz Vittorio Castelli Heng Ji Avirup Sil 26 14 0 15 Apr 2021
Cross-Domain Label-Adaptive Stance Detection Momchil Hardalov Arnav Arora Preslav Nakov Isabelle Augenstein 35 72 0 15 Apr 2021