BIOptimus: Pre-training an Optimal Biomedical Language Model with Curriculum Learning for Named Entity Recognition

16 August 2023

Papers citing "BIOptimus: Pre-training an Optimal Biomedical Language Model with Curriculum Learning for Named Entity Recognition"

23 / 23 papers shown

Title
LinkBERT: Pretraining Language Models with Document Links Michihiro Yasunaga J. Leskovec Percy Liang KELM 106 361 0 29 Mar 2022
"Is Whole Word Masking Always Better for Chinese BERT?": Probing on Chinese Grammatical Error Correction Yong Dai Linyang Li Cong Zhou Zhangyin Feng Enbo Zhao Xipeng Qiu Pijian Li Duyu Tang 56 13 0 01 Mar 2022
Should You Mask 15% in Masked Language Modeling? Alexander Wettig Tianyu Gao Zexuan Zhong Danqi Chen CVBM 74 166 0 16 Feb 2022
Efficient Domain Adaptation of Language Models via Adaptive Tokenization Vin Sachidananda Jason S Kessler Yi-An Lai 54 36 0 15 Sep 2021
Frustratingly Simple Pretraining Alternatives to Masked Language Modeling Atsuki Yamaguchi G. Chrysostomou Katerina Margatina Nikolaos Aletras 56 24 0 04 Sep 2021
Meta-Curriculum Learning for Domain Adaptation in Neural Machine Translation Runzhe Zhan Xuebo Liu Derek F. Wong Lidia S. Chao 77 46 0 03 Mar 2021
Curriculum Learning: A Survey Petru Soviany Radu Tudor Ionescu Paolo Rota N. Sebe ODL 157 359 0 25 Jan 2021
Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing Yu Gu Robert Tinn Hao Cheng Michael R. Lucas Naoto Usuyama Xiaodong Liu Tristan Naumann Jianfeng Gao Hoifung Poon LM&MA AI4CE 116 1,783 0 31 Jul 2020
On the Importance of Word and Sentence Representation Learning in Implicit Discourse Relation Classification Xin Liu Jiefu Ou Yangqiu Song Xin Jiang 40 90 0 27 Apr 2020
Don't Stop Pretraining: Adapt Language Models to Domains and Tasks Suchin Gururangan Ana Marasović Swabha Swayamdipta Kyle Lo Iz Beltagy Doug Downey Noah A. Smith VLM AI4CE CLL 164 2,435 0 23 Apr 2020
Train No Evil: Selective Masking for Task-Guided Pre-Training Yuxian Gu Zhengyan Zhang Xiaozhi Wang Zhiyuan Liu Maosong Sun 119 59 0 21 Apr 2020
Inexpensive Domain Adaptation of Pretrained Language Models: Case Studies on Biomedical NER and Covid-19 QA Nina Poerner Ulli Waltinger Hinrich Schütze OOD 57 50 0 07 Apr 2020
Unsupervised Domain Clusters in Pretrained Language Models Roee Aharoni Yoav Goldberg 94 251 0 05 Apr 2020
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 485 20,342 0 23 Oct 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 373 6,469 0 26 Sep 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 686 24,557 0 26 Jul 2019
Simple and Effective Curriculum Pointer-Generator Networks for Reading Comprehension over Long Narratives Yi Tay Shuohang Wang Anh Tuan Luu Jie Fu Minh C. Phan Xingdi Yuan J. Rao S. Hui Aston Zhang 88 110 0 26 May 2019
ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission Kexin Huang Jaan Altosaar Rajesh Ranganath OOD 124 909 0 10 Apr 2019
SciBERT: A Pretrained Language Model for Scientific Text Iz Beltagy Kyle Lo Arman Cohan 168 2,986 0 26 Mar 2019
BioBERT: a pre-trained biomedical language representation model for biomedical text mining Jinhyuk Lee Wonjin Yoon Sungdong Kim Donghyeon Kim Sunkyu Kim Chan Ho So Jaewoo Kang OOD 180 5,674 0 25 Jan 2019
Dissecting Contextual Word Embeddings: Architecture and Representation Matthew E. Peters Mark Neumann Luke Zettlemoyer Wen-tau Yih 101 431 0 27 Aug 2018
SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing Taku Kudo John Richardson 206 3,531 0 19 Aug 2018
Neural Machine Translation of Rare Words with Subword Units Rico Sennrich Barry Haddow Alexandra Birch 235 7,760 0 31 Aug 2015