Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing

31 July 2020

Xiaodong Liu

Papers citing "Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing"

50 / 241 papers shown

Title
AF Adapter: Continual Pretraining for Building Chinese Biomedical Language Model Yongyu Yan Kui Xue Xiaoming Shi Qi Ye Jingping Liu Tong Ruan CLL 47 1 0 21 Nov 2022
Combining Contrastive Learning and Knowledge Graph Embeddings to develop medical word embeddings for the Italian language Denys Amore Bondarenko Roger Ferrod Luigi Di Caro 21 1 0 09 Nov 2022
Unsupervised Domain Adaptation for Sparse Retrieval by Filling Vocabulary and Word Frequency Gaps Hiroki Iida Naoaki Okazaki 42 4 0 08 Nov 2022
CCS Explorer: Relevance Prediction, Extractive Summarization, and Named Entity Recognition from Clinical Cohort Studies Irfan Al-Hussaini D. An Albert Lee Sarah Bi Cassie S. Mitchell 11 7 0 01 Nov 2022
Influence Functions for Sequence Tagging Models Sarthak Jain Varun Manjunatha Byron C. Wallace A. Nenkova TDI 35 8 0 25 Oct 2022
How Long Is Enough? Exploring the Optimal Intervals of Long-Range Clinical Note Language Modeling Samuel Cahyawijaya Bryan Wilie Holy Lovenia Huang Zhong Mingqian Zhong Yuk-Yu Nancy Ip Pascale Fung LM&MA 25 2 0 25 Oct 2022
Exploring the Value of Pre-trained Language Models for Clinical Named Entity Recognition Yuping Wu Lifeng Han Valerio Antonini Goran Nenadic 33 4 0 23 Oct 2022
Understanding Domain Learning in Language Models Through Subpopulation Analysis Zheng Zhao Yftah Ziser Shay B. Cohen 36 6 0 22 Oct 2022
BioLORD: Learning Ontological Representations from Definitions (for Biomedical Concepts and their Textual Descriptions) François Remy Kris Demuynck Thomas Demeester 27 14 0 21 Oct 2022
BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining Renqian Luo Liai Sun Yingce Xia Tao Qin Sheng Zhang Hoifung Poon Tie-Yan Liu MedIm AI4CE LM&MA 62 790 0 19 Oct 2022
Deep Bidirectional Language-Knowledge Graph Pretraining Michihiro Yasunaga Antoine Bosselut Hongyu Ren Xikun Zhang Christopher D. Manning Percy Liang J. Leskovec 36 194 0 17 Oct 2022
Developing a general-purpose clinical language inference model from a large corpus of clinical notes Madhumita Sushil Dana Ludwig A. Butte V. Rudrapatna LM&MA 14 12 0 12 Oct 2022
Enriching Biomedical Knowledge for Low-resource Language Through Large-Scale Translation Long Phan Tai Dang H. Tran Trieu H. Trinh Vy Phan Lam D. Chau Minh-Thang Luong 20 8 0 11 Oct 2022
How to tackle an emerging topic? Combining strong and weak labels for Covid news NER Aleksander Ficek Fangyu Liu Nigel Collier 33 1 0 29 Sep 2022
DR.BENCH: Diagnostic Reasoning Benchmark for Clinical Natural Language Processing Yanjun Gao Dmitriy Dligach Timothy A. Miller John R. Caskey Brihat Sharma M. Churpek Majid Afshar ELM LRM 34 17 0 29 Sep 2022
Variational Open-Domain Question Answering Valentin Liévin Andreas Geert Motzfeldt Ida Riis Jensen Ole Winther OOD BDL 38 8 0 23 Sep 2022
Generalizing through Forgetting -- Domain Generalization for Symptom Event Extraction in Clinical Notes Sitong Zhou K. Lybarger Meliha Yetisgen-Yildiz Mari Ostendorf 40 2 0 20 Sep 2022
Prompt Combines Paraphrase: Teaching Pre-trained Models to Understand Rare Biomedical Words Hao Wang Chi-Liang Liu Nuwa Xi Sendong Zhao Meizhi Ju Shiwei Zhang Ziheng Zhang Yefeng Zheng Bing Qin Ting Liu VLM AAML LM&MA 41 6 0 14 Sep 2022
AILAB-Udine@SMM4H 22: Limits of Transformers and BERT Ensembles Beatrice Portelli Simone Scaboro Emmanuele Chersoni Enrico Santus Giuseppe Serra MoE 16 5 0 07 Sep 2022
Increasing Adverse Drug Events extraction robustness on social media: case study on negation and speculation Simone Scaboro Beatrice Portelli Emmanuele Chersoni Enrico Santus G. Serra 32 5 0 06 Sep 2022
Optimizing Bi-Encoder for Named Entity Recognition via Contrastive Learning Sheng Zhang Hao Cheng Jianfeng Gao Hoifung Poon 38 47 0 30 Aug 2022
A Multimodal Transformer: Fusing Clinical Notes with Structured EHR Data for Interpretable In-Hospital Mortality Prediction Weimin Lyu Xinyu Dong Rachel Wong Songzhu Zheng Kayley Abell-Hart Fusheng Wang Chao Chen 26 47 0 09 Aug 2022
Cross-Lingual Knowledge Transfer for Clinical Phenotyping Jens-Michalis Papaioannou Paul Grundmann Betty van Aken A. Samaras Ilias Kyparissidis G. Giannakoulas Felix Alexander Gers Alexander Loser 42 6 0 03 Aug 2022
Can large language models reason about medical questions? Valentin Liévin C. Hother Andreas Geert Motzfeldt Ole Winther ELM LM&MA AI4MH LRM 29 300 0 17 Jul 2022
PLM-ICD: Automatic ICD Coding with Pretrained Language Models Chao-Wei Huang Shang-Chi Tsai Yun-Nung Chen 40 49 0 12 Jul 2022
Developing an NLP-based Recommender System for the Ethical, Legal, and Social Implications of Synthetic Biology Damien Dablain Lilian Huang Brandon Sepulvado 21 0 0 10 Jul 2022
Training Subset Selection for Weak Supervision Hunter Lang Aravindan Vijayaraghavan David Sontag NoLa 16 21 0 06 Jun 2022
Learning to Automate Follow-up Question Generation using Process Knowledge for Depression Triage on Reddit Posts Shrey Gupta Anmol Agarwal Manas Gaur Kaushik Roy Vignesh Narayanan Ponnurangam Kumaraguru Amit P. Sheth AI4MH 22 34 0 27 May 2022
K-12BERT: BERT for K-12 education Vasu Goel Dhruv Sahnan Venktesh V Gaurav Sharma Deep Dwivedi Mukesh Mohania AI4Ed 22 5 0 24 May 2022
A reproducible experimental survey on biomedical sentence similarity: a string-based method sets the state of the art Alicia Lara-Clares Juan J. Lastra-Díaz A. García-Serrano 27 0 0 18 May 2022
CascadER: Cross-Modal Cascading for Knowledge Graph Link Prediction Tara Safavi Doug Downey Tom Hope 32 6 0 16 May 2022
Clinical Prompt Learning with Frozen Language Models Niall Taylor Yi Zhang Dan W Joyce A. Nevado-Holgado Andrey Kormilitzin VLM LM&MA 16 31 0 11 May 2022
SuMe: A Dataset Towards Summarizing Biomedical Mechanisms Mohaddeseh Bastan N. Shankar Mihai Surdeanu Niranjan Balasubramanian 26 3 0 10 May 2022
Assigning Species Information to Corresponding Genes by a Sequence Labeling Framework Ling Luo Chih-Hsuan Wei Po-Ting Lai Qingyu Chen R. Dogan Zhiyong Lu 28 3 0 08 May 2022
Explaining the Effectiveness of Multi-Task Learning for Efficient Knowledge Extraction from Spine MRI Reports Arijit Sehanobish M. Sandora Nabila Abraham Jayashri Pawar Danielle Torres Anasuya Das M. Becker Richard Herzog Benjamin Odry Ron Vianu 26 3 0 06 May 2022
Decorate the Examples: A Simple Method of Prompt Design for Biomedical Relation Extraction Hui-Syuan Yeh Thomas Lavergne Pierre Zweigenbaum 26 10 0 21 Apr 2022
Multi-label classification for biomedical literature: an overview of the BioCreative VII LitCovid Track for COVID-19 literature topic annotations Qingyu Chen Alexis Allot Robert Leaman Rezarta Islamaj Dougan Jingcheng Du ... S. Pujari Mariia Chizhikova Naveen Sivadasan Naveen Sivadasan Zhiyong Lu 27 30 0 20 Apr 2022
LitMC-BERT: transformer-based multi-label classification of biomedical literature with an application on COVID-19 literature curation Qingyu Chen Jingcheng Du Alexis Allot Zhiyong Lu 24 25 0 19 Apr 2022
Multi-label topic classification for COVID-19 literature with Bioformer Li Fang Kai Wang 17 4 0 14 Apr 2022
BioRED: A Rich Biomedical Relation Extraction Dataset Ling Luo Po-Ting Lai Chih-Hsuan Wei C. Arighi Zhiyong Lu 44 122 0 08 Apr 2022
BioBART: Pretraining and Evaluation of A Biomedical Generative Language Model Hongyi Yuan Zheng Yuan Ruyi Gan Jiaxing Zhang Yutao Xie Sheng Yu LM&MA 33 123 0 08 Apr 2022
Hierarchical Annotation for Building A Suite of Clinical Natural Language Processing Tasks: Progress Note Understanding Yanjun Gao Dmitriy Dligach Timothy A. Miller S. Tesch Ryan Laffin M. Churpek Majid Afshar 18 17 0 06 Apr 2022
Global Readiness of Language Technology for Healthcare: What would it Take to Combat the Next Pandemic? Ishani Mondal Kabir Ahuja Mohit Jain Jacki O Neil Kalika Bali Monojit Choudhury ELM LM&MA 29 4 0 06 Apr 2022
A sequence-to-sequence approach for document-level relation extraction John Giorgi Gary D. Bader Bo Wang 41 52 0 03 Apr 2022
LinkBERT: Pretraining Language Models with Document Links Michihiro Yasunaga J. Leskovec Percy Liang KELM 29 353 0 29 Mar 2022
Towards Structuring Real-World Data at Scale: Deep Learning for Extracting Key Oncology Information from Clinical Text with Patient-Level Supervision Sam Preston Mu-Hsin Wei Rajesh Rao Robert Tinn Naoto Usuyama ... Paul D. Tittel Naveen Valluri Tristan Naumann Carlo Bifulco Hoifung Poon 21 6 0 20 Mar 2022
BIOS: An Algorithmically Generated Biomedical Knowledge Graph Sheng Yu Zheng Yuan Jun Xia Shengxuan Luo Huaiyuan Ying ... Yucong Lin Keming Lu Jing Wang Yutao Xie H. Shum 31 16 0 18 Mar 2022
Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks Fatemehsadat Mireshghallah Kartik Goyal Archit Uniyal Taylor Berg-Kirkpatrick Reza Shokri MIALM 32 152 0 08 Mar 2022
BioADAPT-MRC: Adversarial Learning-based Domain Adaptation Improves Biomedical Machine Reading Comprehension Task Maria Mahbub Sudarshan Srinivasan Edmon Begoli Gregory D. Peterson 27 11 0 26 Feb 2022
TimeLMs: Diachronic Language Models from Twitter Daniel Loureiro Francesco Barbieri Leonardo Neves Luis Espinosa Anke Jose Camacho-Collados 41 249 0 08 Feb 2022