v1v2v3v4 (latest)

Pre-trained Language Models in Biomedical Domain: A Systematic Survey

11 October 2021

Papers citing "Pre-trained Language Models in Biomedical Domain: A Systematic Survey"

50 / 223 papers shown

Title
Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing Yu Gu Robert Tinn Hao Cheng Michael R. Lucas Naoto Usuyama Xiaodong Liu Tristan Naumann Jianfeng Gao Hoifung Poon LM&MA AI4CE 114 1,782 0 31 Jul 2020
Ethics of Artificial Intelligence in Surgery Frank Rudzicz Raeid Saqur SyDa 8 13 0 28 Jul 2020
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 556 2,099 0 28 Jul 2020
ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computing Ahmed Elnaggar M. Heinzinger Christian Dallago Ghalia Rehawi Yu Wang ... Tamas B. Fehér Christoph Angerer Martin Steinegger D. Bhowmik B. Rost DRL 66 951 0 13 Jul 2020
Transferability of Natural Language Inference to Biomedical Question Answering Minbyul Jeong Mujeen Sung Gangwoo Kim Donghyeon Kim Wonjin Yoon J. Yoo Jaewoo Kang 62 40 0 01 Jul 2020
Answering Questions on COVID-19 in Real-Time Jinhyuk Lee Sean S. Yi Minbyul Jeong Mujeen Sung Wonjin Yoon Yonghwa Choi Miyoung Ko Jaewoo Kang 68 43 0 29 Jun 2020
BERTology Meets Biology: Interpreting Attention in Protein Language Models Jesse Vig Ali Madani Lav Varshney Caiming Xiong R. Socher Nazneen Rajani 85 293 0 26 Jun 2020
Evaluation of Text Generation: A Survey Asli Celikyilmaz Elizabeth Clark Jianfeng Gao ELM LM&MA 115 387 0 26 Jun 2020
CO-Search: COVID-19 Information Retrieval with Semantic Search, Question Answering, and Abstractive Summarization A. Esteva Anuprit Kale Romain Paulus Kazuma Hashimoto Wenpeng Yin Dragomir R. Radev R. Socher 96 65 0 17 Jun 2020
Results of the seventh edition of the BioASQ Challenge A. Nentidis K. Bougiatiotis Anastasia Krithara George Giannakopoulos 75 62 0 16 Jun 2020
Document Classification for COVID-19 Literature Bernal Jiménez Gutiérrez Juncheng Zeng Dongdong Zhang Ping Zhang Yu-Chuan Su OOD 46 32 0 15 Jun 2020
Bootstrap your own latent: A new approach to self-supervised Learning Jean-Bastien Grill Florian Strub Florent Altché Corentin Tallec Pierre Harvey Richemond ... M. G. Azar Bilal Piot Koray Kavukcuoglu Rémi Munos Michal Valko SSL 395 6,837 0 13 Jun 2020
On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines Marius Mosbach Maksym Andriushchenko Dietrich Klakow 171 360 0 08 Jun 2020
Automatic Text Summarization of COVID-19 Medical Research Articles using BERT and GPT-2 V. Kieuvongngam Bowen Tan Yiming Niu AI4MH 42 96 0 03 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 874 42,379 0 28 May 2020
Language (Technology) is Power: A Critical Survey of "Bias" in NLP Su Lin Blodgett Solon Barocas Hal Daumé Hanna M. Wallach 157 1,248 0 28 May 2020
Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction L. Rasmy Yang Xiang Z. Xie Cui Tao Degui Zhi AI4MH LM&MA 99 693 0 22 May 2020
BERTweet: A pre-trained language model for English Tweets Dat Quoc Nguyen Thanh Tien Vu A. Nguyen VLM 99 919 0 20 May 2020
COVID-Twitter-BERT: A Natural Language Processing Model to Analyse COVID-19 Content on Twitter Martin Müller M. Salathé P. Kummervold VLM MedIm AI4MH 79 361 0 15 May 2020
Entity-Enriched Neural Models for Clinical Question Answering Bhanu Pratap Singh Rawat W. Weng So Yeon Min Preethi Raghavan Peter Szolovits 61 18 0 13 May 2020
On the Generation of Medical Dialogues for COVID-19 Wenmian Yang Guangtao Zeng Bowen Tan Zeqian Ju Subrato Chakravorty ... Xingyi Yang Qingyang Wu Zhou Yu Eric Xing P. Xie MedIm LM&MA 49 30 0 11 May 2020
Comparative Analysis of Text Classification Approaches in Electronic Health Records Aurelie Mascio Z. Kraljevic D. Bean Richard J. B. Dobson Robert Stewart R. Bendayan A. Roberts 67 46 0 08 May 2020
Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Models Bill Yuchen Lin Seyeon Lee Rahul Khanna Xiang Ren AIMat 73 158 0 02 May 2020
Attend to Medical Ontologies: Content Selection for Clinical Abstractive Summarization Sajad Sotudeh Nazli Goharian Ross W. Filice 60 56 0 01 May 2020
What Happens To BERT Embeddings During Fine-tuning? Amil Merchant Elahe Rahimtoroghi Ellie Pavlick Ian Tenney 78 188 0 29 Apr 2020
Rapidly Bootstrapping a Question Answering Dataset for COVID-19 Raphael Tang Rodrigo Nogueira Edwin Zhang Nikhil Gupta P. Cẩm Kyunghyun Cho Jimmy J. Lin 56 70 0 23 Apr 2020
Don't Stop Pretraining: Adapt Language Models to Domains and Tasks Suchin Gururangan Ana Marasović Swabha Swayamdipta Kyle Lo Iz Beltagy Doug Downey Noah A. Smith VLM AI4CE CLL 164 2,435 0 23 Apr 2020
CORD-19: The COVID-19 Open Research Dataset Lucy Lu Wang Kyle Lo Yoganand Chandrasekhar Russell Reas Jiangjiang Yang ... Boya Xie Douglas A. Raymond Daniel S. Weld Oren Etzioni Sebastian Kohlmeier 98 811 0 22 Apr 2020
Robustly Pre-trained Neural Model for Direct Temporal Relation Extraction Hong Guan Jianfu Li Hua Xu M. Devarakonda 15 10 0 13 Apr 2020
Longformer: The Long-Document Transformer Iz Beltagy Matthew E. Peters Arman Cohan RALM VLM 179 4,092 0 10 Apr 2020
DynaBERT: Dynamic BERT with Adaptive Width and Depth Lu Hou Zhiqi Huang Lifeng Shang Xin Jiang Xiao Chen Qun Liu MQ 81 322 0 08 Apr 2020
The Russian Drug Reaction Corpus and Neural Models for Drug Reactions and Effectiveness Detection in User Reviews E. Tutubalina I. Alimova Z. Miftahutdinov Andrey Sakhovskiy Valentin Malykh Sergey I. Nikolenko 53 44 0 07 Apr 2020
Inexpensive Domain Adaptation of Pretrained Language Models: Case Studies on Biomedical NER and Covid-19 QA Nina Poerner Ulli Waltinger Hinrich Schütze OOD 57 50 0 07 Apr 2020
MedDialog: Two Large-scale Medical Dialogue Datasets Xuehai He Shu Chen Zeqian Ju Xiangyu Dong Hongchao Fang ... Ruisi Zhang Ruoyu Zhang Meng Zhou Penghui Zhu P. Xie LM&MA MedIm 71 179 0 07 Apr 2020
Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers Zhicheng Huang Zhaoyang Zeng Bei Liu Dongmei Fu Jianlong Fu ViT 150 440 0 02 Apr 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 366 1,489 0 18 Mar 2020
A Survey on Contextual Embeddings Qi Liu Matt J. Kusner Phil Blunsom 253 150 0 16 Mar 2020
Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings H. Zhang Amy X. Lu Mohamed Abdalla Matthew B. A. McDermott Marzyeh Ghassemi 64 175 0 11 Mar 2020
ProGen: Language Modeling for Protein Generation Ali Madani Bryan McCann Nikhil Naik N. Keskar N. Anand Raphael R. Eguchi Po-Ssu Huang R. Socher 69 288 0 08 Mar 2020
AraBERT: Transformer-based Model for Arabic Language Understanding Wissam Antoun Fady Baly Hazem M. Hajj 132 973 0 28 Feb 2020
CodeBERT: A Pre-Trained Model for Programming and Natural Languages Zhangyin Feng Daya Guo Duyu Tang Nan Duan Xiaocheng Feng ... Linjun Shou Bing Qin Ting Liu Daxin Jiang Ming Zhou 165 2,667 0 19 Feb 2020
A Simple Framework for Contrastive Learning of Visual Representations Ting-Li Chen Simon Kornblith Mohammad Norouzi Geoffrey E. Hinton SSL 378 18,866 0 13 Feb 2020
MT-BioNER: Multi-task Learning for Biomedical Named Entity Recognition using Deep Bidirectional Transformers Muhammad Raza Khan Morteza Ziyadi M. Abdelhady MedIm 47 44 0 24 Jan 2020
Encoding word order in complex embeddings Benyou Wang Donghao Zhao Christina Lioma Qiuchi Li Peng Zhang J. Simonsen 59 113 0 27 Dec 2019
Clinical XLNet: Modeling Sequential Clinical Notes and Predicting Prolonged Mechanical Ventilation Kexin Huang Abhishek Singh Sitong Chen E. Moseley Chih-ying Deng Naomi George C. Lindvall 99 59 0 27 Dec 2019
End-to-end Named Entity Recognition and Relation Extraction using Pre-trained Language Models John Giorgi Xindi Wang Nicola Sahar W. Shin Gary D. Bader Bo Wang 67 37 0 20 Dec 2019
Self-Supervised Contextual Language Representation of Radiology Reports to Improve the Identification of Communication Urgency Xing Meng C. Ganoe R. Sieberg Yvonne Y. Cheung Saeid Hassanpour MedIm 29 12 0 05 Dec 2019
Momentum Contrast for Unsupervised Visual Representation Learning Kaiming He Haoqi Fan Yuxin Wu Saining Xie Ross B. Girshick SSL 213 12,124 0 13 Nov 2019
ConveRT: Efficient and Accurate Conversational Representations from Transformers Matthew Henderson I. Casanueva Nikola Mrkvsić Pei-hao Su Tsung-Hsien Ivan Vulić 95 200 0 09 Nov 2019
Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reports Yuhao Zhang Derek Merck E. Tsai Christopher D. Manning C. Langlotz MedIm HILM 62 186 0 06 Nov 2019