SciBERT: A Pretrained Language Model for Scientific Text

26 March 2019

Kyle Lo

Arman Cohan

Papers citing "SciBERT: A Pretrained Language Model for Scientific Text"

50 / 454 papers shown

Title
Paying More Attention to Self-attention: Improving Pre-trained Language Models via Attention Guiding Shanshan Wang Zhumin Chen Zhaochun Ren Huasheng Liang Qiang Yan Pengjie Ren 33 9 0 06 Apr 2022
NC-DRE: Leveraging Non-entity Clue Information for Document-level Relation Extraction Li Zhang Yidong Cheng 13 2 0 01 Apr 2022
LinkBERT: Pretraining Language Models with Document Links Michihiro Yasunaga J. Leskovec Percy Liang KELM 29 353 0 29 Mar 2022
Don't Say What You Don't Know: Improving the Consistency of Abstractive Summarization by Constraining Beam Search Daniel King Zejiang Shen Nishant Subramani Daniel S. Weld Iz Beltagy Doug Downey HILM 28 31 0 16 Mar 2022
SciNLI: A Corpus for Natural Language Inference on Scientific Text Mobashir Sadat Cornelia Caragea AILaw 32 35 0 13 Mar 2022
FiNER: Financial Numeric Entity Recognition for XBRL Tagging Lefteris Loukas Manos Fergadiotis Ilias Chalkidis Eirini Spyropoulou Prodromos Malakasiotis Ion Androutsopoulos George Giannakopoulos 21 56 0 12 Mar 2022
AIFB-WebScience at SemEval-2022 Task 12: Relation Extraction First -- Using Relation Extraction to Identify Entities Nicholas Popovic Walter Laurito Michael Färber 33 10 0 10 Mar 2022
Adaptor: Objective-Centric Adaptation Framework for Language Models Michal vStefánik Vít Novotný Nikola Groverová Petr Sojka 35 10 0 08 Mar 2022
BioADAPT-MRC: Adversarial Learning-based Domain Adaptation Improves Biomedical Machine Reading Comprehension Task Maria Mahbub Sudarshan Srinivasan Edmon Begoli Gregory D. Peterson 27 11 0 26 Feb 2022
Deep Graph Learning for Anomalous Citation Detection Jiaying Liu Feng Xia Xu Feng Jing Ren Huan Liu 26 41 0 23 Feb 2022
Utilizing Out-Domain Datasets to Enhance Multi-Task Citation Analysis Dominique Mercier Syed Tahseen Raza Rizvi Vikas Rajashekar Sheraz Ahmed Andreas Dengel 13 1 0 22 Feb 2022
Identifying the Adoption or Rejection of Misinformation Targeting COVID-19 Vaccines in Twitter Discourse Maxwell Weinzierl S. Harabagiu 32 18 0 18 Feb 2022
Neighborhood Contrastive Learning for Scientific Document Representations with Citation Embeddings Malte Ostendorff Nils Rethmeier Isabelle Augenstein Bela Gipp Georg Rehm 28 73 0 14 Feb 2022
AdaPrompt: Adaptive Model Training for Prompt-based NLP Yulong Chen Yang Liu Li Dong Shuohang Wang Chenguang Zhu Michael Zeng Yue Zhang VLM 27 45 0 10 Feb 2022
TimeLMs: Diachronic Language Models from Twitter Daniel Loureiro Francesco Barbieri Leonardo Neves Luis Espinosa Anke Jose Camacho-Collados 41 249 0 08 Feb 2022
HistBERT: A Pre-trained Language Model for Diachronic Lexical Semantic Analysis Wen Qiu Yang Xu 6 18 0 08 Feb 2022
The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymization Ildikó Pilán Pierre Lison Lilja Ovrelid Anthia Papadopoulou David Sánchez Montserrat Batet AILaw 40 78 0 25 Jan 2022
Improving Chest X-Ray Report Generation by Leveraging Warm Starting Aaron Nicolson Jason Dowling Bevan Koopman ViT LM&MA MedIm 30 90 0 24 Jan 2022
SciBERTSUM: Extractive Summarization for Scientific Documents Athar Sefid C. Lee Giles 28 10 0 21 Jan 2022
Why Did You Not Compare With That? Identifying Papers for Use as Baselines Manjot Bedi Tanisha Pandey S. Bhatia Tanmoy Chakraborty 24 7 0 20 Jan 2022
TourBERT: A pretrained language model for the tourism industry Veronika Arefieva R. Egger 14 4 0 19 Jan 2022
Transferability in Deep Learning: A Survey Junguang Jiang Yang Shu Jianmin Wang Mingsheng Long OOD 34 101 0 15 Jan 2022
Establishing Strong Baselines for TripClick Health Retrieval Sebastian Hofstatter Sophia Althammer Mete Sertkan Allan Hanbury MedIm 36 12 0 02 Jan 2022
LAME: Layout Aware Metadata Extraction Approach for Research Articles Jonghyun Choi Hyesoo Kong Hwamook Yoon Heung-Seon Oh Yuchul Jung 33 3 0 23 Dec 2021
Efficient Hierarchical Domain Adaptation for Pretrained Language Models Alexandra Chronopoulou Matthew E. Peters Jesse Dodge 33 42 0 16 Dec 2021
Generating Fluent Fact Checking Explanations with Unsupervised Post-Editing Shailza Jolly Pepa Atanasova Isabelle Augenstein 38 13 0 13 Dec 2021
A Scoping Review of Publicly Available Language Tasks in Clinical Natural Language Processing Yanjun Gao Dmitriy Dligach Leslie A. Christensen S. Tesch Ryan Laffin Dongfang Xu Timothy A. Miller Özlem Uzuner M. Churpek Majid Afshar LM&MA 41 25 0 07 Dec 2021
Explainable Deep Learning in Healthcare: A Methodological Survey from an Attribution View Di Jin Elena Sergeeva W. Weng Geeticka Chauhan Peter Szolovits OOD 53 55 0 05 Dec 2021
MultiVerS: Improving scientific claim verification with weak supervision and full-document context David Wadden Bertie Vidgen Lucy Lu Wang Dirk Hovy J. Pierrehumbert Hannaneh Hajishirzi 31 153 0 02 Dec 2021
Local Citation Recommendation with Hierarchical-Attention Text Encoder and SciBERT-based Reranking Nianlong Gu Yingqiang Gao Richard H. R. Hahnloser 29 22 0 02 Dec 2021
Building astroBERT, a language model for Astronomy & Astrophysics Félix Grèzes Sergi Blanco-Cuaresma Alberto Accomazzi Michael J. Kurtz Golnaz Shapurian ... Kelly E. Lockhart N. Martinovic Shinyi Chen Christy Tanner P. Protopapas 11 20 0 01 Dec 2021
NER-BERT: A Pre-trained Model for Low-Resource Entity Tagging Zihan Liu Feijun Jiang Yuxiang Hu Chen Shi Pascale Fung 22 37 0 01 Dec 2021
Easy Semantification of Bioassays Marco Anteghini Jennifer D'Souza V. A. M. D. Santos Sören Auer 31 4 0 30 Nov 2021
Does constituency analysis enhance domain-specific pre-trained BERT models for relation extraction? Anfu Tang Louise Deléger Robert Bossy Pierre Zweigenbaum Claire Nédellec 17 2 0 25 Nov 2021
Exploring Language Patterns in a Medical Licensure Exam Item Bank Swati Padhee Kimberly Swygert Ian Micir 21 1 0 20 Nov 2021
Testing the Generalization of Neural Language Models for COVID-19 Misinformation Detection Jan Philip Wahle Nischal Ashok Kumar Terry Ruas Norman Meuschke Tirthankar Ghosal Bela Gipp 38 17 0 15 Nov 2021
Cross-Lingual Citations in English Papers: A Large-Scale Analysis of Prevalence, Usage, and Impact T. Saier Michael Färber Tornike Tsereteli 31 9 0 07 Nov 2021
Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey Bonan Min Hayley L Ross Elior Sulem Amir Pouran Ben Veyseh Thien Huu Nguyen Oscar Sainz Eneko Agirre Ilana Heinz Dan Roth LM&MA VLM AI4CE 83 1,038 0 01 Nov 2021
FinEAS: Financial Embedding Analysis of Sentiment Asier Gutiérrez-Fandiño M. N. Alonso P. Kolm Jordi Armengol-Estapé AIFin 14 6 0 31 Oct 2021
SciClops: Detecting and Contextualizing Scientific Claims for Assisting Manual Fact-Checking Panayiotis Smeros Carlos Castillo Karl Aberer 31 11 0 25 Oct 2021
SciCap: Generating Captions for Scientific Figures Ting-Yao Hsu C. Lee Giles Ting-Hao 'Kenneth' Huang 27 83 0 22 Oct 2021
Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora Xisen Jin Dejiao Zhang Henghui Zhu Wei Xiao Shang-Wen Li Xiaokai Wei Andrew O. Arnold Xiang Ren KELM CLL 31 113 0 16 Oct 2021
Building Chinese Biomedical Language Models via Multi-Level Text Discrimination Quan Wang Songtai Dai Benfeng Xu Yajuan Lyu Yong Zhu Hua Wu Haifeng Wang 71 14 0 14 Oct 2021
K-Wav2vec 2.0: Automatic Speech Recognition based on Joint Decoding of Graphemes and Syllables Jounghee Kim Pilsung Kang VLM 29 6 0 11 Oct 2021
Advances in Multi-turn Dialogue Comprehension: A Survey ZhuoSheng Zhang Hai Zhao 29 21 0 11 Oct 2021
Self-Supervised Knowledge Assimilation for Expert-Layman Text Style Transfer Wenda Xu Michael Stephen Saxon Misha Sra Wenjie Wang MedIm 19 13 0 06 Oct 2021
JuriBERT: A Masked-Language Model Adaptation for French Legal Text S. Douka Hadi Abdine Michalis Vazirgiannis Rajaa El Hamdani David Restrepo Amariles AILaw 87 31 0 04 Oct 2021
MatSciBERT: A Materials Domain Language Model for Text Mining and Information Extraction Tanishq Gupta Mohd Zaki N. M. A. Krishnan Mausam 55 178 0 30 Sep 2021
Symbolic Brittleness in Sequence Models: on Systematic Generalization in Symbolic Mathematics Sean Welleck Peter West Jize Cao Yejin Choi 32 28 0 28 Sep 2021
Text2Brain: Synthesis of Brain Activation Maps from Free-form Text Query G. Ngo Minh Le Nguyen Nancy F. Chen M. Sabuncu 27 4 0 28 Sep 2021