v1v2 (latest)

A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery

16 June 2024

Yu Zhang

Xiusi Chen

Bowen Jin

Sheng Wang

Shuiwang Ji

Wei Wang

Jiawei Han

ArXiv (abs)PDF HTML Github (577★)

Papers citing "A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery"

23 / 123 papers shown

Title
TLDR: Extreme Summarization of Scientific Documents Isabel Cachola Kyle Lo Arman Cohan Daniel S. Weld 114 217 0 30 Apr 2020
SPECTER: Document-level Representation Learning using Citation-informed Transformers Arman Cohan Sergey Feldman Iz Beltagy Doug Downey Daniel S. Weld AI4TS 84 556 0 15 Apr 2020
Injecting Numerical Reasoning Skills into Language Models Mor Geva Ankit Gupta Jonathan Berant AIMat LRM 73 226 0 09 Apr 2020
TAPAS: Weakly Supervised Table Parsing via Pre-training Jonathan Herzig Pawel Krzysztof Nowak Thomas Müller Francesco Piccinno Julian Martin Eisenschlos LMTD RALM 110 655 0 05 Apr 2020
Molecule Attention Transformer Lukasz Maziarka Tomasz Danel Slawomir Mucha Krzysztof Rataj Jacek Tabor Stanislaw Jastrzebski 87 170 0 19 Feb 2020
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 485 20,317 0 23 Oct 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 677 24,541 0 26 Jul 2019
BEHRT: Transformer for Electronic Health Records Yikuan Li Shishir Rao J. R. A. Solares A. Hassaine D. Canoy Yajie Zhu K. Rahimi G. Salimi-Khorshidi OOD 110 464 0 22 Jul 2019
Invariant Risk Minimization Martín Arjovsky Léon Bottou Ishaan Gulrajani David Lopez-Paz OOD 195 2,242 0 05 Jul 2019
Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets Yifan Peng Shankai Yan Zhiyong Lu LM&MA AI4MH 71 845 0 13 Jun 2019
ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission Kexin Huang Jaan Altosaar Rajesh Ranganath OOD 124 909 0 10 Apr 2019
Publicly Available Clinical BERT Embeddings Emily Alsentzer John R. Murphy Willie Boag W. Weng Di Jin Tristan Naumann Matthew B. A. McDermott AI4MH 176 1,983 0 06 Apr 2019
Probing Biomedical Embeddings from Language Models Qiao Jin Bhuwan Dhingra William W. Cohen Xinghua Lu 65 116 0 03 Apr 2019
Structural Scaffolds for Citation Intent Classification in Scientific Publications Arman Cohan Bridger Waleed Ammar Madeleine van Zuylen Field Cady 67 253 0 02 Apr 2019
SciBERT: A Pretrained Language Model for Scientific Text Iz Beltagy Kyle Lo Arman Cohan 168 2,985 0 26 Mar 2019
BioBERT: a pre-trained biomedical language representation model for biomedical text mining Jinhyuk Lee Wonjin Yoon Sungdong Kim Donghyeon Kim Sunkyu Kim Chan Ho So Jaewoo Kang OOD 180 5,672 0 25 Jan 2019
PadChest: A large chest x-ray image dataset with multi-label annotated reports A. Bustos A. Pertusa J. M. Salinas M. Iglesia-Vayá LM&MA 73 614 0 22 Jan 2019
CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison Jeremy Irvin Pranav Rajpurkar M. Ko Yifan Yu Silviana Ciurea-Ilcus ... D. Larson C. Langlotz Bhavik Patel M. Lungren A. Ng 114 2,601 0 21 Jan 2019
Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction Yi Luan Luheng He Mari Ostendorf Hannaneh Hajishirzi 116 684 0 29 Aug 2018
Lessons from Natural Language Inference in the Clinical Domain Alexey Romanov Chaitanya P. Shivade LM&MA 80 272 0 21 Aug 2018
Construction of the Literature Graph in Semantic Scholar Bridger Waleed Ammar Dirk Groeneveld Chandra Bhagavatula Iz Beltagy Miles Crawford ... Lucy Lu Wang Christopher Wilhelm Zheng Yuan Madeleine van Zuylen Oren Etzioni GNN 76 396 0 06 May 2018
MoleculeNet: A Benchmark for Molecular Machine Learning Zhenqin Wu Bharath Ramsundar Evan N. Feinberg Joseph Gomes C. Geniesse Aneesh S. Pappu K. Leswing Vijay S. Pande OOD 337 1,837 0 02 Mar 2017
Semi-Supervised Classification with Graph Convolutional Networks Thomas Kipf Max Welling GNN SSL 662 29,156 0 09 Sep 2016