A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery

16 June 2024

Yu Zhang

Xiusi Chen

Bowen Jin

Sheng Wang

Shuiwang Ji

Wei Wang

Jiawei Han

ArXiv PDF HTML

Papers citing "A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery"

50 / 111 papers shown

Title
From Theory to Practice: Advancing Multi-Robot Path Planning Algorithms and Applications Teng Guo 59 0 0 11 Jun 2025
Recitation over Reasoning: How Cutting-Edge Language Models Can Fail on Elementary School-Level Reasoning Problems? Kai Yan Yufei Xu Zhengyin Du Xuesong Yao Ziyi Wang Xiaowen Guo Jiecao Chen ReLM ELM LRM 153 5 0 01 Apr 2025
Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoning Yibo Yan Shen Wang Jiahao Huo Jingheng Ye Zhendong Chu Xuming Hu Philip S. Yu Carla P. Gomes B. Selman Qingsong Wen LRM 189 17 0 05 Feb 2025
BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs Sheng Zhang Yanbo Xu Naoto Usuyama Hanwen Xu J. Bagga ... Carlo Bifulco M. Lungren Tristan Naumann Sheng Wang Hoifung Poon LM&MA MedIm 194 226 0 10 Jan 2025
LSHBloom: Memory-efficient, Extreme-scale Document Deduplication A. Khan Robert Underwood Carlo Siebenschuh Y. Babuji Aswathy Ajith Kyle Hippe Ozan Gokdemir Alexander Brace Kyle Chard Ian Foster 55 0 0 06 Nov 2024
Panacea: A foundation model for clinical trial search, summarization, design, and recruitment J. Lin H. Xu Zifeng Wang Sheng Wang Jimeng Sun ELM LM&MA 66 9 0 25 Jun 2024
Efficient Evolutionary Search Over Chemical Space with Large Language Models Haorui Wang Marta Skreta C. Ser Wenhao Gao Lingkai Kong ... Yanqiao Zhu Yuanqi Du Alán Aspuru-Guzik Kirill Neklyudov Chao Zhang 77 17 0 23 Jun 2024
Transforming the Bootstrap: Using Transformers to Compute Scattering Amplitudes in Planar N = 4 Super Yang-Mills Theory Tianji Cai G. W. Merz Franccois Charton Niklas Nolte Matthias Wilhelm K. Cranmer Lance J. Dixon 153 16 0 09 May 2024
Advancing Multimodal Medical Capabilities of Gemini Lin Yang Shawn Xu Andrew Sellergren Timo Kohlberger Yuchen Zhou ... David Steiner Rory Pilgrim Christopher J. Kelly Shekoofeh Azizi Daniel Golden MedIm 67 66 0 06 May 2024
Rho-1: Not All Tokens Are What You Need Zheng-Wen Lin Zhibin Gou Yeyun Gong Xiao Liu Yelong Shen ... Chen Lin Yujiu Yang Jian Jiao Nan Duan Weizhu Chen CLL 85 68 0 11 Apr 2024
Wu's Method can Boost Symbolic AI to Rival Silver Medalists and AlphaGeometry to Outperform Gold Medalists at IMO Geometry Shiven Sinha Ameya Prabhu Ponnurangam Kumaraguru Siddharth Bhat Matthias Bethge AIMat 21 5 0 09 Apr 2024
TableLLM: Enabling Tabular Data Manipulation by LLMs in Real Office Usage Scenarios Yanling Wang Jing Zhang Zeyao Ma Yang Li Bohan Zhang ... Didong Li Shu Zhao Juan-Zi Li Jie Tang J. Tang LMTD RALM 71 32 0 28 Mar 2024
Large Language Model for Participatory Urban Planning Zhilun Zhou Yuming Lin Depeng Jin Yong Li LLMAG 83 29 0 27 Feb 2024
Towards Building Multilingual Language Model for Medicine Pengcheng Qiu Chaoyi Wu Xiaoman Zhang Weixiong Lin Haicheng Wang Ya Zhang Yanfeng Wang Weidi Xie LM&MA ELM 62 83 0 21 Feb 2024
ChemReasoner: Heuristic Search over a Large Language Model's Knowledge Space using Quantum-Chemical Feedback Henry W Sprueill Carl Edwards Khushbu Agarwal Mariefel V. Olarte Udishnu Sanyal Conrad Johnston Hongbin Liu Heng Ji Sutanay Choudhury LRM 76 9 0 15 Feb 2024
SceMQA: A Scientific College Entrance Level Multimodal Question Answering Benchmark Zhenwen Liang Kehan Guo Gang Liu Taicheng Guo Yujun Zhou Tianyu Yang Jiajun Jiao Renjie Pi Jipeng Zhang Xiangliang Zhang ELM 69 22 0 06 Feb 2024
A Survey of Pre-trained Language Models for Processing Scientific Text Xanh Ho Anh-Khoa Duong Nguyen Tuan-An Dao Junfeng Jiang Yuki Chida Kaito Sugimoto H. To Florian Boudin Akiko Aizawa 38 7 0 31 Jan 2024
AstroLLaMA-Chat: Scaling AstroLLaMA with Conversational and Diverse Datasets Ernest Perkowski Rui Pan Tuan Dung Nguyen Yuan-Sen Ting Sandor Kruk ... Michael J. Smith Huiling Liu Kevin Schawinski K. Iyer I. Ciucă AI4MH 61 12 0 03 Jan 2024
G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model Jiahui Gao Renjie Pi Jipeng Zhang Jiacheng Ye Wanjun Zhong ... Lanqing Hong Jianhua Han Hang Xu Zhenguo Li Lingpeng Kong SyDa ReLM LRM 86 114 0 18 Dec 2023
Large Language Models on Graphs: A Comprehensive Survey Bowen Jin Gang Liu Chi Han Meng Jiang Heng Ji Jiawei Han AI4CE 85 156 0 05 Dec 2023
BioCLIP: A Vision Foundation Model for the Tree of Life Samuel Stevens Jiaman Wu Matthew J Thompson Elizabeth G Campolongo Chan Hee Song ... Wasila M Dahdul Charles V. Stewart Tanya Berger-Wolf Wei-Lun Chao Yu-Chuan Su 86 73 0 30 Nov 2023
MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI Xiang Yue Yuansheng Ni Kai Zhang Tianyu Zheng Ruoqi Liu ... Yibo Liu Wenhao Huang Huan Sun Yu-Chuan Su Wenhu Chen OSLM ELM VLM 204 901 0 27 Nov 2023
HuatuoGPT-II, One-stage Training for Medical Adaption of LLMs Junying Chen Xidong Wang Anningzhe Gao Feng Jiang Shunian Chen ... Chuyi Kong Jianquan Li Xiang Wan Haizhou Li Benyou Wang LM&MA 67 66 0 16 Nov 2023
MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter Zhiyuan Liu Changhao Nai Yancheng Luo Hao Fei Yixin Cao Kenji Kawaguchi Xiang Wang Tat-Seng Chua 68 90 0 19 Oct 2023
MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning Ke Wang Houxing Ren Aojun Zhou Zimu Lu Sichun Luo Weikang Shi Renrui Zhang Linqi Song Mingjie Zhan Hongsheng Li ReLM LRM SyDa 88 101 0 05 Oct 2023
MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts Pan Lu Hritik Bansal Tony Xia Jiacheng Liu Chun-yue Li Hannaneh Hajishirzi Hao Cheng Kai-Wei Chang Michel Galley Jianfeng Gao LRM MLLM 107 614 0 03 Oct 2023
Can large language models provide useful feedback on research papers? A large-scale empirical analysis Weixin Liang Yuhui Zhang Hancheng Cao Binglu Wang Daisy Ding ... Siyu He D. Smith Yian Yin Daniel A. McFarland James Y. Zou ALM LM&MA 72 140 0 03 Oct 2023
MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning Xiang Yue Xingwei Qu Ge Zhang Yao Fu Wenhao Huang Huan Sun Yu-Chuan Su Wenhu Chen AIMat LRM 133 399 0 11 Sep 2023
Large Language Models for Automated Open-domain Scientific Hypotheses Discovery Zonglin Yang Xinya Du Junxian Li Jie Zheng Soujanya Poria Min Zhang LRM 48 54 0 06 Sep 2023
WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct Haipeng Luo Qingfeng Sun Can Xu Pu Zhao Jian-Guang Lou ... Xiubo Geng Qingwei Lin Shifeng Chen Yansong Tang Dongmei Zhang LRM OSLM 191 454 0 18 Aug 2023
GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Text Peng Liu Yiming Ren Jun Tao Zhixiang Ren AI4CE 59 82 0 14 Aug 2023
MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrieval Qiao Jin Won Kim Qingyu Chen Donald C. Comeau Lana Yeganova John Wilbur Zhiyong Lu LM&MA MedIm 63 111 0 02 Jul 2023
HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution Eric N. D. Nguyen Michael Poli Marjan Faizi A. Thomas Callum Birch-Sykes ... Stefano Massaroli Yoshua Bengio Stefano Ermon S. Baccus Christopher Ré MedIm 76 251 0 27 Jun 2023
DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genome Zhihan Zhou Yanrong Ji Weijian Li Pratik Dutta R. Davuluri Han Liu 63 188 0 26 Jun 2023
Quilt-1M: One Million Image-Text Pairs for Histopathology Wisdom O. Ikezogwo M. S. Seyfioglu Fatemeh Ghezloo Dylan Stefan Chan Geva Fatwir Sheikh Mohammed Pavan Kumar Anand Ranjay Krishna Linda G. Shapiro CLIP VLM 278 125 0 20 Jun 2023
Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Images Ming Y. Lu Bowen Chen Andrew Zhang Drew F. K. Williamson Richard J. Chen Tong Ding L. Le Yung-Sung Chuang Faisal Mahmood VLM MedIm 170 102 0 13 Jun 2023
K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization Cheng Deng Tianhang Zhang Zhongmou He Yi Xu Qiyuan Chen ... Weinan Zhang Xinbing Wang Cheng Zhou Zhouhan Lin Junxian He ALM 63 63 0 08 Jun 2023
Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understanding Yu Zhang Hao Cheng Zhihong Shen Xiaodong Liu Yejiang Wang Jianfeng Gao 57 14 0 23 May 2023
W-MAE: Pre-trained weather model with masked autoencoder for multi-variable weather forecasting Xin Man Chenghong Zhang Jin Feng Changyu Li Jie Shao AI4Cl 79 25 0 18 Apr 2023
FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Lead Kan Chen Tao Han Junchao Gong Lei Bai Fenghua Ling ... Rui Su Yuanzheng Ci Bin Li Xiaokang Yang Wanli Ouyang AI4Cl AI4CE 176 182 0 06 Apr 2023
Large Language Models as Master Key: Unlocking the Secrets of Materials Science with GPT Tong Xie Yuwei Wan Wei-Ping Huang Yufei Zhou Yixuan Liu ... Shaozhou Wang Chunyu Kit Clara Grazian Weinan Zhang Hoex 57 52 0 05 Apr 2023
PK-Chat: Pointer Network Guided Knowledge Driven Generative Dialogue Model Cheng Deng Bo Tong Luoyi Fu Jiaxin Ding Dexing Cao Xinbing Wang Chenghu Zhou 52 3 0 02 Apr 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 1.4K 14,313 0 15 Mar 2023
A Text-guided Protein Design Framework Shengchao Liu Yanjing Li Zhuoxinran Li A. Gitter Yutao Zhu ... Arvind Ramanathan Chaowei Xiao Jian Tang Hongyu Guo Anima Anandkumar 101 63 0 09 Feb 2023
Structure-informed Language Models Are Protein Designers Zaixiang Zheng Yifan Deng Dongyu Xue Yi Zhou YE Fei Quanquan Gu 78 95 0 03 Feb 2023
ProtST: Multi-Modality Learning of Protein Sequences and Biomedical Texts Minghao Xu Xinyu Yuan Santiago Miret Jian Tang AI4TS 68 102 0 28 Jan 2023
Ankh: Optimized Protein Language Model Unlocks General-Purpose Modelling Ahmed Elnaggar Hazem Essam Wafaa Salah-Eldin Walid Moustafa Mohamed Elkerdawy Charlotte Rochereau B. Rost 200 98 0 16 Jan 2023
Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing Shruthi Bannur Stephanie L. Hyland Qianchu Liu Fernando Pérez-García Maximilian Ilse ... Maria T. A. Wetscherek M. Lungren A. Nori Javier Alvarez-Valle Ozan Oktay 59 124 0 11 Jan 2023
BARTSmiles: Generative Masked Language Models for Molecular Representations Gayane Chilingaryan Hovhannes Tamoyan Ani Tevosyan N. Babayan L. Khondkaryan Karen Hambardzumyan Zaven Navoyan Hrant Khachatrian Armen Aghajanyan SSL 77 28 0 29 Nov 2022
ReasTAP: Injecting Table Reasoning Skills During Pre-training via Synthetic Reasoning Examples Yilun Zhao Linyong Nan Zhenting Qi Rui Zhang Dragomir R. Radev ReLM LMTD LRM 74 39 0 22 Oct 2022