Towards understanding evolution of science through language model series

15 September 2024

Papers citing "Towards understanding evolution of science through language model series"

37 / 37 papers shown

Title
BioMedLM: A 2.7B Parameter Language Model Trained On Biomedical Text Elliot Bolton Abhinav Venigalla Michihiro Yasunaga David Leo Wright Hall Betty Xiong ... R. Daneshjou Jonathan Frankle Percy Liang Michael Carbin Christopher D. Manning LM&MA MedIm 61 59 0 27 Mar 2024
A Survey of Pre-trained Language Models for Processing Scientific Text Xanh Ho Anh-Khoa Duong Nguyen Tuan-An Dao Junfeng Jiang Yuki Chida Kaito Sugimoto H. To Florian Boudin Akiko Aizawa 38 7 0 31 Jan 2024
Time is Encoded in the Weights of Finetuned Language Models Kai Nylund Suchin Gururangan Noah A. Smith AI4TS 91 25 0 20 Dec 2023
Language Models Represent Space and Time Wes Gurnee Max Tegmark 104 157 0 03 Oct 2023
Mitigating Temporal Misalignment by Discarding Outdated Facts Michael J.Q. Zhang Eunsol Choi KELM HILM 60 20 0 24 May 2023
A Comprehensive Survey of Continual Learning: Theory, Method and Application Liyuan Wang Xingxing Zhang Hang Su Jun Zhu KELM CLL 151 679 0 31 Jan 2023
Editing Models with Task Arithmetic Gabriel Ilharco Marco Tulio Ribeiro Mitchell Wortsman Suchin Gururangan Ludwig Schmidt Hannaneh Hajishirzi Ali Farhadi KELM MoMe MU 182 493 0 08 Dec 2022
BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining Renqian Luo Liai Sun Yingce Xia Tao Qin Sheng Zhang Hoifung Poon Tie-Yan Liu MedIm AI4CE LM&MA 90 831 0 19 Oct 2022
ChemBERTa-2: Towards Chemical Foundation Models Walid Ahmad Elana Simon Seyone Chithrananda Gabriel Grand Bharath Ramsundar AI4CE 48 133 0 05 Sep 2022
SsciBERT: A Pre-trained Language Model for Social Science Texts Si Shen Jiang-fu Liu Litao Lin Ying Huang Lin Zhang Changting Liu Yu Feng Dongbo Wang 55 28 0 09 Jun 2022
The Diminishing Returns of Masked Language Models to Science Zhi Hong Aswathy Ajith Gregory Pauloski Eamon Duede Kyle Chard Ian Foster 73 25 0 23 May 2022
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time Mitchell Wortsman Gabriel Ilharco S. Gadre Rebecca Roelofs Raphael Gontijo-Lopes ... Hongseok Namkoong Ali Farhadi Y. Carmon Simon Kornblith Ludwig Schmidt MoMe 125 981 1 10 Mar 2022
Pretraining without Wordpieces: Learning Over a Vocabulary of Millions of Words Zhangyin Feng Duyu Tang Cong Zhou Junwei Liao Shuangzhi Wu Xiaocheng Feng Bing Qin Yunbo Cao Shuming Shi VLM 45 9 0 24 Feb 2022
Generalizing Graph Neural Networks on Out-Of-Distribution Graphs Shaohua Fan Xiao Wang Chuan Shi Peng Cui Bai Wang CML OOD OODD AI4CE 120 87 0 20 Nov 2021
Merging Models with Fisher-Weighted Averaging Michael Matena Colin Raffel FedML MoMe 87 390 0 18 Nov 2021
Time Waits for No One! Analysis and Challenges of Temporal Misalignment Kelvin Luu Daniel Khashabi Suchin Gururangan Karishma Mandyam Noah A. Smith 73 89 0 14 Nov 2021
Time-Aware Language Models as Temporal Knowledge Bases Bhuwan Dhingra Jeremy R. Cole Julian Martin Eisenschlos D. Gillick Jacob Eisenstein William W. Cohen KELM 90 279 0 29 Jun 2021
Large-Scale Chemical Language Representations Capture Molecular Structure and Properties Jerret Ross Brian M. Belgodere Vijil Chenthamarakshan Inkit Padhi Youssef Mroueh Payel Das AI4CE 55 288 0 17 Jun 2021
MathBERT: A Pre-Trained Model for Mathematical Formula Understanding Shuai Peng Ke Yuan Liangcai Gao Zhi Tang AIMat 81 108 0 02 May 2021
OAG-BERT: Towards A Unified Backbone Language Model For Academic Knowledge Services Xiao Liu Da Yin Jingnan Zheng Xingjian Zhang Peng Zhang Hongxia Yang Yuxiao Dong Jie Tang VLM 55 32 0 03 Mar 2021
Mind the Gap: Assessing Temporal Generalization in Neural Language Models Angeliki Lazaridou A. Kuncoro E. Gribovskaya Devang Agrawal Adam Liska ... Sebastian Ruder Dani Yogatama Kris Cao Susannah Young Phil Blunsom VLM 111 216 0 03 Feb 2021
Self-Adaptive Training: Bridging Supervised and Self-Supervised Learning Lang Huang Chaoning Zhang Hongyang R. Zhang SSL 57 25 0 21 Jan 2021
BioALBERT: A Simple and Effective Pre-trained Language Model for Biomedical Named Entity Recognition Usman Naseem Matloob Khushi V. Reddy S. Rajendran Imran Razzak Jinman Kim 58 63 0 19 Sep 2020
Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing Yu Gu Robert Tinn Hao Cheng Michael R. Lucas Naoto Usuyama Xiaodong Liu Tristan Naumann Jianfeng Gao Hoifung Poon LM&MA AI4CE 77 1,763 0 31 Jul 2020
Don't Stop Pretraining: Adapt Language Models to Domains and Tasks Suchin Gururangan Ana Marasović Swabha Swayamdipta Kyle Lo Iz Beltagy Doug Downey Noah A. Smith VLM AI4CE CLL 152 2,424 0 23 Apr 2020
ScienceExamCER: A High-Density Fine-Grained Science-Domain Corpus for Common Entity Recognition Hannah Smith Zeyu Zhang John M. Culnan Peter Alexander Jansen 20 10 0 24 Nov 2019
Stacking Models for Nearly Optimal Link Prediction in Complex Networks Amir Ghasemian Homa Hosseinmardi Aram Galstyan E. Airoldi A. Clauset 42 130 0 17 Sep 2019
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Nils Reimers Iryna Gurevych 1.3K 12,193 0 27 Aug 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 618 24,431 0 26 Jul 2019
Structural Scaffolds for Citation Intent Classification in Scientific Publications Arman Cohan Bridger Waleed Ammar Madeleine van Zuylen Field Cady 52 252 0 02 Apr 2019
SciBERT: A Pretrained Language Model for Scientific Text Iz Beltagy Kyle Lo Arman Cohan 130 2,969 0 26 Mar 2019
BioBERT: a pre-trained biomedical language representation model for biomedical text mining Jinhyuk Lee Wonjin Yoon Sungdong Kim Donghyeon Kim Sunkyu Kim Chan Ho So Jaewoo Kang OOD 156 5,648 0 25 Jan 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.7K 94,770 0 11 Oct 2018
Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction Yi Luan Luheng He Mari Ostendorf Hannaneh Hajishirzi 102 682 0 29 Aug 2018
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 687 131,526 0 12 Jun 2017
Inductive Representation Learning on Large Graphs William L. Hamilton Z. Ying J. Leskovec 501 15,232 0 07 Jun 2017
SemEval 2017 Task 10: ScienceIE - Extracting Keyphrases and Relations from Scientific Publications Isabelle Augenstein Mrinal Das Sebastian Riedel Lakshmi Vikraman Andrew McCallum 66 339 0 10 Apr 2017