v1v2v3 (latest)

Pre-training Language Model Incorporating Domain-specific Heterogeneous Knowledge into A Unified Representation

2 September 2021

Lei Hou

Papers citing "Pre-training Language Model Incorporating Domain-specific Heterogeneous Knowledge into A Unified Representation"

27 / 27 papers shown

Title
MEG: Medical Knowledge-Augmented Large Language Models for Question Answering Laura Cabello Carmen Martin-Turrero Uchenna Akujuobi Anders Søgaard Carlos Bobed AI4MH 324 1 0 06 Nov 2024
HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models Bernal Jiménez Gutiérrez Yiheng Shu Yu Gu Michihiro Yasunaga Yu-Chuan Su RALM CLL 126 47 0 23 May 2024
ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation Yu Sun Shuohuan Wang Shikun Feng Siyu Ding Chao Pang ... Ouyang Xuan Dianhai Yu Hao Tian Hua Wu Haifeng Wang 112 473 0 05 Jul 2021
NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned Sewon Min Jordan L. Boyd-Graber Chris Alberti Danqi Chen Eunsol Choi ... Dmytro Okhonko Michael Schlichtkrull Sonal Gupta Yashar Mehdad Wen-tau Yih 76 62 0 01 Jan 2021
LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention Ikuya Yamada Akari Asai Hiroyuki Shindo Hideaki Takeda Yuji Matsumoto 118 672 0 02 Oct 2020
Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing Yu Gu Robert Tinn Hao Cheng Michael R. Lucas Naoto Usuyama Xiaodong Liu Tristan Naumann Jianfeng Gao Hoifung Poon LM&MA AI4CE 126 1,785 0 31 Jul 2020
TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data Pengcheng Yin Graham Neubig Wen-tau Yih Sebastian Riedel RALM LMTD 101 608 0 17 May 2020
ZeroShotCeres: Zero-Shot Relation Extraction from Semi-Structured Webpages Colin Lockard Prashant Shiralkar Xin Luna Dong Hannaneh Hajishirzi 40 56 0 14 May 2020
Don't Stop Pretraining: Adapt Language Models to Domains and Tasks Suchin Gururangan Ana Marasović Swabha Swayamdipta Kyle Lo Iz Beltagy Doug Downey Noah A. Smith VLM AI4CE CLL 167 2,440 0 23 Apr 2020
MT-Clinical BERT: Scaling Clinical Information Extraction with Multitask Learning Andriy Mulyar Bridget T. McInnes 69 56 0 21 Apr 2020
KdConv: A Chinese Multi-domain Dialogue Dataset Towards Multi-turn Knowledge-driven Conversation Hao Zhou Chujie Zheng Kaili Huang Minlie Huang Xiaoyan Zhu 73 123 0 08 Apr 2020
Continual Domain-Tuning for Pretrained Language Models Subendhu Rongali Abhyuday N. Jagannatha Bhanu Pratap Singh Rawat Hong-ye Yu CLL KELM 47 7 0 05 Apr 2020
K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters Ruize Wang Duyu Tang Nan Duan Zhongyu Wei Xuanjing Huang Jianshu Ji Guihong Cao Daxin Jiang Ming Zhou KELM 129 556 0 05 Feb 2020
Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model Wenhan Xiong Jingfei Du William Yang Wang Veselin Stoyanov SSL KELM 99 201 0 20 Dec 2019
KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation Xiaozhi Wang Tianyu Gao Zhaocheng Zhu Zhengyan Zhang Zhiyuan Liu Juan-Zi Li Jian Tang 128 672 0 13 Nov 2019
K-BERT: Enabling Language Representation with Knowledge Graph Weijie Liu Peng Zhou Zhe Zhao Zhiruo Wang Qi Ju Haotang Deng Ping Wang 313 793 0 17 Sep 2019
KG-BERT: BERT for Knowledge Graph Completion Liang Yao Chengsheng Mao Yuan Luo SLR 82 593 0 07 Sep 2019
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 587 2,680 0 03 Sep 2019
FinBERT: Financial Sentiment Analysis with Pre-trained Language Models Dogu Araci AIFin 122 647 0 27 Aug 2019
DocRED: A Large-Scale Document-Level Relation Extraction Dataset Yuan Yao Deming Ye Peng Li Xu Han Yankai Lin Zhenghao Liu Zhiyuan Liu Lixin Huang Jie Zhou Maosong Sun 61 455 0 14 Jun 2019
ERNIE: Enhanced Language Representation with Informative Entities Zhengyan Zhang Xu Han Zhiyuan Liu Xin Jiang Maosong Sun Qun Liu 121 1,402 0 17 May 2019
Logician: A Unified End-to-End Neural Approach for Open-Domain Information Extraction Mingming Sun Xu Li Xin Wang M. Fan Yue Feng Ping Li 85 56 0 29 Apr 2019
ERNIE: Enhanced Representation through Knowledge Integration Yu Sun Shuohuan Wang Yukun Li Shikun Feng Xuyi Chen Han Zhang Xin Tian Danxiang Zhu Hao Tian Hua Wu 121 906 0 19 Apr 2019
ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission Kexin Huang Jaan Altosaar Rajesh Ranganath OOD 133 909 0 10 Apr 2019
SciBERT: A Pretrained Language Model for Scientific Text Iz Beltagy Kyle Lo Arman Cohan 180 2,990 0 26 Mar 2019
BioBERT: a pre-trained biomedical language representation model for biomedical text mining Jinhyuk Lee Wonjin Yoon Sungdong Kim Donghyeon Kim Sunkyu Kim Chan Ho So Jaewoo Kang OOD 182 5,684 0 25 Jan 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 260 3,747 0 09 Jan 2019