LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention

2 October 2020

ArXiv (abs)PDF HTML Github (724★)

Papers citing "LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention"

36 / 36 papers shown

Title
Comparative Performance of Advanced NLP Models and LLMs in Multilingual Geo-Entity Detection Kalin Kopanov 65 4 0 31 Dec 2024
ReLiK: Retrieve and LinK, Fast and Accurate Entity Linking and Relation Extraction on an Academic Budget Adam Gould Pere-Lluis Huguet-Cabot S. Dadhania Francesca Toni 158 9 0 31 Jul 2024
K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters Ruize Wang Duyu Tang Nan Duan Zhongyu Wei Xuanjing Huang Jianshu Ji Guihong Cao Daxin Jiang Ming Zhou KELM 120 556 0 05 Feb 2020
Learning Cross-Context Entity Representations from Text Jeffrey Ling Nicholas FitzGerald Zifei Shan Livio Baldini Soares Thibault Févry David J. Weiss Tom Kwiatkowski 47 32 0 11 Jan 2020
Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model Wenhan Xiong Jingfei Du William Yang Wang Veselin Stoyanov SSL KELM 95 201 0 20 Dec 2019
KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation Xiaozhi Wang Tianyu Gao Zhaocheng Zhu Zhengyan Zhang Zhiyuan Liu Juan-Zi Li Jian Tang 121 668 0 13 Nov 2019
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension M. Lewis Yinhan Liu Naman Goyal Marjan Ghazvininejad Abdel-rahman Mohamed Omer Levy Veselin Stoyanov Luke Zettlemoyer AIMat VLM 264 10,851 0 29 Oct 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 475 20,317 0 23 Oct 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 373 6,467 0 26 Sep 2019
Knowledge Enhanced Contextual Word Representations Matthew E. Peters Mark Neumann IV RobertL.Logan Roy Schwartz Vidur Joshi Sameer Singh Noah A. Smith 279 660 0 09 Sep 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 677 24,541 0 26 Jul 2019
SpanBERT: Improving Pre-training by Representing and Predicting Spans Mandar Joshi Danqi Chen Yinhan Liu Daniel S. Weld Luke Zettlemoyer Omer Levy 153 1,967 0 24 Jul 2019
XLNet: Generalized Autoregressive Pretraining for Language Understanding Zhilin Yang Zihang Dai Yiming Yang J. Carbonell Ruslan Salakhutdinov Quoc V. Le AI4CE 236 8,447 0 19 Jun 2019
What Does BERT Look At? An Analysis of BERT's Attention Kevin Clark Urvashi Khandelwal Omer Levy Christopher D. Manning MILM 226 1,602 0 11 Jun 2019
Matching the Blanks: Distributional Similarity for Relation Learning Livio Baldini Soares Nicholas FitzGerald Jeffrey Ling Tom Kwiatkowski 64 773 0 07 Jun 2019
Visualizing and Measuring the Geometry of BERT Andy Coenen Emily Reif Ann Yuan Been Kim Adam Pearce F. Viégas Martin Wattenberg MILM 78 418 0 06 Jun 2019
ERNIE: Enhanced Language Representation with Informative Entities Zhengyan Zhang Xu Han Zhiyuan Liu Xin Jiang Maosong Sun Qun Liu 109 1,400 0 17 May 2019
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems Alex Jinpeng Wang Yada Pruksachatkun Nikita Nangia Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 274 2,323 0 02 May 2019
Cloze-driven Pretraining of Self-attention Networks Alexei Baevski Sergey Edunov Yinhan Liu Luke Zettlemoyer Michael Auli 47 198 0 19 Mar 2019
ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehension Sheng Zhang Xiaodong Liu Jingjing Liu Jianfeng Gao Kevin Duh Benjamin Van Durme 81 314 0 30 Oct 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,175 0 11 Oct 2018
Graph Convolution over Pruned Dependency Trees Improves Relation Extraction Yuhao Zhang Peng Qi Christopher D. Manning GNN 136 728 0 26 Sep 2018
Ultra-Fine Entity Typing Eunsol Choi Omer Levy Yejin Choi Luke Zettlemoyer AI4TS 65 210 0 13 Jul 2018
Deep contextualized word representations Matthew E. Peters Mark Neumann Mohit Iyyer Matt Gardner Christopher Clark Kenton Lee Luke Zettlemoyer NAI 233 11,566 0 15 Feb 2018
Simple and Effective Multi-Paragraph Reading Comprehension Christopher Clark Matt Gardner RALM 99 459 0 29 Oct 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 783 132,363 0 12 Jun 2017
Learning Distributed Representations of Texts and Entities from Knowledge Base Ikuya Yamada Hiroyuki Shindo Hideaki Takeda Yoshiyasu Takefuji 56 98 0 06 May 2017
Deep Joint Entity Disambiguation with Local Neural Attention O. Ganea Thomas Hofmann DRL FedML 62 328 0 17 Apr 2017
Bidirectional Attention Flow for Machine Comprehension Minjoon Seo Aniruddha Kembhavi Ali Farhadi Hannaneh Hajishirzi 133 2,091 0 05 Nov 2016
Layer Normalization Jimmy Lei Ba J. Kiros Geoffrey E. Hinton 426 10,531 0 21 Jul 2016
Gaussian Error Linear Units (GELUs) Dan Hendrycks Kevin Gimpel 174 5,042 0 27 Jun 2016
Complex Embeddings for Simple Link Prediction Théo Trouillon Johannes Welbl Sebastian Riedel Éric Gaussier Guillaume Bouchard BDL 93 2,984 0 20 Jun 2016
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 316 8,169 0 16 Jun 2016
Neural Architectures for Named Entity Recognition Guillaume Lample Miguel Ballesteros Sandeep Subramanian Kazuya Kawakami Chris Dyer 226 4,016 0 04 Mar 2016
Joint Learning of the Embedding of Words and Entities for Named Entity Disambiguation Ikuya Yamada Hiroyuki Shindo Hideaki Takeda Yoshiyasu Takefuji 195 321 0 06 Jan 2016
Embedding Entities and Relations for Learning and Inference in Knowledge Bases Bishan Yang Wen-tau Yih Xiaodong He Jianfeng Gao Li Deng NAI 107 3,203 0 20 Dec 2014