SpanBERT: Improving Pre-training by Representing and Predicting Spans

24 July 2019

Luke Zettlemoyer

Papers citing "SpanBERT: Improving Pre-training by Representing and Predicting Spans"

50 / 950 papers shown

Title
T3-Vis: a visual analytic framework for Training and fine-Tuning Transformers in NLP Raymond Li Wen Xiao Lanjun Wang Hyeju Jang Giuseppe Carenini ViT 31 23 0 31 Aug 2021
Towards Consistent Document-level Entity Linking: Joint Models for Entity Linking and Coreference Resolution Klim Zaporojets Johannes Deleu Yiwei Jiang Thomas Demeester Chris Develder 29 10 0 30 Aug 2021
NEREL: A Russian Dataset with Nested Named Entities, Relations and Events Natalia Loukachevitch Ekaterina Artemova Tatiana Batura Pavel Braslavski Ilia Denisov V. Ivanov S. Manandhar Alexander Pugachev E. Tutubalina NAI 19 18 0 30 Aug 2021
Span Fine-tuning for Pre-trained Language Models Rongzhou Bao Zhuosheng Zhang Hai Zhao 14 2 0 29 Aug 2021
SMedBERT: A Knowledge-Enhanced Pre-trained Language Model with Structured Semantics for Medical Text Mining Taolin Zhang Zerui Cai Chengyu Wang Minghui Qiu Bite Yang Xiaofeng He AI4MH 28 52 0 20 Aug 2021
MvSR-NAT: Multi-view Subset Regularization for Non-Autoregressive Machine Translation Pan Xie Zexian Li Xiaohui Hu 34 11 0 19 Aug 2021
Zero-shot Task Transfer for Invoice Extraction via Class-aware QA Ensemble Prithiviraj Damodaran Prabhkaran Singh Josemon Achankuju 22 2 0 13 Aug 2021
AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processing Katikapalli Subramanyam Kalyan A. Rajasekharan S. Sangeetha VLM LM&MA 31 261 0 12 Aug 2021
BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documents Teakgyu Hong Donghyun Kim Mingi Ji Wonseok Hwang Daehyun Nam Sungrae Park VLM 34 152 0 10 Aug 2021
Musical Speech: A Transformer-based Composition Tool Jason dÉon Sri Harsha Dumpala Chandramouli Shama Sastry Daniel Oore Sageev Oore 20 1 0 02 Aug 2021
LICHEE: Improving Language Model Pre-training with Multi-grained Tokenization Weidong Guo Mingjun Zhao Lusheng Zhang Di Niu Jinwen Luo Zhenhua Liu Zhenyang Li J. Tang 32 8 0 02 Aug 2021
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing Pengfei Liu Weizhe Yuan Jinlan Fu Zhengbao Jiang Hiroaki Hayashi Graham Neubig VLM SyDa 102 3,858 0 28 Jul 2021
Multilingual Coreference Resolution with Harmonized Annotations O. Pražák Miloslav Konopík Jakub Sido 17 16 0 26 Jul 2021
Context-aware Adversarial Training for Name Regularity Bias in Named Entity Recognition Abbas Ghaddar Philippe Langlais Ahmad Rashid Mehdi Rezagholizadeh 44 43 0 24 Jul 2021
Target-Oriented Fine-tuning for Zero-Resource Named Entity Recognition Ying Zhang Fandong Meng Jinan Xu Jinan Xu Jie Zhou 35 10 0 22 Jul 2021
Improving Sentence-Level Relation Extraction through Curriculum Learning Seongsik Park Harksoo Kim 16 14 0 20 Jul 2021
BERT-like Pre-training for Symbolic Piano Music Classification Tasks Yi-Hui Chou I-Chun Chen Chin-Jui Chang Joann Ching Yi-Hsuan Yang 48 25 0 12 Jul 2021
ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation Yu Sun Shuohuan Wang Shikun Feng Siyu Ding Chao Pang ... Ouyang Xuan Dianhai Yu Hao Tian Hua Wu Haifeng Wang 61 454 0 05 Jul 2021
End-to-end Neural Coreference Resolution Revisited: A Simple yet Effective Baseline T. Lai Trung Bui Doo Soon Kim 20 12 0 04 Jul 2021
CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understanding Dong Wang Ning Ding Pijian Li Haitao Zheng AAML 39 115 0 01 Jul 2021
XLM-E: Cross-lingual Language Model Pre-training via ELECTRA Zewen Chi Shaohan Huang Li Dong Shuming Ma Bo Zheng ... Payal Bajaj Xia Song Xian-Ling Mao Heyan Huang Furu Wei 56 119 0 30 Jun 2021
ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information Zijun Sun Xiaoya Li Xiaofei Sun Yuxian Meng Xiang Ao Qing He Fei Wu Jiwei Li SSeg 57 184 0 30 Jun 2021
SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption Dara Bahri Heinrich Jiang Yi Tay Donald Metzler SSL 31 164 0 29 Jun 2021
Overview of BioASQ 2020: The eighth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering A. Nentidis Anastasia Krithara K. Bougiatiotis Martin Krallinger Carlos Rodríguez-Penagos Marta Villegas George Giannakopoulos 20 32 0 28 Jun 2021
WVOQ at SemEval-2021 Task 6: BART for Span Detection and Classification Cees Roele 33 3 0 27 Jun 2021
OKGIT: Open Knowledge Graph Link Prediction with Implicit Types Chandrahas Partha P. Talukdar 29 15 0 24 Jun 2021
LV-BERT: Exploiting Layer Variety for BERT Weihao Yu Zihang Jiang Fei Chen Qibin Hou Jiashi Feng MQ 26 0 0 22 Jun 2021
Towards Long-Form Video Understanding Chaoxia Wu Philipp Krahenbuhl VLM ViT 59 166 0 21 Jun 2021
VIMPAC: Video Pre-Training via Masked Token Prediction and Contrastive Learning Hao Tan Jie Lei Thomas Wolf Joey Tianyi Zhou 39 65 0 21 Jun 2021
Out of Context: A New Clue for Context Modeling of Aspect-based Sentiment Analysis Bowen Xing Ivor W. Tsang 30 11 0 21 Jun 2021
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models Elad Ben-Zaken Shauli Ravfogel Yoav Goldberg 100 1,164 0 18 Jun 2021
Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning Colin Wei Sang Michael Xie Tengyu Ma 24 97 0 17 Jun 2021
Coreference-Aware Dialogue Summarization Zhengyuan Liu Ke Shi Nancy F. Chen 29 59 0 16 Jun 2021
BEiT: BERT Pre-Training of Image Transformers Hangbo Bao Li Dong Songhao Piao Furu Wei ViT 68 2,751 0 15 Jun 2021
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units Wei-Ning Hsu Benjamin Bolte Yao-Hung Hubert Tsai Kushal Lakhotia Ruslan Salakhutdinov Abdel-rahman Mohamed SSL 55 2,806 0 14 Jun 2021
SAS: Self-Augmentation Strategy for Language Model Pre-training Yifei Xu Jingqiao Zhang Ru He Liangzhu Ge Chao Yang Cheng Yang Ying Wu 42 1 0 14 Jun 2021
Pre-Trained Models: Past, Present and Future Xu Han Zhengyan Zhang Ning Ding Yuxian Gu Xiao Liu ... Jie Tang Ji-Rong Wen Jinhui Yuan Wayne Xin Zhao Jun Zhu AIFin MQ AI4MH 58 818 0 14 Jun 2021
Can Transformer Language Models Predict Psychometric Properties? Antonio Laverghetta Animesh Nighojkar Jamshidbek Mirzakhalov John Licato LM&MA 38 14 0 12 Jun 2021
Hybrid Generative-Contrastive Representation Learning Saehoon Kim Sungwoong Kim Juho Lee SSL 22 11 0 11 Jun 2021
CAT: Cross Attention in Vision Transformer Hezheng Lin Xingyi Cheng Xiangyu Wu Fan Yang Dong Shen Zhongyuan Wang Qing Song Wei Yuan ViT 35 149 0 10 Jun 2021
Linguistically Informed Masking for Representation Learning in the Patent Domain Sophia Althammer Mark Buckley Sebastian Hofstatter Allan Hanbury 45 11 0 10 Jun 2021
MusicBERT: Symbolic Music Understanding with Large-Scale Pre-Training Mingliang Zeng Xu Tan Rui Wang Zeqian Ju Tao Qin Tie-Yan Liu 22 129 0 10 Jun 2021
Bayesian Attention Belief Networks Shujian Zhang Xinjie Fan Bo Chen Mingyuan Zhou BDL 32 30 0 09 Jun 2021
MERLOT: Multimodal Neural Script Knowledge Models Rowan Zellers Ximing Lu Jack Hessel Youngjae Yu J. S. Park Jize Cao Ali Farhadi Yejin Choi VLM LRM 38 374 0 04 Jun 2021
Neural semi-Markov CRF for Monolingual Word Alignment Wuwei Lan Chao Jiang Wei Xu BDL 14 18 0 04 Jun 2021
ERNIE-Tiny : A Progressive Distillation Framework for Pretrained Transformer Compression Weiyue Su Xuyi Chen Shi Feng Jiaxiang Liu Weixin Liu Yu Sun Hao Tian Hua Wu Haifeng Wang 34 13 0 04 Jun 2021
Transformers are Deep Infinite-Dimensional Non-Mercer Binary Kernel Machines Matthew A. Wright Joseph E. Gonzalez 42 20 0 02 Jun 2021
Cross-document Coreference Resolution over Predicted Mentions Arie Cattan Alon Eirew Gabriel Stanovsky Mandar Joshi Ido Dagan 26 40 0 02 Jun 2021
RevCore: Review-augmented Conversational Recommendation Yu Lu Junwei Bao Yan Song Zichen Ma Shuguang Cui Youzheng Wu Xiaodong He 35 74 0 02 Jun 2021
OntoGUM: Evaluating Contextualized SOTA Coreference Resolution on 12 More Genres Yilun Zhu Sameer Pradhan Amir Zeldes 24 22 0 02 Jun 2021