SpanBERT: Improving Pre-training by Representing and Predicting Spans

24 July 2019

Luke Zettlemoyer

Papers citing "SpanBERT: Improving Pre-training by Representing and Predicting Spans"

50 / 950 papers shown

Title
Learning to Ignore: Long Document Coreference with Bounded Memory Neural Networks Shubham Toshniwal Sam Wiseman Allyson Ettinger Karen Livescu Kevin Gimpel 26 48 0 06 Oct 2020
Neural Mask Generator: Learning to Generate Adaptive Word Maskings for Language Model Adaptation Minki Kang Moonsu Han Sung Ju Hwang OOD 25 18 0 06 Oct 2020
PMI-Masking: Principled masking of correlated spans Yoav Levine Barak Lenz Opher Lieber Omri Abend Kevin Leyton-Brown Moshe Tennenholtz Y. Shoham 22 72 0 05 Oct 2020
On Losses for Modern Language Models Stephane Aroca-Ouellette Frank Rudzicz 22 32 0 04 Oct 2020
LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention Ikuya Yamada Akari Asai Hiroyuki Shindo Hideaki Takeda Yuji Matsumoto 34 663 0 02 Oct 2020
Unsupervised Text Style Transfer with Padded Masked Language Models Eric Malmi Aliaksei Severyn S. Rothe 22 11 0 02 Oct 2020
Which *BERT? A Survey Organizing Contextualized Encoders Patrick Xia Shijie Wu Benjamin Van Durme 26 50 0 02 Oct 2020
MaP: A Matrix-based Prediction Approach to Improve Span Extraction in Machine Reading Comprehension Huaishao Luo Yu Shi Ming Gong Linjun Shou Tianrui Li 14 4 0 29 Sep 2020
CokeBERT: Contextual Knowledge Selection and Embedding towards Enhanced Pre-Trained Language Models Yusheng Su Xu Han Zhengyan Zhang Peng Li Zhiyuan Liu Yankai Lin Jie Zhou Maosong Sun ODL 23 24 0 29 Sep 2020
A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generation Dinghan Shen Ming Zheng Yelong Shen Yanru Qu Weizhu Chen AAML 29 130 0 29 Sep 2020
Incomplete Utterance Rewriting as Semantic Segmentation Qian Liu Bei Chen Jian-Guang Lou Bin Zhou Dongmei Zhang 20 56 0 28 Sep 2020
SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrieval Tiancheng Zhao Xiaopeng Lu Kyusong Lee RALM 21 86 0 28 Sep 2020
Unsupervised Pre-training for Biomedical Question Answering Vaishnavi Kommaraju K. Gunasekaran Kun Li Trapit Bansal Andrew McCallum Ivana Williams Ana-Maria Istrate SSL MedIm 22 18 0 27 Sep 2020
A Brief Survey and Comparative Study of Recent Development of Pronoun Coreference Resolution Hongming Zhang Xinran Zhao Yangqiu Song 24 9 0 27 Sep 2020
Revealing the Myth of Higher-Order Inference in Coreference Resolution Liyan Xu Jinho Choi 26 101 0 25 Sep 2020
MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems Zhaojiang Lin Andrea Madotto Genta Indra Winata Pascale Fung 40 172 0 25 Sep 2020
Type B Reflexivization as an Unambiguous Testbed for Multilingual Multi-Task Gender Bias Ana Valeria González Maria Barrett Rasmus Hvingelby Kellie Webster Anders Søgaard 21 26 0 24 Sep 2020
Streamlining Cross-Document Coreference Resolution: Evaluation and Modeling Arie Cattan Alon Eirew Gabriel Stanovsky Mandar Joshi Ido Dagan 21 35 0 23 Sep 2020
AutoRC: Improving BERT Based Relation Classification Models via Architecture Search Wei-wei Zhu Xipeng Qiu Yuan Ni Guotong Xie 24 18 0 22 Sep 2020
Latin BERT: A Contextual Language Model for Classical Philology David Bamman P. Burns 22 79 0 21 Sep 2020
Improving Robustness and Generality of NLP Models Using Disentangled Representations Jiawei Wu Xiaoya Li Xiang Ao Yuxian Meng Fei Wu Jiwei Li OOD DRL 16 11 0 21 Sep 2020
Understanding Mention Detector-Linker Interaction in Neural Coreference Resolution Zhaofeng Wu Matt Gardner 28 9 0 20 Sep 2020
Knowledge Transfer via Pre-training for Recommendation: A Review and Prospect Zheni Zeng Chaojun Xiao Yuan Yao Ruobing Xie Zhiyuan Liu Fen Lin Leyu Lin Maosong Sun VLM 35 40 0 19 Sep 2020
Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data Jonathan Pilault Amine Elhattami C. Pal CLL MoE 30 89 0 19 Sep 2020
Improving Coreference Resolution by Leveraging Entity-Centric Features with Graph Neural Networks and Second-order Inference Lu Liu Zhenqiao Song Xiaoqing Zheng Jun He 23 6 0 10 Sep 2020
QED: A Framework and Dataset for Explanations in Question Answering Matthew Lamm J. Palomaki Chris Alberti D. Andor Eunsol Choi Livio Baldini Soares Michael Collins 18 68 0 08 Sep 2020
E-BERT: A Phrase and Product Knowledge Enhanced Language Model for E-commerce Denghui Zhang Zixuan Yuan Yanchi Liu Fuzhen Zhuang Haifeng Chen Hui Xiong 25 33 0 07 Sep 2020
Attention Flows: Analyzing and Comparing Attention Mechanisms in Language Models Joseph F DeRose Jiayao Wang M. Berger 17 83 0 03 Sep 2020
Entity and Evidence Guided Relation Extraction for DocRED Kevin Huang Guangtao Wang Tengyu Ma Jing Huang 25 9 0 27 Aug 2020
AMBERT: A Pre-trained Language Model with Multi-Grained Tokenization Xinsong Zhang Pengshuai Li Hang Li 18 51 0 27 Aug 2020
syrapropa at SemEval-2020 Task 11: BERT-based Models Design For Propagandistic Technique and Span Detection Jinfen Li Lu Xiao 9 8 0 24 Aug 2020
ConvBERT: Improving BERT with Span-based Dynamic Convolution Zihang Jiang Weihao Yu Daquan Zhou Yunpeng Chen Jiashi Feng Shuicheng Yan 45 157 0 06 Aug 2020
A Survey on Text Classification: From Shallow to Deep Learning Qian Li Hao Peng Jianxin Li Congyin Xia Renyu Yang Lichao Sun Philip S. Yu Lifang He VLM 28 329 0 02 Aug 2020
On Learning Universal Representations Across Languages Xiangpeng Wei Rongxiang Weng Yue Hu Luxi Xing Heng Yu Weihua Luo SSL VLM 38 85 0 31 Jul 2020
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 288 2,028 0 28 Jul 2020
Multi-task learning for natural language processing in the 2020s: where are we going? Joseph Worsham Jugal Kalita AIMat 24 76 0 22 Jul 2020
IITK-RSA at SemEval-2020 Task 5: Detecting Counterfactuals Anirudh Ojha Rohin Garg Shashank Gupta Ashutosh Modi 14 6 0 21 Jul 2020
newsSweeper at SemEval-2020 Task 11: Context-Aware Rich Feature Representations For Propaganda Classification Paramansh Singh Siraj Sandhu Subham Kumar Ashutosh Modi 6 9 0 21 Jul 2020
What's in a Name? Are BERT Named Entity Representations just as Good for any other Name? S. Balasubramanian Naman Jain G. Jindal Abhijeet Awasthi Sunita Sarawagi OOD 31 25 0 14 Jul 2020
LMVE at SemEval-2020 Task 4: Commonsense Validation and Explanation using Pretraining Language Model Shilei Liu Yu Guo Bochao Li Feiliang Ren LRM 31 4 0 06 Jul 2020
EmotionGIF-Yankee: A Sentiment Classifier with Robust Model Based Ensemble Methods Wei-Yao Wang Kai-Shiang Chang Yu-Chien Tang 11 4 0 05 Jul 2020
A Survey on Self-supervised Pre-training for Sequential Transfer Learning in Neural Networks H. H. Mao BDL SSL 14 50 0 01 Jul 2020
Iterative Paraphrastic Augmentation with Discriminative Span Alignment Ryan Culkin J. E. Hu Elias Stengel-Eskin Guanghui Qin Benjamin Van Durme 30 6 0 01 Jul 2020
Exploring Software Naturalness through Neural Language Models Luca Buratti Saurabh Pujar Mihaela A. Bornea Scott McCarley Yunhui Zheng ... Alessandro Morari Jim Laredo Veronika Thost Yufan Zhuang Giacomo Domeniconi 25 97 0 22 Jun 2020
New Vietnamese Corpus for Machine Reading Comprehension of Health News Articles Kiet Van Nguyen Tin Van Huynh Duc-Vu Nguyen A. Nguyen Ngan Luu-Thuy Nguyen 24 40 0 19 Jun 2020
To Pretrain or Not to Pretrain: Examining the Benefits of Pretraining on Resource Rich Tasks Sinong Wang Madian Khabsa Hao Ma 18 26 0 15 Jun 2020
Self-supervised Learning: Generative or Contrastive Xiao Liu Fanjin Zhang Zhenyu Hou Zhaoyu Wang Li Mian Jing Zhang Jie Tang SSL 54 1,588 0 15 Jun 2020
Mathematical Reasoning via Self-supervised Skip-tree Training M. Rabe Dennis Lee Kshitij Bansal Christian Szegedy ReLM LRM 16 2 0 08 Jun 2020
A Cross-Task Analysis of Text Span Representations Shubham Toshniwal Freda Shi Bowen Shi Lingyu Gao Karen Livescu Kevin Gimpel 25 36 0 06 Jun 2020
DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations John Giorgi Osvald Nitski Bo Wang Gary D. Bader SSL 39 490 0 05 Jun 2020