SpanBERT: Improving Pre-training by Representing and Predicting Spans

24 July 2019

Luke Zettlemoyer

Papers citing "SpanBERT: Improving Pre-training by Representing and Predicting Spans"

50 / 950 papers shown

Title
Blank Language Models T. Shen Victor Quach Regina Barzilay Tommi Jaakkola 203 73 0 08 Feb 2020
ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation Dongling Xiao Han Zhang Yukun Li Yu Sun Hao Tian Hua Wu Haifeng Wang 31 126 0 26 Jan 2020
RobBERT: a Dutch RoBERTa-based Language Model Pieter Delobelle Thomas Winters Bettina Berendt 18 233 0 17 Jan 2020
ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training Weizhen Qi Yu Yan Yeyun Gong Dayiheng Liu Nan Duan Jiusheng Chen Ruofei Zhang Ming Zhou AI4TS 27 446 0 13 Jan 2020
PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization Jingqing Zhang Yao-Min Zhao Mohammad Saleh Peter J. Liu RALM 3DGS 72 2,019 0 18 Dec 2019
Cross-Lingual Ability of Multilingual BERT: An Empirical Study Karthikeyan K Zihan Wang Stephen D. Mayhew Dan Roth LRM 36 333 0 17 Dec 2019
WaLDORf: Wasteless Language-model Distillation On Reading-comprehension J. Tian A. Kreuzer Pai-Hung Chen Hans-Martin Will VLM 42 3 0 13 Dec 2019
Personalized Patent Claim Generation and Measurement Jieh-Sheng Lee 16 4 0 07 Dec 2019
12-in-1: Multi-Task Vision and Language Representation Learning Jiasen Lu Vedanuj Goswami Marcus Rohrbach Devi Parikh Stefan Lee VLM ObjD 40 476 0 05 Dec 2019
Deep Contextualized Acoustic Representations For Semi-Supervised Speech Recognition Shaoshi Ling Yuzong Liu Julian Salazar Katrin Kirchhoff SSL 24 139 0 03 Dec 2019
A Transformer-based approach to Irony and Sarcasm detection Rolandos Alexandros Potamias Georgios Siolas A. Stafylopatis 33 206 0 23 Nov 2019
A Cluster Ranking Model for Full Anaphora Resolution Juntao Yu Alexandra Uma Massimo Poesio 22 18 0 21 Nov 2019
What do you mean, BERT? Assessing BERT as a Distributional Semantics Model Timothee Mickus Denis Paperno Mathieu Constant Kees van Deemter 29 45 0 13 Nov 2019
Attending to Entities for Better Text Understanding Pengxiang Cheng K. Erk LRM 24 37 0 11 Nov 2019
Improving BERT Fine-tuning with Embedding Normalization Wenxuan Zhou Junyi Du Xiang Ren 21 6 0 10 Nov 2019
CamemBERT: a Tasty French Language Model Louis Martin Benjamin Muller Pedro Ortiz Suarez Yoann Dupont Laurent Romary Eric Villemonte de la Clergerie Djamé Seddah Benoît Sagot 42 956 0 10 Nov 2019
INSET: Sentence Infilling with INter-SEntential Transformer Yichen Huang Yizhe Zhang Oussama Elachqar Yu Cheng 30 1 0 10 Nov 2019
Generalizing Natural Language Analysis through Span-relation Representations Zhengbao Jiang Wenyuan Xu Jun Araki Graham Neubig 33 60 0 10 Nov 2019
Improving Machine Reading Comprehension via Adversarial Training Ziqing Yang Yiming Cui Wanxiang Che Ting Liu Shijin Wang Guoping Hu 27 17 0 09 Nov 2019
SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization Haoming Jiang Pengcheng He Weizhu Chen Xiaodong Liu Jianfeng Gao T. Zhao 40 559 0 08 Nov 2019
Blockwise Self-Attention for Long Document Understanding J. Qiu Hao Ma Omer Levy Scott Yih Sinong Wang Jie Tang 11 252 0 07 Nov 2019
SentiLARE: Sentiment-Aware Language Representation Learning with Linguistic Knowledge Pei Ke Haozhe Ji Siyang Liu Xiaoyan Zhu Minlie Huang 25 7 0 06 Nov 2019
Coreference Resolution as Query-based Span Prediction Wei Wu Fei Wang Arianna Yuan Fei Wu Jiwei Li LRM 36 180 0 05 Nov 2019
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension M. Lewis Yinhan Liu Naman Goyal Marjan Ghazvininejad Abdel-rahman Mohamed Omer Levy Veselin Stoyanov Luke Zettlemoyer AIMat VLM 41 10,635 0 29 Oct 2019
SpeechBERT: An Audio-and-text Jointly Learned Language Model for End-to-end Spoken Question Answering Yung-Sung Chuang Chi-Liang Liu Hung-yi Lee Lin-shan Lee AuLLM 30 39 0 25 Oct 2019
An Empirical Study of Efficient ASR Rescoring with Transformers Hongzhao Huang Fuchun Peng KELM 19 22 0 24 Oct 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 141 19,578 0 23 Oct 2019
A Neural Entity Coreference Resolution Review Nikolaos Stylianou I. Vlahavas 24 38 0 21 Oct 2019
A Mutual Information Maximization Perspective of Language Representation Learning Lingpeng Kong Cyprien de Masson dÁutume Wang Ling Lei Yu Zihang Dai Dani Yogatama SSL 226 166 0 18 Oct 2019
vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations Alexei Baevski Steffen Schneider Michael Auli SSL 28 661 0 12 Oct 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 115 6,380 0 26 Sep 2019
Cross-Lingual Natural Language Generation via Pre-Training Zewen Chi Li Dong Furu Wei Wenhui Wang Xian-Ling Mao Heyan Huang 27 136 0 23 Sep 2019
TinyBERT: Distilling BERT for Natural Language Understanding Xiaoqi Jiao Yichun Yin Lifeng Shang Xin Jiang Xiao Chen Linlin Li F. Wang Qun Liu VLM 11 1,823 0 23 Sep 2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 245 1,836 0 17 Sep 2019
Span-based Joint Entity and Relation Extraction with Transformer Pre-training Markus Eberts A. Ulges LRM ViT 164 381 0 17 Sep 2019
K-BERT: Enabling Language Representation with Knowledge Graph Weijie Liu Peng Zhou Zhe Zhao Zhiruo Wang Qi Ju Haotang Deng Ping Wang 231 778 0 17 Sep 2019
Span Selection Pre-training for Question Answering Michael R. Glass A. Gliozzo Rishav Chakravarti Anthony Ferritto Lin Pan G P Shrivatsa Bhargav Dinesh Garg Avirup Sil RALM 38 70 0 09 Sep 2019
Pretrained AI Models: Performativity, Mobility, and Change Lav Varshney N. Keskar R. Socher 18 20 0 07 Sep 2019
Effective Use of Transformer Networks for Entity Tracking Aditya Gupta Greg Durrett 30 20 0 05 Sep 2019
Semantics-aware BERT for Language Understanding ZhuoSheng Zhang Yuwei Wu Zhao Hai Z. Li Shuailiang Zhang Xi Zhou Xiang Zhou 21 365 0 05 Sep 2019
Ellipsis Resolution as Question Answering: An Evaluation Rahul Aralikatte Matthew Lamm D. Hardt Anders Søgaard 17 3 0 29 Aug 2019
Partially-supervised Mention Detection Lesly Miculicich James Henderson 21 6 0 26 Aug 2019
BERT for Coreference Resolution: Baselines and Analysis Mandar Joshi Omer Levy Daniel S. Weld Luke Zettlemoyer 28 320 0 24 Aug 2019
StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding Wei Wang Bin Bi Ming Yan Chen Henry Wu Zuyi Bao Jiangnan Xia Liwei Peng Luo Si 31 260 0 13 Aug 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 180 23,959 0 26 Jul 2019
BERTphone: Phonetically-Aware Encoder Representations for Utterance-Level Speaker and Language Recognition Shaoshi Ling Julian Salazar Yuzong Liu Katrin Kirchhoff SSL 30 28 0 30 Jun 2019
Pre-Training with Whole Word Masking for Chinese BERT Yiming Cui Wanxiang Che Ting Liu Bing Qin Ziqing Yang 35 179 0 19 Jun 2019
Better Long-Range Dependency By Bootstrapping A Mutual Information Regularizer Yanshuai Cao Peng Xu 17 2 0 28 May 2019
Dual Co-Matching Network for Multi-choice Reading Comprehension Shuailiang Zhang Zhao Hai Yuwei Wu ZhuoSheng Zhang Xi Zhou Xiaoping Zhou 39 131 0 27 Jan 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 304 7,005 0 20 Apr 2018