v1v2v3 (latest)

SpanBERT: Improving Pre-training by Representing and Predicting Spans

24 July 2019

Luke Zettlemoyer

Papers citing "SpanBERT: Improving Pre-training by Representing and Predicting Spans"

50 / 955 papers shown

Title
TRANS-BLSTM: Transformer with Bidirectional LSTM for Language Understanding Zhiheng Huang Peng Xu Davis Liang Ajay K. Mishra Bing Xiang 40 31 0 16 Mar 2020
UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training Hangbo Bao Li Dong Furu Wei Wenhui Wang Nan Yang ... Yu Wang Songhao Piao Jianfeng Gao Ming Zhou H. Hon AI4CE 88 397 0 28 Feb 2020
A Primer in BERTology: What we know about how BERT works Anna Rogers Olga Kovaleva Anna Rumshisky OffRL 143 1,511 0 27 Feb 2020
MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers Wenhui Wang Furu Wei Li Dong Hangbo Bao Nan Yang Ming Zhou VLM 241 1,285 0 25 Feb 2020
Training Question Answering Models From Synthetic Data Raul Puri Ryan Spring M. Patwary Mohammad Shoeybi Bryan Catanzaro ELM 81 160 0 22 Feb 2020
REALM: Retrieval-Augmented Language Model Pre-Training Kelvin Guu Kenton Lee Zora Tung Panupong Pasupat Ming-Wei Chang RALM 154 2,128 0 10 Feb 2020
Blank Language Models T. Shen Victor Quach Regina Barzilay Tommi Jaakkola 288 73 0 08 Feb 2020
ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation Dongling Xiao Han Zhang Yukun Li Yu Sun Hao Tian Hua Wu Haifeng Wang 85 127 0 26 Jan 2020
RobBERT: a Dutch RoBERTa-based Language Model Pieter Delobelle Thomas Winters Bettina Berendt 86 240 0 17 Jan 2020
ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training Weizhen Qi Yu Yan Yeyun Gong Dayiheng Liu Nan Duan Jiusheng Chen Ruofei Zhang Ming Zhou AI4TS 140 450 0 13 Jan 2020
PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization Jingqing Zhang Yao-Min Zhao Mohammad Saleh Peter J. Liu RALM 3DGS 307 2,057 0 18 Dec 2019
Cross-Lingual Ability of Multilingual BERT: An Empirical Study Karthikeyan K Zihan Wang Stephen D. Mayhew Dan Roth LRM 96 340 0 17 Dec 2019
WaLDORf: Wasteless Language-model Distillation On Reading-comprehension J. Tian A. Kreuzer Pai-Hung Chen Hans-Martin Will VLM 60 3 0 13 Dec 2019
Personalized Patent Claim Generation and Measurement Jieh-Sheng Lee 22 4 0 07 Dec 2019
12-in-1: Multi-Task Vision and Language Representation Learning Jiasen Lu Vedanuj Goswami Marcus Rohrbach Devi Parikh Stefan Lee VLM ObjD 131 481 0 05 Dec 2019
Deep Contextualized Acoustic Representations For Semi-Supervised Speech Recognition Shaoshi Ling Yuzong Liu Julian Salazar Katrin Kirchhoff SSL 86 139 0 03 Dec 2019
A Transformer-based approach to Irony and Sarcasm detection Rolandos Alexandros Potamias Georgios Siolas A. Stafylopatis 47 213 0 23 Nov 2019
A Cluster Ranking Model for Full Anaphora Resolution Juntao Yu Alexandra Uma Massimo Poesio 55 18 0 21 Nov 2019
What do you mean, BERT? Assessing BERT as a Distributional Semantics Model Timothee Mickus Denis Paperno Mathieu Constant Kees van Deemter 86 46 0 13 Nov 2019
Attending to Entities for Better Text Understanding Pengxiang Cheng K. Erk LRM 59 38 0 11 Nov 2019
Improving BERT Fine-tuning with Embedding Normalization Wenxuan Zhou Junyi Du Xiang Ren 36 6 0 10 Nov 2019
CamemBERT: a Tasty French Language Model Louis Martin Benjamin Muller Pedro Ortiz Suarez Yoann Dupont Laurent Romary Eric Villemonte de la Clergerie Djamé Seddah Benoît Sagot 143 981 0 10 Nov 2019
INSET: Sentence Infilling with INter-SEntential Transformer Yichen Huang Yizhe Zhang Oussama Elachqar Yu Cheng 50 1 0 10 Nov 2019
Generalizing Natural Language Analysis through Span-relation Representations Zhengbao Jiang Wenyuan Xu Jun Araki Graham Neubig 81 60 0 10 Nov 2019
Improving Machine Reading Comprehension via Adversarial Training Ziqing Yang Yiming Cui Wanxiang Che Ting Liu Shijin Wang Guoping Hu 59 17 0 09 Nov 2019
SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization Haoming Jiang Pengcheng He Weizhu Chen Xiaodong Liu Jianfeng Gao T. Zhao 137 563 0 08 Nov 2019
Blockwise Self-Attention for Long Document Understanding J. Qiu Hao Ma Omer Levy Scott Yih Sinong Wang Jie Tang 111 254 0 07 Nov 2019
SentiLARE: Sentiment-Aware Language Representation Learning with Linguistic Knowledge Pei Ke Haozhe Ji Siyang Liu Xiaoyan Zhu Minlie Huang 64 7 0 06 Nov 2019
Coreference Resolution as Query-based Span Prediction Wei Wu Fei Wang Arianna Yuan Leilei Gan Jiwei Li LRM 90 180 0 05 Nov 2019
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension M. Lewis Yinhan Liu Naman Goyal Marjan Ghazvininejad Abdel-rahman Mohamed Omer Levy Veselin Stoyanov Luke Zettlemoyer AIMat VLM 268 10,913 0 29 Oct 2019
SpeechBERT: An Audio-and-text Jointly Learned Language Model for End-to-end Spoken Question Answering Yung-Sung Chuang Chi-Liang Liu Hung-yi Lee Lin-shan Lee AuLLM 92 39 0 25 Oct 2019
An Empirical Study of Efficient ASR Rescoring with Transformers Hongzhao Huang Fuchun Peng KELM 35 22 0 24 Oct 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 659 20,418 0 23 Oct 2019
A Neural Entity Coreference Resolution Review Nikolaos Stylianou I. Vlahavas 104 40 0 21 Oct 2019
A Mutual Information Maximization Perspective of Language Representation Learning Lingpeng Kong Cyprien de Masson dÁutume Wang Ling Lei Yu Zihang Dai Dani Yogatama SSL 284 167 0 18 Oct 2019
vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations Alexei Baevski Steffen Schneider Michael Auli SSL 185 668 0 12 Oct 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 498 6,482 0 26 Sep 2019
Cross-Lingual Natural Language Generation via Pre-Training Zewen Chi Li Dong Furu Wei Wenhui Wang Xian-Ling Mao Heyan Huang 101 138 0 23 Sep 2019
TinyBERT: Distilling BERT for Natural Language Understanding Xiaoqi Jiao Yichun Yin Lifeng Shang Xin Jiang Xiao Chen Linlin Li F. Wang Qun Liu VLM 130 1,881 0 23 Sep 2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism Mohammad Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 363 1,926 0 17 Sep 2019
Span-based Joint Entity and Relation Extraction with Transformer Pre-training Markus Eberts A. Ulges LRM ViT 240 387 0 17 Sep 2019
K-BERT: Enabling Language Representation with Knowledge Graph Weijie Liu Peng Zhou Zhe Zhao Zhiruo Wang Qi Ju Haotang Deng Ping Wang 316 796 0 17 Sep 2019
Span Selection Pre-training for Question Answering Michael R. Glass A. Gliozzo Rishav Chakravarti Anthony Ferritto Lin Pan G P Shrivatsa Bhargav Dinesh Garg Avirup Sil RALM 97 73 0 09 Sep 2019
Pretrained AI Models: Performativity, Mobility, and Change Lav Varshney N. Keskar R. Socher 68 20 0 07 Sep 2019
Effective Use of Transformer Networks for Entity Tracking Aditya Gupta Greg Durrett 59 20 0 05 Sep 2019
Semantics-aware BERT for Language Understanding Zhuosheng Zhang Yuwei Wu Zhao Hai Z. Li Shuailiang Zhang Xi Zhou Xiang Zhou 67 370 0 05 Sep 2019
Ellipsis Resolution as Question Answering: An Evaluation Rahul Aralikatte Matthew Lamm D. Hardt Anders Søgaard 49 3 0 29 Aug 2019
Partially-supervised Mention Detection Lesly Miculicich James Henderson 46 7 0 26 Aug 2019
BERT for Coreference Resolution: Baselines and Analysis Mandar Joshi Omer Levy Daniel S. Weld Luke Zettlemoyer 114 322 0 24 Aug 2019
StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding Wei Wang Bin Bi Ming Yan Chen Henry Wu Zuyi Bao Jiangnan Xia Liwei Peng Luo Si 93 264 0 13 Aug 2019