SpanBERT: Improving Pre-training by Representing and Predicting Spans

24 July 2019

Luke Zettlemoyer

Papers citing "SpanBERT: Improving Pre-training by Representing and Predicting Spans"

50 / 950 papers shown

Title
Self-Supervised Test-Time Learning for Reading Comprehension Pratyay Banerjee Tejas Gokhale Chitta Baral SSL 17 28 0 20 Mar 2021
GLM: General Language Model Pretraining with Autoregressive Blank Infilling Zhengxiao Du Yujie Qian Xiao Liu Ming Ding J. Qiu Zhilin Yang Jie Tang BDL AI4CE 53 1,496 0 18 Mar 2021
Structure Inducing Pre-Training Matthew B. A. McDermott Brendan Yap Peter Szolovits Marinka Zitnik 42 18 0 18 Mar 2021
Cooperative Self-training of Machine Reading Comprehension Hongyin Luo Shang-Wen Li Ming Gao Seunghak Yu James R. Glass SyDa RALM 20 11 0 12 Mar 2021
FairFil: Contrastive Neural Debiasing Method for Pretrained Text Encoders Pengyu Cheng Weituo Hao Siyang Yuan Shijing Si Lawrence Carin 33 100 0 11 Mar 2021
Select, Substitute, Search: A New Benchmark for Knowledge-Augmented Visual Question Answering Aman Jain Mayank Kothyari Vishwajeet Kumar Preethi Jyothi Ganesh Ramakrishnan Soumen Chakrabarti 27 34 0 09 Mar 2021
OAG-BERT: Towards A Unified Backbone Language Model For Academic Knowledge Services Xiao Liu Da Yin Jingnan Zheng Xingjian Zhang Peng Zhang Hongxia Yang Yuxiao Dong Jie Tang VLM 45 31 0 03 Mar 2021
NLRG at SemEval-2021 Task 5: Toxic Spans Detection Leveraging BERT-based Token Classification and Span Prediction Techniques Gunjan Chhablani Abheesht Sharma Harshit Pandey Yash Bhartia S. Suthaharan 11 14 0 24 Feb 2021
Using Prior Knowledge to Guide BERT's Attention in Semantic Textual Matching Tasks Tingyu Xia Yue Wang Yuan Tian Yi-Ju Chang 30 51 0 22 Feb 2021
WebRED: Effective Pretraining And Finetuning For Relation Extraction On The Web Róbert Ormándi Mohammad Saleh Erin Winter Vinay Rao 20 11 0 18 Feb 2021
MUDES: Multilingual Detection of Offensive Spans Tharindu Ranasinghe Marcos Zampieri 27 41 0 18 Feb 2021
Less is More: Pre-train a Strong Text Encoder for Dense Retrieval Using a Weak Decoder Shuqi Lu Di He Chenyan Xiong Guolin Ke Waleed Malik Zhicheng Dou Paul N. Bennett Tie-Yan Liu Arnold Overwijk RALM 43 11 0 18 Feb 2021
COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining Yu Meng Chenyan Xiong Payal Bajaj Saurabh Tiwary Paul N. Bennett Jiawei Han Xia Song 127 203 0 16 Feb 2021
Boosting Low-Resource Biomedical QA via Entity-Aware Masking Strategies Gabriele Pergola E. Kochkina Lin Gui Maria Liakata Yulan He 88 31 0 16 Feb 2021
DOBF: A Deobfuscation Pre-Training Objective for Programming Languages Baptiste Roziere Marie-Anne Lachaux Marc Szafraniec Guillaume Lample AI4CE 52 138 0 15 Feb 2021
Decontextualization: Making Sentences Stand-Alone Eunsol Choi J. Palomaki Matthew Lamm Tom Kwiatkowski Dipanjan Das Michael Collins 9 95 0 09 Feb 2021
CSS-LM: A Contrastive Framework for Semi-supervised Fine-tuning of Pre-trained Language Models Yusheng Su Xu Han Yankai Lin Zhengyan Zhang Zhiyuan Liu Peng Li Jie Zhou Maosong Sun 19 10 0 07 Feb 2021
Do Question Answering Modeling Improvements Hold Across Benchmarks? Nelson F. Liu Tony Lee Robin Jia Percy Liang 28 13 0 01 Feb 2021
CD2CR: Co-reference Resolution Across Documents and Domains James Ravenscroft Arie Cattan A. Clare Ido Dagan Maria Liakata 90 8 0 29 Jan 2021
LOME: Large Ontology Multilingual Extraction Patrick Xia Guanghui Qin Siddharth Vashishtha Yunmo Chen Tongfei Chen Chandler May Craig Harman Kyle Rawlins A. White Benjamin Van Durme 21 40 0 28 Jan 2021
Modeling Context in Answer Sentence Selection Systems on a Latency Budget Rujun Han Luca Soldaini Alessandro Moschitti 32 14 0 28 Jan 2021
BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data Demetres Kostas Stephane Aroca-Ouellette Frank Rudzicz SSL 56 203 0 28 Jan 2021
KoreALBERT: Pretraining a Lite BERT Model for Korean Language Understanding HyunJae Lee Jaewoong Yoon Bonggyu Hwang Seongho Joe Seungjai Min Youngjune Gwon SSeg 31 16 0 27 Jan 2021
Joint Coreference Resolution and Character Linking for Multiparty Conversation Jiaxin Bai Hongming Zhang Yangqiu Song Kun Xu 54 7 0 27 Jan 2021
WangchanBERTa: Pretraining transformer-based Thai Language Models Lalita Lowphansirikul Charin Polpanumas Nawat Jantrakulchai Sarana Nutanong 13 74 0 24 Jan 2021
LIME: Learning Inductive Bias for Primitives of Mathematical Reasoning Yuhuai Wu M. Rabe Wenda Li Jimmy Ba Roger C. Grosse Christian Szegedy AIMat LRM 82 53 0 15 Jan 2021
Structured Prediction as Translation between Augmented Natural Languages Giovanni Paolini Ben Athiwaratkun Jason Krone Jie Ma Alessandro Achille Rishita Anubhai Cicero Nogueira dos Santos Bing Xiang Stefano Soatto 27 285 0 14 Jan 2021
BERT & Family Eat Word Salad: Experiments with Text Understanding Ashim Gupta Giorgi Kvernadze Vivek Srikumar 211 73 0 10 Jan 2021
SF-QA: Simple and Fair Evaluation Library for Open-domain Question Answering Xiaopeng Lu Kyusong Lee Tiancheng Zhao 21 0 0 06 Jan 2021
EfficientQA : a RoBERTa Based Phrase-Indexed Question-Answering System Sofian Chaybouti Achraf Saghe A. Shabou RALM 45 8 0 06 Jan 2021
Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering Fengbin Zhu Wenqiang Lei Chao Wang Jianming Zheng Soujanya Poria Tat-Seng Chua RALM 225 253 0 04 Jan 2021
Few-Shot Question Answering by Pretraining Span Selection Ori Ram Yuval Kirstain Jonathan Berant Amir Globerson Omer Levy 36 97 0 02 Jan 2021
Coreference Resolution without Span Representations Yuval Kirstain Ori Ram Omer Levy 32 72 0 02 Jan 2021
Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting Wangchunshu Zhou Tao Ge Canwen Xu Ke Xu Furu Wei LRM 16 15 0 02 Jan 2021
BanglaBERT: Language Model Pretraining and Benchmarks for Low-Resource Language Understanding Evaluation in Bangla Abhik Bhattacharjee Tahmid Hasan Wasi Uddin Ahmad Kazi Samin Mubasshir Md. Saiful Islam Anindya Iqbal M. Rahman Rifat Shahriyar SSL VLM 33 166 0 01 Jan 2021
Studying Strategically: Learning to Mask for Closed-book QA Qinyuan Ye Belinda Z. Li Sinong Wang Benjamin Bolte Hao Ma Wen-tau Yih Xiang Ren Madian Khabsa OffRL 27 11 0 31 Dec 2020
MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers Wenhui Wang Hangbo Bao Shaohan Huang Li Dong Furu Wei MQ 30 257 0 31 Dec 2020
CoCoLM: COmplex COmmonsense Enhanced Language Model with Discourse Relations Changlong Yu Hongming Zhang Yangqiu Song Wilfred Ng 80 21 0 31 Dec 2020
UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learning Wei Li Can Gao Guocheng Niu Xinyan Xiao Hao Liu Jiachen Liu Hua Wu Haifeng Wang 31 375 0 31 Dec 2020
ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning Yujia Qin Yankai Lin Ryuichi Takanobu Zhiyuan Liu Peng Li Heng Ji Minlie Huang Maosong Sun Jie Zhou 60 125 0 30 Dec 2020
CMV-BERT: Contrastive multi-vocab pretraining of BERT Wei-wei Zhu Daniel Cheung SSL VLM 19 0 0 29 Dec 2020
BURT: BERT-inspired Universal Representation from Learning Meaningful Segment Yian Li Hai Zhao SSL 21 0 0 28 Dec 2020
An Embarrassingly Simple Model for Dialogue Relation Extraction Fuzhao Xue Aixin Sun Hao Zhang Jinjie Ni Eng Siong Chng 27 27 0 27 Dec 2020
Bridging Textual and Tabular Data for Cross-Domain Text-to-SQL Semantic Parsing Xi Lin R. Socher Caiming Xiong LMTD 25 207 0 23 Dec 2020
Learning Dense Representations of Phrases at Scale Jinhyuk Lee Mujeen Sung Jaewoo Kang Danqi Chen RALM DML NAI 27 117 0 23 Dec 2020
A Survey on Visual Transformer Kai Han Yunhe Wang Hanting Chen Xinghao Chen Jianyuan Guo ... Chunjing Xu Yixing Xu Zhaohui Yang Yiman Zhang Dacheng Tao ViT 23 2,135 0 23 Dec 2020
Motif-Driven Contrastive Learning of Graph Representations Shichang Zhang Ziniu Hu Arjun Subramonian Yizhou Sun SSL 30 8 0 23 Dec 2020
Multi-Head Self-Attention with Role-Guided Masks Dongsheng Wang Casper Hansen Lucas Chaves Lima Christian B. Hansen Maria Maistro J. Simonsen Christina Lioma 26 1 0 22 Dec 2020
Recognizing Emotion Cause in Conversations Soujanya Poria Navonil Majumder Devamanyu Hazarika Deepanway Ghosal Rishabh Bhardwaj ... Romila Ghosh Abhinaba Roy Niyati Chhaya Alexander Gelbukh Rada Mihalcea 54 124 0 22 Dec 2020
Narrative Incoherence Detection Deng Cai Yizhe Zhang Yichen Huang Wai Lam Bill Dolan 19 5 0 21 Dec 2020