v1v2v3 (latest)

SpanBERT: Improving Pre-training by Representing and Predicting Spans

24 July 2019

Luke Zettlemoyer

Papers citing "SpanBERT: Improving Pre-training by Representing and Predicting Spans"

50 / 955 papers shown

Title
Self-supervised Contrastive Cross-Modality Representation Learning for Spoken Question Answering Chenyu You Nuo Chen Yuexian Zou SSL 102 64 0 08 Sep 2021
Generate & Rank: A Multi-task Framework for Math Word Problems Jianhao Shen Yichun Yin Lin Li Lifeng Shang Xin Jiang Ming Zhang Qun Liu AIMat 87 133 0 07 Sep 2021
Sent2Span: Span Detection for PICO Extraction in the Biomedical Text without Span Annotations Shifeng Liu Yifang Sun Bing Li Wei Wang Florence T. Bourgeois A. Dunn 52 14 0 06 Sep 2021
Data Efficient Masked Language Modeling for Vision and Language Yonatan Bitton Gabriel Stanovsky Michael Elhadad Roy Schwartz VLM 82 20 0 05 Sep 2021
FewshotQA: A simple framework for few-shot learning of question answering tasks using pre-trained text-to-text models Rakesh Chada P. Natarajan 89 46 0 04 Sep 2021
Frustratingly Simple Pretraining Alternatives to Masked Language Modeling Atsuki Yamaguchi G. Chrysostomou Katerina Margatina Nikolaos Aletras 69 25 0 04 Sep 2021
Adapted End-to-End Coreference Resolution System for Anaphoric Identities in Dialogues Liyan Xu Jinho Choi 47 10 0 01 Sep 2021
Robust Retrieval Augmented Generation for Zero-shot Slot Filling Michael R. Glass Gaetano Rossiello Md. Faisal Mahbub Chowdhury A. Gliozzo RALM 71 33 0 31 Aug 2021
T3-Vis: a visual analytic framework for Training and fine-Tuning Transformers in NLP Raymond Li Wen Xiao Lanjun Wang Hyeju Jang Giuseppe Carenini ViT 90 23 0 31 Aug 2021
Towards Consistent Document-level Entity Linking: Joint Models for Entity Linking and Coreference Resolution Klim Zaporojets Johannes Deleu Yiwei Jiang Thomas Demeester Chris Develder 137 10 0 30 Aug 2021
NEREL: A Russian Dataset with Nested Named Entities, Relations and Events Natalia Loukachevitch Ekaterina Artemova Tatiana Batura Pavel Braslavski Ilia Denisov V. Ivanov S. Manandhar Alexander Pugachev E. Tutubalina NAI 48 20 0 30 Aug 2021
Span Fine-tuning for Pre-trained Language Models Rongzhou Bao Zhuosheng Zhang Hai Zhao 46 2 0 29 Aug 2021
SMedBERT: A Knowledge-Enhanced Pre-trained Language Model with Structured Semantics for Medical Text Mining Taolin Zhang Zerui Cai Chengyu Wang Minghui Qiu Bite Yang Xiaofeng He AI4MH 73 54 0 20 Aug 2021
MvSR-NAT: Multi-view Subset Regularization for Non-Autoregressive Machine Translation Pan Xie Zexian Li Xiaohui Hu 44 11 0 19 Aug 2021
Zero-shot Task Transfer for Invoice Extraction via Class-aware QA Ensemble Prithiviraj Damodaran Prabhkaran Singh Josemon Achankuju 36 2 0 13 Aug 2021
AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processing Katikapalli Subramanyam Kalyan A. Rajasekharan S. Sangeetha VLM LM&MA 107 270 0 12 Aug 2021
BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documents Teakgyu Hong Donghyun Kim Mingi Ji Wonseok Hwang Daehyun Nam Sungrae Park VLM 124 154 0 10 Aug 2021
Musical Speech: A Transformer-based Composition Tool Jason dÉon Sri Harsha Dumpala Chandramouli Shama Sastry Daniel Oore Sageev Oore 57 1 0 02 Aug 2021
LICHEE: Improving Language Model Pre-training with Multi-grained Tokenization Weidong Guo Mingjun Zhao Lusheng Zhang Di Niu Jinwen Luo Zhenhua Liu Zhenyang Li J. Tang 55 8 0 02 Aug 2021
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing Pengfei Liu Weizhe Yuan Jinlan Fu Zhengbao Jiang Hiroaki Hayashi Graham Neubig VLM SyDa 283 4,044 0 28 Jul 2021
Multilingual Coreference Resolution with Harmonized Annotations O. Pražák Miloslav Konopík Jakub Sido 44 16 0 26 Jul 2021
Context-aware Adversarial Training for Name Regularity Bias in Named Entity Recognition Abbas Ghaddar Philippe Langlais Ahmad Rashid Mehdi Rezagholizadeh 124 44 0 24 Jul 2021
Target-Oriented Fine-tuning for Zero-Resource Named Entity Recognition Ying Zhang Fandong Meng Jinan Xu Jinan Xu Jie Zhou 76 10 0 22 Jul 2021
Improving Sentence-Level Relation Extraction through Curriculum Learning Seongsik Park Harksoo Kim 60 14 0 20 Jul 2021
BERT-like Pre-training for Symbolic Piano Music Classification Tasks Yi-Hui Chou I-Chun Chen Chin-Jui Chang Joann Ching Yi-Hsuan Yang 96 25 0 12 Jul 2021
ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation Yu Sun Shuohuan Wang Shikun Feng Siyu Ding Chao Pang ... Ouyang Xuan Dianhai Yu Hao Tian Hua Wu Haifeng Wang 114 475 0 05 Jul 2021
End-to-end Neural Coreference Resolution Revisited: A Simple yet Effective Baseline T. Lai Trung Bui Doo Soon Kim 60 12 0 04 Jul 2021
CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understanding Dong Wang Ning Ding Pijian Li Haitao Zheng AAML 68 118 0 01 Jul 2021
XLM-E: Cross-lingual Language Model Pre-training via ELECTRA Zewen Chi Shaohan Huang Li Dong Shuming Ma Bo Zheng ... Payal Bajaj Xia Song Xian-Ling Mao Heyan Huang Furu Wei 111 121 0 30 Jun 2021
ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information Zijun Sun Xiaoya Li Xiaofei Sun Yuxian Meng Xiang Ao Qing He Leilei Gan Jiwei Li SSeg 144 190 0 30 Jun 2021
SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption Dara Bahri Heinrich Jiang Yi Tay Donald Metzler SSL 74 178 0 29 Jun 2021
Overview of BioASQ 2020: The eighth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering A. Nentidis Anastasia Krithara K. Bougiatiotis Martin Krallinger Carlos Rodríguez-Penagos Marta Villegas George Giannakopoulos 127 32 0 28 Jun 2021
WVOQ at SemEval-2021 Task 6: BART for Span Detection and Classification Cees Roele 54 3 0 27 Jun 2021
OKGIT: Open Knowledge Graph Link Prediction with Implicit Types Chandrahas Partha P. Talukdar 69 15 0 24 Jun 2021
LV-BERT: Exploiting Layer Variety for BERT Weihao Yu Zihang Jiang Fei Chen Qibin Hou Jiashi Feng MQ 49 0 0 22 Jun 2021
Towards Long-Form Video Understanding Chaoxia Wu Philipp Krahenbuhl VLM ViT 117 170 0 21 Jun 2021
VIMPAC: Video Pre-Training via Masked Token Prediction and Contrastive Learning Hao Tan Jie Lei Thomas Wolf Joey Tianyi Zhou 118 66 0 21 Jun 2021
Out of Context: A New Clue for Context Modeling of Aspect-based Sentiment Analysis Bowen Xing Ivor W. Tsang 46 11 0 21 Jun 2021
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models Elad Ben-Zaken Shauli Ravfogel Yoav Goldberg 246 1,245 0 18 Jun 2021
Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning Colin Wei Sang Michael Xie Tengyu Ma 148 100 0 17 Jun 2021
Coreference-Aware Dialogue Summarization Zhengyuan Liu Ke Shi Nancy F. Chen 82 60 0 16 Jun 2021
BEiT: BERT Pre-Training of Image Transformers Hangbo Bao Li Dong Songhao Piao Furu Wei ViT 343 2,853 0 15 Jun 2021
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units Wei-Ning Hsu Benjamin Bolte Yao-Hung Hubert Tsai Kushal Lakhotia Ruslan Salakhutdinov Abdel-rahman Mohamed SSL 190 3,013 0 14 Jun 2021
SAS: Self-Augmentation Strategy for Language Model Pre-training Yifei Xu Jingqiao Zhang Ru He Liangzhu Ge Chao Yang Cheng Yang Ying Wu 59 1 0 14 Jun 2021
Pre-Trained Models: Past, Present and Future Xu Han Zhengyan Zhang Ning Ding Yuxian Gu Xiao Liu ... Jie Tang Ji-Rong Wen Jinhui Yuan Wayne Xin Zhao Jun Zhu AIFin MQ AI4MH 174 863 0 14 Jun 2021
Can Transformer Language Models Predict Psychometric Properties? Antonio Laverghetta Animesh Nighojkar Jamshidbek Mirzakhalov John Licato LM&MA 71 14 0 12 Jun 2021
Hybrid Generative-Contrastive Representation Learning Saehoon Kim Sungwoong Kim Juho Lee SSL 68 11 0 11 Jun 2021
CAT: Cross Attention in Vision Transformer Hezheng Lin Xingyi Cheng Xiangyu Wu Fan Yang Dong Shen Zhongyuan Wang Qing Song Wei Yuan ViT 65 157 0 10 Jun 2021
Linguistically Informed Masking for Representation Learning in the Patent Domain Sophia Althammer Mark Buckley Sebastian Hofstatter Allan Hanbury 59 11 0 10 Jun 2021
MusicBERT: Symbolic Music Understanding with Large-Scale Pre-Training Mingliang Zeng Xu Tan Rui Wang Zeqian Ju Tao Qin Tie-Yan Liu 65 136 0 10 Jun 2021