SpanBERT: Improving Pre-training by Representing and Predicting Spans

24 July 2019

Luke Zettlemoyer

Papers citing "SpanBERT: Improving Pre-training by Representing and Predicting Spans"

50 / 950 papers shown

Title
PQuAD: A Persian Question Answering Dataset Kasra Darvishi Newsha Shahbodagh Zahra Abbasiantaeb S. Momtazi 12 17 0 13 Feb 2022
mSLAM: Massively multilingual joint pre-training for speech and text Ankur Bapna Colin Cherry Yu Zhang Ye Jia Melvin Johnson Yong Cheng Simran Khanuja Jason Riesa Alexis Conneau VLM 30 111 0 03 Feb 2022
Reasoning Like Program Executors Xinyu Pi Qian Liu Bei Chen Morteza Ziyadi Zeqi Lin Qiang Fu Yan Gao Jian-Guang Lou Weizhu Chen ReLM LRM 253 52 0 27 Jan 2022
Linguistically-driven Multi-task Pre-training for Low-resource Neural Machine Translation Zhuoyuan Mao Chenhui Chu Sadao Kurohashi 17 6 0 20 Jan 2022
Transferability in Deep Learning: A Survey Junguang Jiang Yang Shu Jianmin Wang Mingsheng Long OOD 36 101 0 15 Jan 2022
Reasoning Through Memorization: Nearest Neighbor Knowledge Graph Embeddings Peng Wang Xin Xie Xiaohan Wang Ningyu Zhang RALM 37 16 0 14 Jan 2022
Progressively Optimized Bi-Granular Document Representation for Scalable Embedding Based Retrieval Shitao Xiao Zheng Liu Weihao Han Jianjin Zhang Yingxia Shao ... Hao Sun Denvy Deng Liangjie Zhang Qi Zhang Xing Xie 35 17 0 14 Jan 2022
Making a (Counterfactual) Difference One Rationale at a Time Michael J. Plyler Michal Green Min Chi 26 11 0 13 Jan 2022
Grow-and-Clip: Informative-yet-Concise Evidence Distillation for Answer Explanation Yuyan Chen Yanghua Xiao Bang Liu 17 16 0 13 Jan 2022
Multi-task Pre-training Language Model for Semantic Network Completion Da Li Sen Yang Kele Xu Ming Yi Yukai He Huaimin Wang 48 30 0 13 Jan 2022
MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound Rowan Zellers Jiasen Lu Ximing Lu Youngjae Yu Yanpeng Zhao Mohammadreza Salehi Aditya Kusupati Jack Hessel Ali Farhadi Yejin Choi 48 207 0 07 Jan 2022
Fortunately, Discourse Markers Can Enhance Language Models for Sentiment Analysis L. Ein-Dor Ilya Shnayderman Artem Spector Lena Dankin R. Aharonov Noam Slonim 41 8 0 06 Jan 2022
Budget Sensitive Reannotation of Noisy Relation Classification Data Using Label Hierarchy Akshay Parekh Ashish Anand Amit Awekar 13 0 0 26 Dec 2021
ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation Shuohuan Wang Yu Sun Yang Xiang Zhihua Wu Siyu Ding ... Tian Wu Wei Zeng Ge Li Wen Gao Haifeng Wang ELM 39 79 0 23 Dec 2021
Word Graph Guided Summarization for Radiology Findings Jinpeng Hu Jianling Li Zhihong Chen Yaling Shen Yan Song Xiang Wan Tsung-Hui Chang 17 37 0 18 Dec 2021
Learning Rich Representation of Keyphrases from Text Mayank Kulkarni Debanjan Mahata Ravneet Arora Rajarshi Bhowmik VLM 32 65 0 16 Dec 2021
DocAMR: Multi-Sentence AMR Representation and Evaluation Tahira Naseem Austin Blodgett Yara Rizk Timothy J. O'Gorman Young-Suk Lee Jeffrey Flanigan Ramón Fernández Astudillo Radu Florian Salim Roukos Nathan Schneider 40 16 0 15 Dec 2021
LongT5: Efficient Text-To-Text Transformer for Long Sequences Mandy Guo Joshua Ainslie David C. Uthus Santiago Ontanon Jianmo Ni Yun-hsuan Sung Yinfei Yang VLM 31 306 0 15 Dec 2021
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena Letitia Parcalabescu Michele Cafagna Lilitta Muradjan Anette Frank Iacer Calixto Albert Gatt CoGe 34 110 0 14 Dec 2021
TopNet: Learning from Neural Topic Model to Generate Long Stories Yazheng Yang Boyuan Pan Deng Cai Huan Sun 18 9 0 14 Dec 2021
WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models Benjamin Minixhofer Fabian Paischer Navid Rekabsaz 29 74 0 13 Dec 2021
General Facial Representation Learning in a Visual-Linguistic Manner Yinglin Zheng Hao Yang Ting Zhang Jianmin Bao Dongdong Chen Yangyu Huang Lu Yuan Dong Chen Ming Zeng Fang Wen CVBM 146 164 0 06 Dec 2021
MetaQA: Combining Expert Agents for Multi-Skill Question Answering Haritz Puerto Gözde Gül Sahin Iryna Gurevych LLMAG 40 20 0 03 Dec 2021
DKPLM: Decomposable Knowledge-enhanced Pre-trained Language Model for Natural Language Understanding Taolin Zhang Chengyu Wang Nan Hu Minghui Qiu Chengguang Tang Xiaofeng He Jun Huang KELM VLM 27 30 0 02 Dec 2021
Domain-oriented Language Pre-training with Adaptive Hybrid Masking and Optimal Transport Alignment Denghui Zhang Zixuan Yuan Yanchi Liu Hao Liu Fuzhen Zhuang Hui Xiong Haifeng Chen VLM 18 3 0 01 Dec 2021
Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling Xumin Yu Lulu Tang Yongming Rao Tiejun Huang Jie Zhou Jiwen Lu 3DPC 53 655 0 29 Nov 2021
An analysis of document graph construction methods for AMR summarization Fei-Tzin Lee Chris Kedzie Nakul Verma Kathleen McKeown 17 8 0 27 Nov 2021
PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformers Xiaoyi Dong Jianmin Bao Ting Zhang Dongdong Chen Weiming Zhang Lu Yuan Dong Chen Fang Wen Nenghai Yu Baining Guo ViT 52 239 0 24 Nov 2021
ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning V. Aribandi Yi Tay Tal Schuster J. Rao H. Zheng ... Jianmo Ni Jai Gupta Kai Hui Sebastian Ruder Donald Metzler MoE 34 214 0 22 Nov 2021
LAnoBERT: System Log Anomaly Detection based on BERT Masked Language Model Yukyung Lee Jina Kim Pilsung Kang 17 79 0 18 Nov 2021
Document AI: Benchmarks, Models and Applications Lei Cui Yiheng Xu Tengchao Lv Furu Wei VLM 29 70 0 16 Nov 2021
A Survey on Green Deep Learning Jingjing Xu Wangchunshu Zhou Zhiyi Fu Hao Zhou Lei Li VLM 86 83 0 08 Nov 2021
Machine-in-the-Loop Rewriting for Creative Image Captioning Vishakh Padmakumar He He 64 17 0 07 Nov 2021
Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey Bonan Min Hayley L Ross Elior Sulem Amir Pouran Ben Veyseh Thien Huu Nguyen Oscar Sainz Eneko Agirre Ilana Heinz Dan Roth LM&MA VLM AI4CE 88 1,039 0 01 Nov 2021
Alignment Attention by Matching Key and Query Distributions Shujian Zhang Xinjie Fan Huangjie Zheng Korawat Tanwisuth Mingyuan Zhou OOD 40 10 0 25 Oct 2021
Interpreting Deep Learning Models in Natural Language Processing: A Review Xiaofei Sun Diyi Yang Xiaoya Li Tianwei Zhang Yuxian Meng Han Qiu Guoyin Wang Eduard H. Hovy Jiwei Li 24 45 0 20 Oct 2021
SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Training Ankur Bapna Yu-An Chung Na Wu Anmol Gulati Ye Jia J. Clark Melvin Johnson Jason Riesa Alexis Conneau Yu Zhang VLM 64 94 0 20 Oct 2021
GNN-LM: Language Modeling based on Global Contexts via GNN Yuxian Meng Shi Zong Xiaoya Li Xiaofei Sun Tianwei Zhang Fei Wu Jiwei Li LRM 29 37 0 17 Oct 2021
On the Robustness of Reading Comprehension Models to Entity Renaming Jun Yan Yang Xiao Sagnik Mukherjee Bill Yuchen Lin Robin Jia Xiang Ren 26 20 0 16 Oct 2021
Metadata Shaping: Natural Language Annotations for the Tail Simran Arora Sen Wu Enci Liu Christopher Ré 38 0 0 16 Oct 2021
Tracing Origins: Coreference-aware Machine Reading Comprehension Baorong Huang Zhuosheng Zhang Hai Zhao 19 5 0 15 Oct 2021
Attacking Open-domain Question Answering by Injecting Misinformation Liangming Pan Wenhu Chen Min-Yen Kan Wenjie Wang HILM AAML 217 22 0 15 Oct 2021
Building Chinese Biomedical Language Models via Multi-Level Text Discrimination Quan Wang Songtai Dai Benfeng Xu Yajuan Lyu Yong Zhu Hua Wu Haifeng Wang 71 14 0 14 Oct 2021
Mengzi: Towards Lightweight yet Ingenious Pre-trained Models for Chinese Zhuosheng Zhang Hanqing Zhang Keming Chen Yuhang Guo Jingyun Hua Yulong Wang Ming Zhou VLM 59 71 0 13 Oct 2021
EventBERT: A Pre-Trained Model for Event Correlation Reasoning Yucheng Zhou Xiubo Geng Tao Shen Guodong Long Daxin Jiang 44 46 0 13 Oct 2021
Anatomy of OntoGUM--Adapting GUM to the OntoNotes Scheme to Evaluate Robustness of SOTA Coreference Algorithms Yilun Zhu Sameer Pradhan Amir Zeldes 32 5 0 12 Oct 2021
Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting Zahra Fatemi Chen Xing Wenhao Liu Caiming Xiong CLL 35 33 0 11 Oct 2021
PoNet: Pooling Network for Efficient Token Mixing in Long Sequences Chao-Hong Tan Qian Chen Wen Wang Qinglin Zhang Siqi Zheng Zhenhua Ling ViT 22 11 0 06 Oct 2021
Is Attention always needed? A Case Study on Language Identification from Speech A. Mandal Santanu Pal Indranil Dutta Mahidas Bhattacharya S. Naskar 27 6 0 05 Oct 2021
SPaR.txt, a cheap Shallow Parsing approach for Regulatory texts Ruben Kruiper Ioannis Konstas A. Gray Farhad Sadeghineko Richard Watson B. Kumar 32 6 0 04 Oct 2021