RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 10,695 papers shown

Title
DUMA: Reading Comprehension with Transposition Thinking Pengfei Zhu Hai Zhao Xiaoguang Li AI4CE 69 35 0 26 Jan 2020
ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation Dongling Xiao Han Zhang Yukun Li Yu Sun Hao Tian Hua Wu Haifeng Wang 85 127 0 26 Jan 2020
Generating Representative Headlines for News Stories Xiaotao Gu Yuning Mao Jiawei Han Jialu Liu Hongkun Yu You Wu Cong Yu Daniel Finnie Jiaqi Zhai Nicholas Zukoski 97 70 0 26 Jan 2020
TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrieval Jie Lei Licheng Yu Tamara L. Berg Joey Tianyi Zhou 221 288 0 24 Jan 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 663 4,935 0 23 Jan 2020
Multilingual Denoising Pre-training for Neural Machine Translation Yinhan Liu Jiatao Gu Naman Goyal Xian Li Sergey Edunov Marjan Ghazvininejad M. Lewis Luke Zettlemoyer AI4CE AIMat 128 1,817 0 22 Jan 2020
ManyModalQA: Modality Disambiguation and QA over Diverse Inputs Darryl Hannan Akshay Jain Joey Tianyi Zhou AAML 74 59 0 22 Jan 2020
ImageBERT: Cross-modal Pre-training with Large-scale Weak-supervised Image-Text Data Di Qi Lin Su Jianwei Song Edward Cui Taroon Bharti Arun Sacheti VLM 104 263 0 22 Jan 2020
Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference Timo Schick Hinrich Schütze 365 1,627 0 21 Jan 2020
RobBERT: a Dutch RoBERTa-based Language Model Pieter Delobelle Thomas Winters Bettina Berendt 86 240 0 17 Jan 2020
A BERT based Sentiment Analysis and Key Entity Detection Approach for Online Financial Texts Lin Zhao Lin Li Xinhao Zheng 91 67 0 14 Jan 2020
CLUENER2020: Fine-grained Named Entity Recognition Dataset and Benchmark for Chinese Liang Xu Yu Tong Qianqian Dong Yixuan Liao Cong Yu Yin Tian Weitang Liu Lu Li Caiquan Liu Xuanwei Zhang 93 54 0 13 Jan 2020
AdaBERT: Task-Adaptive BERT Compression with Differentiable Neural Architecture Search Daoyuan Chen Yaliang Li Minghui Qiu Zhen Wang Bofang Li Bolin Ding Hongbo Deng Jun Huang Wei Lin Jingren Zhou MQ 88 104 0 13 Jan 2020
ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training Weizhen Qi Yu Yan Yeyun Gong Dayiheng Liu Nan Duan Jiusheng Chen Ruofei Zhang Ming Zhou AI4TS 134 450 0 13 Jan 2020
PatentTransformer-2: Controlling Patent Text Generation by Structural Metadata Jieh-Sheng Lee J. Hsiang 29 10 0 11 Jan 2020
LayoutLM: Pre-training of Text and Layout for Document Image Understanding Yiheng Xu Minghao Li Lei Cui Shaohan Huang Furu Wei Ming Zhou 155 718 0 31 Dec 2019
oLMpics -- On what Language Model Pre-training Captures Alon Talmor Yanai Elazar Yoav Goldberg Jonathan Berant LRM 105 304 0 31 Dec 2019
Learning and Evaluating Contextual Embedding of Source Code Aditya Kanade Petros Maniatis Gogul Balakrishnan Kensen Shi ELM 87 77 0 21 Dec 2019
Are Transformers universal approximators of sequence-to-sequence functions? Chulhee Yun Srinadh Bhojanapalli A. S. Rawat Sashank J. Reddi Sanjiv Kumar 140 358 0 20 Dec 2019
Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model Wenhan Xiong Jingfei Du William Yang Wang Veselin Stoyanov SSL KELM 105 201 0 20 Dec 2019
BERTje: A Dutch BERT Model Wietse de Vries Andreas van Cranenburgh Arianna Bisazza Tommaso Caselli Gertjan van Noord Malvina Nissim VLM SSeg 95 295 0 19 Dec 2019
Curriculum Learning Strategies for IR: An Empirical Study on Conversation Response Ranking Gustavo Penha C. Hauff 77 24 0 18 Dec 2019
Cross-Lingual Ability of Multilingual BERT: An Empirical Study Karthikeyan K Zihan Wang Stephen D. Mayhew Dan Roth LRM 79 340 0 17 Dec 2019
Multilingual is not enough: BERT for Finnish Antti Virtanen Jenna Kanerva Rami Ilo Jouni Luoma Juhani Luotolahti T. Salakoski Filip Ginter S. Pyysalo 88 281 0 15 Dec 2019
WaLDORf: Wasteless Language-model Distillation On Reading-comprehension J. Tian A. Kreuzer Pai-Hung Chen Hans-Martin Will VLM 58 3 0 13 Dec 2019
FlauBERT: Unsupervised Language Model Pre-training for French Hang Le Loïc Vial Jibril Frej Vincent Segonne Maximin Coavoux Benjamin Lecouteux A. Allauzen Benoît Crabbé Laurent Besacier D. Schwab AI4CE 98 401 0 11 Dec 2019
Automatic Spanish Translation of the SQuAD Dataset for Multilingual Question Answering C. Carrino Marta R. Costa-jussá José A. R. Fonollosa 64 89 0 11 Dec 2019
Large-scale Pretraining for Visual Dialog: A Simple State-of-the-Art Baseline Vishvak Murahari Dhruv Batra Devi Parikh Abhishek Das VLM 109 117 0 05 Dec 2019
12-in-1: Multi-Task Vision and Language Representation Learning Jiasen Lu Vedanuj Goswami Marcus Rohrbach Devi Parikh Stefan Lee VLM ObjD 116 481 0 05 Dec 2019
Bimodal Speech Emotion Recognition Using Pre-Trained Language Models Verena Heusser Niklas Freymuth Stefan Constantin A. Waibel 92 26 0 29 Nov 2019
Do Attention Heads in BERT Track Syntactic Dependencies? Phu Mon Htut Jason Phang Shikha Bordia Samuel R. Bowman 83 137 0 27 Nov 2019
Taking a Stance on Fake News: Towards Automatic Disinformation Assessment via Deep Bidirectional Transformer Language Models for Stance Detection Chris Dulhanty Jason L. Deglint Ibrahim Ben Daya A. Wong 44 22 0 27 Nov 2019
Evaluating Commonsense in Pre-trained Language Models Xuhui Zhou Yue Zhang Leyang Cui Dandan Huang AI4MH LRM 88 185 0 27 Nov 2019
PIQA: Reasoning about Physical Commonsense in Natural Language Yonatan Bisk Rowan Zellers Ronan Le Bras Jianfeng Gao Yejin Choi OOD LRM 221 1,851 0 26 Nov 2019
Pre-Training of Deep Bidirectional Protein Sequence Representations with Structural Information Seonwoo Min Seunghyun Park Siwon Kim Hyun-Soo Choi Byunghan Lee Sungroh Yoon SSL 71 63 0 25 Nov 2019
Unsupervised Domain Adaptation of Language Models for Reading Comprehension Kosuke Nishida Kyosuke Nishida Itsumi Saito Hisako Asano J. Tomita 95 26 0 25 Nov 2019
Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering Akari Asai Kazuma Hashimoto Hannaneh Hajishirzi R. Socher Caiming Xiong RALM KELM LRM 126 289 0 24 Nov 2019
A Transformer-based approach to Irony and Sarcasm detection Rolandos Alexandros Potamias Georgios Siolas A. Stafylopatis 47 213 0 23 Nov 2019
Automatic Text-based Personality Recognition on Monologues and Multiparty Dialogues Using Attentive Networks and Contextual Embeddings Hang Jiang Xianzhe Zhang Jinho Choi 47 59 0 21 Nov 2019
Global Greedy Dependency Parsing Z. Li Zhao Hai Kevin Parnow 113 31 0 20 Nov 2019
Working Memory Graphs Ricky Loynd Roland Fernandez Asli Celikyilmaz Adith Swaminathan Matthew J. Hausknecht 72 40 0 17 Nov 2019
The Eighth Dialog System Technology Challenge Seokhwan Kim Michel Galley Chulaka Gunasekara Sungjin Lee Adam Atkinson ... Tim K. Marks Abhinav Rastogi Xiaoxue Zang Srinivas Sunkara Raghav Gupta VLM 71 65 0 14 Nov 2019
Sato: Contextual Semantic Type Detection in Tables Dan Zhang Yoshihiko Suhara Jinfeng Li Madelon Hulsebos cCaugatay Demiralp W. Tan LMTD 46 15 0 14 Nov 2019
What do you mean, BERT? Assessing BERT as a Distributional Semantics Model Timothee Mickus Denis Paperno Mathieu Constant Kees van Deemter 77 46 0 13 Nov 2019
Adapting and evaluating a deep learning language model for clinical why-question answering Andrew Wen Mohamed Y. Elwazir Sungrim Moon Jungwei Fan LM&MA 54 31 0 13 Nov 2019
Neural Duplicate Question Detection without Labeled Training Data Andreas Rucklé N. Moosavi Iryna Gurevych OOD AAML 57 12 0 13 Nov 2019
KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation Xiaozhi Wang Tianyu Gao Zhaocheng Zhu Zhengyan Zhang Zhiyuan Liu Juan-Zi Li Jian Tang 153 675 0 13 Nov 2019
word2ket: Space-efficient Word Embeddings inspired by Quantum Entanglement Ali (Aliakbar) Panahi Seyran Saeedi Tom Arodz 61 32 0 12 Nov 2019
Attending to Entities for Better Text Understanding Pengxiang Cheng K. Erk LRM 59 38 0 11 Nov 2019
TANDA: Transfer and Adapt Pre-Trained Transformer Models for Answer Sentence Selection Siddhant Garg Thuy Vu Alessandro Moschitti 100 216 0 11 Nov 2019