RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 10,703 papers shown

Title
Multi-Step Inference for Reasoning Over Paragraphs Jiangming Liu Matt Gardner Shay B. Cohen Mirella Lapata ReLM LRM 49 18 0 06 Apr 2020
Evaluating the Evaluation of Diversity in Natural Language Generation Guy Tevet Jonathan Berant 101 99 0 06 Apr 2020
MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices Zhiqing Sun Hongkun Yu Xiaodan Song Renjie Liu Yiming Yang Denny Zhou MQ 130 820 0 06 Apr 2020
Evaluating Models' Local Decision Boundaries via Contrast Sets Matt Gardner Yoav Artzi Victoria Basmova Jonathan Berant Ben Bogin ... Sanjay Subramanian Reut Tsarfaty Eric Wallace Ally Zhang Ben Zhou ELM 118 84 0 06 Apr 2020
Residual Energy-Based Models for Text A. Bakhtin Yuntian Deng Sam Gross Myle Ott MarcÁurelio Ranzato Arthur Szlam 73 13 0 06 Apr 2020
Deep Learning Based Text Classification: A Comprehensive Review Shervin Minaee Nal Kalchbrenner Min Zhang Narjes Nikzad M. Asgari-Chenaghlu Jianfeng Gao AILaw VLM AI4TS 116 1,114 0 06 Apr 2020
TAPAS: Weakly Supervised Table Parsing via Pre-training Jonathan Herzig Pawel Krzysztof Nowak Thomas Müller Francesco Piccinno Julian Martin Eisenschlos LMTD RALM 150 658 0 05 Apr 2020
Continual Domain-Tuning for Pretrained Language Models Subendhu Rongali Abhyuday N. Jagannatha Bhanu Pratap Singh Rawat Hong-ye Yu CLL KELM 50 7 0 05 Apr 2020
FastBERT: a Self-distilling BERT with Adaptive Inference Time Weijie Liu Peng Zhou Zhe Zhao Zhiruo Wang Haotang Deng Qi Ju 95 361 0 05 Apr 2020
Unsupervised Domain Clusters in Pretrained Language Models Roee Aharoni Yoav Goldberg 101 252 0 05 Apr 2020
Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Space Chunyuan Li Xiang Gao Yuan Li Baolin Peng Xiujun Li Yizhe Zhang Jianfeng Gao SSL DRL 86 182 0 05 Apr 2020
Leveraging Multi-Source Weak Social Supervision for Early Detection of Fake News Kai Shu Guoqing Zheng Yichuan Li Subhabrata Mukherjee Ahmed Hassan Awadallah Scott W. Ruston Huan Liu 142 55 0 03 Apr 2020
XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation Yaobo Liang Nan Duan Yeyun Gong Ning Wu Fenfei Guo ... Shuguang Liu Fan Yang Daniel Fernando Campos Rangan Majumder Ming Zhou ELM VLM 115 350 0 03 Apr 2020
Deep Entity Matching with Pre-Trained Language Models Yuliang Li Jinfeng Li Yoshihiko Suhara A. Doan W. Tan VLM 108 391 0 01 Apr 2020
Give your Text Representation Models some Love: the Case for Basque Rodrigo Agerri Iñaki San Vicente Jon Ander Campos Ander Barrena X. Saralegi Aitor Soroa Etxabe Eneko Agirre 46 63 0 31 Mar 2020
Abstractive Summarization with Combination of Pre-trained Sequence-to-Sequence and Saliency Models Itsumi Saito Kyosuke Nishida Kosuke Nishida J. Tomita 77 29 0 29 Mar 2020
VIOLIN: A Large-Scale Dataset for Video-and-Language Inference J. Liu Wenhu Chen Yu Cheng Zhe Gan Licheng Yu Yiming Yang Jingjing Liu MLLM VGen 99 70 0 25 Mar 2020
Felix: Flexible Text Editing Through Tagging and Insertion Jonathan Mallinson Aliaksei Severyn Eric Malmi Guillermo Garrido 82 76 0 24 Mar 2020
Word2Vec: Optimal Hyper-Parameters and Their Impact on NLP Downstream Tasks Tosin Adewumi F. Liwicki Marcus Liwicki VLM 42 20 0 23 Mar 2020
TNT-KID: Transformer-based Neural Tagger for Keyword Identification Matej Martinc Blaž Škrlj Senja Pollak 87 38 0 20 Mar 2020
Enhancing Factual Consistency of Abstractive Summarization Chenguang Zhu William Fu-Hinthorn Ruochen Xu Qingkai Zeng Michael Zeng Xuedong Huang Meng Jiang HILM KELM 270 40 0 19 Mar 2020
TTTTTackling WinoGrande Schemas Sheng-Chieh Lin Jheng-Hong Yang Rodrigo Nogueira Ming-Feng Tsai Chuan-Ju Wang Jimmy Lin 55 6 0 18 Mar 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 388 1,498 0 18 Mar 2020
Transformer Networks for Trajectory Forecasting Francesco Giuliari Irtiza Hasan Marco Cristani Fabio Galasso 182 391 0 18 Mar 2020
Calibration of Pre-trained Transformers Shrey Desai Greg Durrett UQLM 337 302 0 17 Mar 2020
PowerNorm: Rethinking Batch Normalization in Transformers Sheng Shen Z. Yao A. Gholami Michael W. Mahoney Kurt Keutzer BDL 102 16 0 17 Mar 2020
A Survey on Contextual Embeddings Qi Liu Matt J. Kusner Phil Blunsom 274 151 0 16 Mar 2020
TRANS-BLSTM: Transformer with Bidirectional LSTM for Language Understanding Zhiheng Huang Peng Xu Davis Liang Ajay K. Mishra Bing Xiang 40 31 0 16 Mar 2020
Finnish Language Modeling with Deep Transformer Models Abhilash Jain Aku Rouhe Stig-Arne Gronroos M. Kurimo 14 0 0 14 Mar 2020
Know thy corpus! Robust methods for digital curation of Web corpora S. Sharoff 54 8 0 13 Mar 2020
Learning to Encode Position for Transformer with Continuous Dynamical Model Xuanqing Liu Hsiang-Fu Yu Inderjit Dhillon Cho-Jui Hsieh 85 112 0 13 Mar 2020
Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings H. Zhang Amy X. Lu Mohamed Abdalla Matthew B. A. McDermott Marzyeh Ghassemi 72 176 0 11 Mar 2020
Video2Commonsense: Generating Commonsense Descriptions to Enrich Video Captioning Zhiyuan Fang Tejas Gokhale Pratyay Banerjee Chitta Baral Yezhou Yang 78 63 0 11 Mar 2020
Multi-SimLex: A Large-Scale Evaluation of Multilingual and Cross-Lingual Lexical Semantic Similarity Ivan Vulić Simon Baker Edoardo Ponti Ulla Petti Ira Leviant ... Eden Bar Matt Malone Thierry Poibeau Roi Reichart Anna Korhonen 90 83 0 10 Mar 2020
Efficient Intent Detection with Dual Sentence Encoders I. Casanueva Tadas Temvcinas D. Gerz Matthew Henderson Ivan Vulić VLM 374 480 0 10 Mar 2020
Neuro-symbolic Architectures for Context Understanding A. Oltramari Jonathan M Francis C. Henson Kaixin Ma Ruwan Wickramarachchi NAI AI4CE 66 29 0 09 Mar 2020
Natural Language QA Approaches using Reasoning with External Knowledge Chitta Baral Pratyay Banerjee Kuntal Kumar Pal Arindam Mitra LRM 43 5 0 06 Mar 2020
On the Role of Conceptualization in Commonsense Knowledge Graph Construction Mutian He Yangqiu Song Kun Xu Dong Yu 40 13 0 06 Mar 2020
What the [MASK]? Making Sense of Language-Specific BERT Models Debora Nozza Federico Bianchi Dirk Hovy 162 108 0 05 Mar 2020
HypoNLI: Exploring the Artificial Patterns of Hypothesis-only Bias in Natural Language Inference Tianyu Liu Xin Zheng Baobao Chang Zhifang Sui 111 24 0 05 Mar 2020
Kleister: A novel task for Information Extraction involving Long Documents with Complex Layout Filip Graliñski Tomasz Stanislawek Anna Wróblewska Dawid Lipiñski Agnieszka Kaliska Paulina Rosalska Bartosz Topolski P. Biecek 77 41 0 04 Mar 2020
A Study on Efficiency, Accuracy and Document Structure for Answer Sentence Selection Daniele Bonadiman Alessandro Moschitti RALM 68 10 0 04 Mar 2020
jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models Yada Pruksachatkun Philip Yeres Haokun Liu Jason Phang Phu Mon Htut Alex Jinpeng Wang Ian Tenney Samuel R. Bowman SSeg 34 94 0 04 Mar 2020
Deep Multi-Modal Sets A. Reiter Menglin Jia Pu Yang Ser-Nam Lim BDL 67 4 0 03 Mar 2020
CLUECorpus2020: A Large-scale Chinese Corpus for Pre-training Language Model Liang Xu Xuanwei Zhang Qianqian Dong SSL 63 71 0 03 Mar 2020
Benchmark Performance of Machine And Deep Learning Based Methodologies for Urdu Text Document Classification Muhammad Nabeel Asim M. Ghani Muhammad Ali Ibrahim Sheraz Ahmed Waqar Mahmood Andreas Dengel 39 19 0 03 Mar 2020
Med7: a transferable clinical natural language processing model for electronic health records Andrey Kormilitzin N. Vaci Qiang Liu A. Nevado-Holgado 97 120 0 03 Mar 2020
PhoBERT: Pre-trained language models for Vietnamese Dat Quoc Nguyen A. Nguyen 270 357 0 02 Mar 2020
Style Example-Guided Text Generation using Generative Adversarial Transformers Kuo-Hao Zeng Mohammad Shoeybi Ming-Yuan Liu GAN 93 18 0 02 Mar 2020
AraBERT: Transformer-based Model for Arabic Language Understanding Wissam Antoun Fady Baly Hazem M. Hajj 162 975 0 28 Feb 2020