RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 4,659 papers shown

Title
Effective Unsupervised Domain Adaptation with Adversarially Trained Language Models Thuy-Trang Vu Dinh Q. Phung Gholamreza Haffari 22 24 0 05 Oct 2020
On Losses for Modern Language Models Stephane Aroca-Ouellette Frank Rudzicz 22 32 0 04 Oct 2020
An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot Labels Ilias Chalkidis Manos Fergadiotis Sotiris Kotitsas Prodromos Malakasiotis Nikolaos Aletras Ion Androutsopoulos VLM AI4TS 28 84 0 04 Oct 2020
Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space Dayiheng Liu Yeyun Gong Jie Fu Yu Yan Jiusheng Chen Jiancheng Lv Nan Duan M. Zhou 15 37 0 04 Oct 2020
Semantic Role Labeling Guided Multi-turn Dialogue ReWriter Kun Xu Haochen Tan Linfeng Song Han Wu Haisong Zhang Linqi Song Dong Yu KELM OffRL 21 27 0 03 Oct 2020
Cost-effective Selection of Pretraining Data: A Case Study of Pretraining BERT on Social Media Xiang Dai Sarvnaz Karimi Ben Hachey Cécile Paris 24 35 0 02 Oct 2020
Data-Efficient Pretraining via Contrastive Self-Supervision Nils Rethmeier Isabelle Augenstein 28 20 0 02 Oct 2020
LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention Ikuya Yamada Akari Asai Hiroyuki Shindo Hideaki Takeda Yuji Matsumoto 28 662 0 02 Oct 2020
MultiCQA: Zero-Shot Transfer of Self-Supervised Text Matching Models on a Massive Scale Andreas Rucklé Jonas Pfeiffer Iryna Gurevych 27 37 0 02 Oct 2020
XDA: Accurate, Robust Disassembly with Transfer Learning Kexin Pei Jonas Guan David Williams-King Junfeng Yang Suman Jana 9 58 0 02 Oct 2020
Beyond The Text: Analysis of Privacy Statements through Syntactic and Semantic Role Labeling Yan Shvartzshnaider Ananth Balashankar Vikas Patidar Thomas Wies L. Subramanian 19 4 0 01 Oct 2020
Understanding tables with intermediate pre-training Julian Martin Eisenschlos Syrine Krichene Thomas Müller LMTD 18 119 0 01 Oct 2020
A survey on natural language processing (nlp) and applications in insurance Antoine Ly Benno Uthayasooriyar Tingting Wang 16 14 0 01 Oct 2020
CoLAKE: Contextualized Language and Knowledge Embedding Tianxiang Sun Yunfan Shao Xipeng Qiu Qipeng Guo Yaru Hu Xuanjing Huang Zheng-Wei Zhang KELM 33 181 0 01 Oct 2020
Phonemer at WNUT-2020 Task 2: Sequence Classification Using COVID Twitter BERT and Bagging Ensemble Technique based on Plurality Voting Anshul Wadhawan 24 7 0 01 Oct 2020
RRF102: Meeting the TREC-COVID Challenge with a 100+ Runs Ensemble Michael Bendersky Honglei Zhuang Ji Ma Shuguang Han Keith B. Hall Ryan T. McDonald 21 16 0 01 Oct 2020
Examining the rhetorical capacities of neural language models Zining Zhu Chuer Pan Mohamed Abdalla Frank Rudzicz 36 10 0 01 Oct 2020
CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models Nikita Nangia Clara Vania Rasika Bhalerao Samuel R. Bowman 34 645 0 30 Sep 2020
Bridging Information-Seeking Human Gaze and Machine Reading Comprehension J. Malmaud R. Levy Yevgeni Berzak 30 32 0 30 Sep 2020
Towards a Multi-modal, Multi-task Learning based Pre-training Framework for Document Representation Learning Subhojeet Pramanik Shashank Mujumdar Hima Patel 21 31 0 30 Sep 2020
Parsing with Multilingual BERT, a Small Corpus, and a Small Treebank Ethan C. Chau Lucy H. Lin Noah A. Smith 22 15 0 29 Sep 2020
Utterance-level Dialogue Understanding: An Empirical Study Deepanway Ghosal Navonil Majumder Rada Mihalcea Soujanya Poria 27 23 0 29 Sep 2020
GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing Tao Yu Chien-Sheng Wu Xi Lin Bailin Wang Y. Tan Xinyi Yang Dragomir R. Radev R. Socher Caiming Xiong LMTD 38 248 0 29 Sep 2020
A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generation Dinghan Shen Ming Zheng Yelong Shen Yanru Qu Weizhu Chen AAML 29 130 0 29 Sep 2020
Double Graph Based Reasoning for Document-level Relation Extraction Shuang Zeng Runxin Xu Baobao Chang Lei Li 32 223 0 29 Sep 2020
Improve Transformer Models with Better Relative Position Embeddings Zhiheng Huang Davis Liang Peng Xu Bing Xiang ViT 26 127 0 28 Sep 2020
Conversational Semantic Parsing Armen Aghajanyan Jean Maillard Akshat Shrivastava K. Diedrick Mike Haeger ... Yashar Mehdad Ves Stoyanov Anuj Kumar M. Lewis S. Gupta 19 48 0 28 Sep 2020
What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams Di Jin Eileen Pan Nassim Oufattole W. Weng Hanyi Fang Peter Szolovits FaML ELM LM&MA 31 709 0 28 Sep 2020
What does it mean to be language-agnostic? Probing multilingual sentence encoders for typological properties Rochelle Choenni Ekaterina Shutova 25 37 0 27 Sep 2020
A Brief Survey and Comparative Study of Recent Development of Pronoun Coreference Resolution Hongming Zhang Xinran Zhao Yangqiu Song 18 9 0 27 Sep 2020
KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoning Ye Liu Yao Wan Lifang He Hao Peng Philip S. Yu 32 188 0 26 Sep 2020
RecoBERT: A Catalog Language Model for Text-Based Recommendations Itzik Malkiel Oren Barkan Avi Caciularu Noam Razin Ori Katz Noam Koenigstein 17 13 0 25 Sep 2020
No Answer is Better Than Wrong Answer: A Reflection Model for Document Level Machine Reading Comprehension Xuguang Wang Linjun Shou Ming Gong Nan Duan Daxin Jiang 24 12 0 25 Sep 2020
Machine Knowledge: Creation and Curation of Comprehensive Knowledge Bases Gerhard Weikum Luna Dong Simon Razniewski Fabian M. Suchanek 37 125 0 24 Sep 2020
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models Samuel Gehman Suchin Gururangan Maarten Sap Yejin Choi Noah A. Smith 55 1,134 0 24 Sep 2020
Streamlining Cross-Document Coreference Resolution: Evaluation and Modeling Arie Cattan Alon Eirew Gabriel Stanovsky Mandar Joshi Ido Dagan 16 35 0 23 Sep 2020
Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics Swabha Swayamdipta Roy Schwartz Nicholas Lourie Yizhong Wang Hannaneh Hajishirzi Noah A. Smith Yejin Choi 51 429 0 22 Sep 2020
On Data Augmentation for Extreme Multi-label Classification Danqing Zhang Tao Li H. Zhang Bing Yin 22 25 0 22 Sep 2020
Constructing interval variables via faceted Rasch measurement and multitask deep learning: a hate speech application Chris J. Kennedy Geoff Bacon A. Sahn Claudia von Vacano 25 80 0 22 Sep 2020
Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data Jonathan Pilault Amine Elhattami C. Pal CLL MoE 30 89 0 19 Sep 2020
Self-Supervised Meta-Learning for Few-Shot Natural Language Classification Tasks Trapit Bansal Rishikesh Jha Tsendsuren Munkhdalai Andrew McCallum SSL VLM 33 87 0 17 Sep 2020
A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support Ashish Sharma Adam S. Miner David C. Atkins Tim Althoff AI4MH 25 272 0 17 Sep 2020
GraphCodeBERT: Pre-training Code Representations with Data Flow Daya Guo Shuo Ren Shuai Lu Zhangyin Feng Duyu Tang ... Dawn Drain Neel Sundaresan Jian Yin Daxin Jiang M. Zhou 88 1,102 0 17 Sep 2020
Efficient Transformer-based Large Scale Language Representations using Hardware-friendly Block Structured Pruning Bingbing Li Zhenglun Kong Tianyun Zhang Ji Li Zechao Li Hang Liu Caiwen Ding VLM 32 64 0 17 Sep 2020
Automated Source Code Generation and Auto-completion Using Deep Learning: Comparing and Discussing Current Language-Model-Related Approaches Juan Cruz-Benito Sanjay Vishwakarma Francisco Martín-Fernández Ismael Faro Ibm Quantum 22 30 0 16 Sep 2020
Reasoning about Goals, Steps, and Temporal Ordering with WikiHow Li Zhang Qing Lyu Chris Callison-Burch ReLM LRM 32 85 0 16 Sep 2020
Multi-span Style Extraction for Generative Reading Comprehension Junjie Yang ZhuoSheng Zhang Hai Zhao SyDa 19 14 0 15 Sep 2020
Evaluating representations by the complexity of learning low-loss predictors William F. Whitney M. Song David Brandfonbrener Jaan Altosaar Kyunghyun Cho 25 23 0 15 Sep 2020
Augmented Natural Language for Generative Sequence Labeling Ben Athiwaratkun Cicero Nogueira dos Santos Jason Krone Bing Xiang VLM 19 61 0 15 Sep 2020
Critical Thinking for Language Models Gregor Betz Christian Voigt Kyle Richardson SyDa ReLM LRM AI4CE 26 35 0 15 Sep 2020