RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 4,753 papers shown

Title
A Short Survey of Pre-trained Language Models for Conversational AI-A NewAge in NLP Munazza Zaib Quan Z. Sheng W. Zhang 24 67 0 22 Apr 2021
Modeling Event Plausibility with Consistent Conceptual Abstraction Ian Porada Kaheer Suleman Adam Trischler Jackie C.K. Cheung 113 19 0 20 Apr 2021
Identify, Align, and Integrate: Matching Knowledge Graphs to Commonsense Reasoning Tasks Lisa Bauer Mohit Bansal 19 19 0 20 Apr 2021
Hidden Biases in Unreliable News Detection Datasets Xiang Zhou Heba Elfardy Christos Christodoulopoulos Thomas Butler Joey Tianyi Zhou 24 15 0 20 Apr 2021
Enhancing Cognitive Models of Emotions with Representation Learning Yuting Guo Jinho Choi 33 5 0 20 Apr 2021
skweak: Weak Supervision Made Easy for NLP Pierre Lison Jeremy Barnes A. Hubin 29 43 0 19 Apr 2021
Refining Targeted Syntactic Evaluation of Language Models Benjamin Newman Kai-Siang Ang Julia Gong John Hewitt 29 43 0 19 Apr 2021
Operationalizing a National Digital Library: The Case for a Norwegian Transformer Model P. Kummervold Javier de la Rosa Freddy Wetjen Svein Arne Brygfjeld 22 55 0 19 Apr 2021
BERTić -- The Transformer Language Model for Bosnian, Croatian, Montenegrin and Serbian N. Ljubešić D. Lauc 16 48 0 19 Apr 2021
IIITT@LT-EDI-EACL2021-Hope Speech Detection: There is always Hope in Transformers Karthik Puranik Adeep Hande R. Priyadharshini Sajeetha Thavareesan Bharathi Raja Chakravarthi 28 59 0 19 Apr 2021
Natural Language Generation Using Link Grammar for General Conversational Intelligence Vignav Ramesh Anton Kolonin 19 2 0 19 Apr 2021
Data-Efficient Language-Supervised Zero-Shot Learning with Self-Distillation Rui Cheng Bichen Wu Peizhao Zhang Peter Vajda Joseph E. Gonzalez CLIP VLM 21 31 0 18 Apr 2021
Reference-based Weak Supervision for Answer Sentence Selection using Web Data Vivek Krishnamurthy Thuy Vu Alessandro Moschitti 21 1 0 18 Apr 2021
On the Influence of Masking Policies in Intermediate Pre-training Qinyuan Ye Belinda Z. Li Sinong Wang Benjamin Bolte Hao Ma Wen-tau Yih Xiang Ren Madian Khabsa 20 12 0 18 Apr 2021
SciCo: Hierarchical Cross-Document Coreference for Scientific Concepts Arie Cattan Sophie Johnson Daniel S. Weld Ido Dagan Iz Beltagy Doug Downey Tom Hope 30 23 0 18 Apr 2021
Back-Training excels Self-Training at Unsupervised Domain Adaptation of Question Generation and Passage Retrieval Devang Kulshreshtha Robert Belfer Iulian Serban Siva Reddy OOD 20 16 0 18 Apr 2021
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity Yao Lu Max Bartolo Alastair Moore Sebastian Riedel Pontus Stenetorp AILaw LRM 281 1,124 0 18 Apr 2021
Case-based Reasoning for Natural Language Queries over Knowledge Bases Rajarshi Das Manzil Zaheer Dung Ngoc Thai Ameya Godbole Ethan Perez Jay Yoon Lee Lizhen Tan L. Polymenakos Andrew McCallum 36 163 0 18 Apr 2021
Can NLI Models Verify QA Systems' Predictions? Jifan Chen Eunsol Choi Greg Durrett 42 54 0 18 Apr 2021
A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation Tianyu Liu Yizhe Zhang Chris Brockett Yi Mao Zhifang Sui Weizhu Chen W. Dolan HILM 228 144 0 18 Apr 2021
Knowledge Neurons in Pretrained Transformers Damai Dai Li Dong Y. Hao Zhifang Sui Baobao Chang Furu Wei KELM MU 28 422 0 18 Apr 2021
"Average" Approximates "First Principal Component"? An Empirical Analysis on Representations from Neural Language Models Zihan Wang Chengyu Dong Jingbo Shang FAtt 42 4 0 18 Apr 2021
BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models Nandan Thakur Nils Reimers Andreas Rucklé Abhishek Srivastava Iryna Gurevych VLM 278 977 0 17 Apr 2021
The Topic Confusion Task: A Novel Scenario for Authorship Attribution Malik H. Altakrori Jackie C.K. Cheung Benjamin C. M. Fung 19 18 0 17 Apr 2021
Robust Embeddings Via Distributions Kira A. Selby Yinong Wang Ruizhe Wang Peyman Passban Ahmad Rashid Mehdi Rezagholizadeh Pascal Poupart OOD 34 3 0 17 Apr 2021
ESTER: A Machine Reading Comprehension Dataset for Event Semantic Relation Reasoning Rujun Han I-Hung Hsu Jiao Sun J. Baylón Qiang Ning Dan Roth Nanyun Peng 32 45 0 16 Apr 2021
On the Importance of Effectively Adapting Pretrained Language Models for Active Learning Katerina Margatina Loïc Barrault Nikolaos Aletras 29 36 0 16 Apr 2021
Memorisation versus Generalisation in Pre-trained Language Models Michael Tänzer Sebastian Ruder Marek Rei 94 50 0 16 Apr 2021
Membership Inference Attack Susceptibility of Clinical Language Models Abhyuday N. Jagannatha Bhanu Pratap Singh Rawat Hong-ye Yu MIACV 29 62 0 16 Apr 2021
AMMU : A Survey of Transformer-based Biomedical Pretrained Language Models Katikapalli Subramanyam Kalyan A. Rajasekharan S. Sangeetha LM&MA MedIm 31 164 0 16 Apr 2021
What to Pre-Train on? Efficient Intermediate Task Selection Clifton A. Poth Jonas Pfeiffer Andreas Rucklé Iryna Gurevych 24 95 0 16 Apr 2021
Flexible Instance-Specific Rationalization of NLP Models G. Chrysostomou Nikolaos Aletras 36 14 0 16 Apr 2021
$$Q^{2}$: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answering$ $Q^{2}$ : Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answering Or Honovich Leshem Choshen Roee Aharoni Ella Neeman Idan Szpektor Omri Abend HILM 36 138 0 16 Apr 2021
Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schema Yanai Elazar Hongming Zhang Yoav Goldberg Dan Roth ReLM LRM 45 44 0 16 Apr 2021
Supervising Model Attention with Human Explanations for Robust Natural Language Inference Joe Stacey Yonatan Belinkov Marek Rei 30 45 0 16 Apr 2021
Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders Fangyu Liu Ivan Vulić Anna Korhonen Nigel Collier VLM OffRL 37 117 0 16 Apr 2021
BERT2Code: Can Pretrained Language Models be Leveraged for Code Search? Abdullah Al Ishtiaq Masum Hasan Md. Mahim Anjum Haque Kazi Sajeed Mehrab Tanveer Muttaqueen Tahmid Hasan Anindya Iqbal Rifat Shahriyar 14 5 0 16 Apr 2021
Probing Across Time: What Does RoBERTa Know and When? Leo Z. Liu Yizhong Wang Jungo Kasai Hannaneh Hajishirzi Noah A. Smith KELM 16 80 0 16 Apr 2021
Multivalent Entailment Graphs for Question Answering Nick McKenna Liane Guillou Mohammad Javad Hosseini Sander Bijl de Vroe Mark Johnson Mark Steedman NAI 48 14 0 16 Apr 2021
Exploring Visual Engagement Signals for Representation Learning Menglin Jia Zuxuan Wu A. Reiter Claire Cardie Serge Belongie Ser-Nam Lim 21 13 0 15 Apr 2021
Does BERT Pretrained on Clinical Notes Reveal Sensitive Data? Eric P. Lehman Sarthak Jain Karl Pichotta Yoav Goldberg Byron C. Wallace OOD MIACV 26 119 0 15 Apr 2021
How to Train BERT with an Academic Budget Peter Izsak Moshe Berchansky Omer Levy 23 114 0 15 Apr 2021
Gradient-based Adversarial Attacks against Text Transformers Chuan Guo Alexandre Sablayrolles Hervé Jégou Douwe Kiela SILM 106 230 0 15 Apr 2021
Syntactic Perturbations Reveal Representational Correlates of Hierarchical Phrase Structure in Pretrained Language Models Matteo Alleman J. Mamou Miguel Rio Hanlin Tang Yoon Kim SueYeon Chung NAI 46 17 0 15 Apr 2021
Hierarchical Learning for Generation with Long Source Sequences T. Rohde Xiaoxia Wu Yinhan Liu BDL VLM 25 56 0 15 Apr 2021
Generating Datasets with Pretrained Language Models Timo Schick Hinrich Schütze 24 234 0 15 Apr 2021
Cross-Domain Label-Adaptive Stance Detection Momchil Hardalov Arnav Arora Preslav Nakov Isabelle Augenstein 40 72 0 15 Apr 2021
XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation Sebastian Ruder Noah Constant Jan A. Botha Aditya Siddhant Orhan Firat ... Pengfei Liu Junjie Hu Dan Garrette Graham Neubig Melvin Johnson ELM AAML LRM 29 184 0 15 Apr 2021
Span Pointer Networks for Non-Autoregressive Task-Oriented Semantic Parsing Akshat Shrivastava P. Chuang Arun Babu Shrey Desai Abhinav Arora Alexander Zotov Ahmed Aly 24 21 0 15 Apr 2021
Integration of Pre-trained Networks with Continuous Token Interface for End-to-End Spoken Language Understanding S. Seo Donghyun Kwak Bowon Lee 32 33 0 15 Apr 2021