RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 4,752 papers shown

Title
Are Neural Language Models Good Plagiarists? A Benchmark for Neural Paraphrase Detection Jan Philip Wahle Terry Ruas Norman Meuschke Bela Gipp 30 34 0 23 Mar 2021
Instance-level Image Retrieval using Reranking Transformers Fuwen Tan Jiangbo Yuan Vicente Ordonez ViT 28 89 0 22 Mar 2021
BERT: A Review of Applications in Natural Language Processing and Understanding M. V. Koroteev VLM 25 197 0 22 Mar 2021
Retrieve Fast, Rerank Smart: Cooperative and Joint Approaches for Improved Cross-Modal Retrieval Gregor Geigle Jonas Pfeiffer Nils Reimers Ivan Vulić Iryna Gurevych 40 60 0 22 Mar 2021
Identifying Machine-Paraphrased Plagiarism Jan Philip Wahle Terry Ruas Tomávs Foltýnek Norman Meuschke Bela Gipp 11 30 0 22 Mar 2021
DeepViT: Towards Deeper Vision Transformer Daquan Zhou Bingyi Kang Xiaojie Jin Linjie Yang Xiaochen Lian Zihang Jiang Qibin Hou Jiashi Feng ViT 42 511 0 22 Mar 2021
Exploiting Method Names to Improve Code Summarization: A Deliberation Multi-Task Learning Approach Rui Xie Wei Ye Jinan Sun Shikun Zhang 28 26 0 21 Mar 2021
AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarization Tiezheng Yu Zihan Liu Pascale Fung CLL 51 81 0 21 Mar 2021
API2Com: On the Improvement of Automatically Generated Code Comments Using API Documentations Ramin Shahbazi Rishab Sharma Fatemeh H. Fard 27 25 0 19 Mar 2021
GPT Understands, Too Xiao Liu Yanan Zheng Zhengxiao Du Ming Ding Yujie Qian Zhilin Yang Jie Tang VLM 87 1,147 0 18 Mar 2021
GLM: General Language Model Pretraining with Autoregressive Blank Infilling Zhengxiao Du Yujie Qian Xiao Liu Ming Ding J. Qiu Zhilin Yang Jie Tang BDL AI4CE 53 1,496 0 18 Mar 2021
Model Extraction and Adversarial Transferability, Your BERT is Vulnerable! Xuanli He Lingjuan Lyu Qiongkai Xu Lichao Sun MIACV SILM 36 91 0 18 Mar 2021
On the Role of Images for Analyzing Claims in Social Media Gullal Singh Cheema Sherzod Hakimov Eric Müller-Budack Ralph Ewerth 29 10 0 17 Mar 2021
Towards Few-Shot Fact-Checking via Perplexity Nayeon Lee Yejin Bang Andrea Madotto Madian Khabsa Pascale Fung AAML 13 90 0 17 Mar 2021
Investigating Monolingual and Multilingual BERTModels for Vietnamese Aspect Category Detection D. Thin Lac Si Le V. Hoang Ngan Luu-Thuy Nguyen 31 10 0 17 Mar 2021
Structural Adapters in Pretrained Language Models for AMR-to-text Generation Leonardo F. R. Ribeiro Yue Zhang Iryna Gurevych 43 69 0 16 Mar 2021
LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval Siqi Sun Yen-Chun Chen Linjie Li Shuohang Wang Yuwei Fang Jingjing Liu VLM 41 82 0 16 Mar 2021
How Many Data Points is a Prompt Worth? Teven Le Scao Alexander M. Rush VLM 66 296 0 15 Mar 2021
Deep Discourse Analysis for Generating Personalized Feedback in Intelligent Tutor Systems Matt Grenander Robert Belfer E. Kochmar Iulian Serban Franccois St-Hilaire Jackie C.K. Cheung AI4Ed 30 17 0 13 Mar 2021
Text Mining of Stocktwits Data for Predicting Stock Prices Mukul Jaggi Priyanka Mandal Shreya Narang Usman Naseem Matloob Khushi AIFin 18 41 0 13 Mar 2021
Cooperative Self-training of Machine Reading Comprehension Hongyin Luo Shang-Wen Li Ming Gao Seunghak Yu James R. Glass SyDa RALM 20 11 0 12 Mar 2021
Are NLP Models really able to Solve Simple Math Word Problems? Arkil Patel S. Bhattamishra Navin Goyal ReLM LRM 27 776 0 12 Mar 2021
Inductive Relation Prediction by BERT H. Zha Zhiyu Zoey Chen Xifeng Yan 29 54 0 12 Mar 2021
MERMAID: Metaphor Generation with Symbolism and Discriminative Decoding Tuhin Chakrabarty Xurui Zhang Smaranda Muresan Nanyun Peng 33 68 0 11 Mar 2021
The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models Go Inoue Bashar Alhafni Nurpeiis Baimukan Houda Bouamor Nizar Habash 35 226 0 11 Mar 2021
ReportAGE: Automatically extracting the exact age of Twitter users based on self-reports in tweets A. Klein A. Magge G. Gonzalez-Hernandez 12 20 0 10 Mar 2021
Unified Pre-training for Program Understanding and Generation Wasi Uddin Ahmad Saikat Chakraborty Baishakhi Ray Kai-Wei Chang 41 754 0 10 Mar 2021
CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review Dan Hendrycks Collin Burns Anya Chen Spencer Ball ELM AILaw 25 185 0 10 Mar 2021
BERTese: Learning to Speak to BERT Adi Haviv Jonathan Berant Amir Globerson 30 123 0 09 Mar 2021
Text Simplification by Tagging Kostiantyn Omelianchuk Vipul Raheja Oleksandr Skurzhanskyi 27 45 0 08 Mar 2021
Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges Yoshitomo Matsubara Marco Levorato Francesco Restuccia 33 199 0 08 Mar 2021
Syntax-BERT: Improving Pre-trained Transformers with Syntax Trees Jiangang Bai Yujing Wang Yiren Chen Yaming Yang Jing Bai Jiahao Yu Yunhai Tong 45 104 0 07 Mar 2021
MalBERT: Using Transformers for Cybersecurity and Malicious Software Detection Abir Rahali M. Akhloufi 32 30 0 05 Mar 2021
Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices Max Ryabinin Eduard A. Gorbunov Vsevolod Plokhotnyuk Gennady Pekhimenko 42 33 0 04 Mar 2021
Natural Language Understanding for Argumentative Dialogue Systems in the Opinion Building Domain W. A. Abro Annalena Aicher Niklas Rach Stefan Ultes Wolfgang Minker Guilin Qi 33 32 0 03 Mar 2021
OAG-BERT: Towards A Unified Backbone Language Model For Academic Knowledge Services Xiao Liu Da Yin Jingnan Zheng Xingjian Zhang Peng Zhang Hongxia Yang Yuxiao Dong Jie Tang VLM 45 31 0 03 Mar 2021
The Rediscovery Hypothesis: Language Models Need to Meet Linguistics Vassilina Nikoulina Maxat Tezekbayev Nuradil Kozhakhmet Madina Babazhanova Matthias Gallé Z. Assylbekov 34 8 0 02 Mar 2021
Disentangling Syntax and Semantics in the Brain with Deep Networks Charlotte Caucheteux Alexandre Gramfort J. King 36 70 0 02 Mar 2021
Contrastive Explanations for Model Interpretability Alon Jacovi Swabha Swayamdipta Shauli Ravfogel Yanai Elazar Yejin Choi Yoav Goldberg 49 95 0 02 Mar 2021
ToxCCIn: Toxic Content Classification with Interpretability Tong Xiang Sean MacAvaney Eugene Yang Nazli Goharian 82 15 0 01 Mar 2021
Sentiment Analysis of Users' Reviews on COVID-19 Contact Tracing Apps with a Benchmark Dataset Kashif Ahmad Firoj Alam Junaid Qadir Basheer Qolomany Imran Khan ... M. Suleman Naina Said Syed Zohaib Hassan Asma Gul Ala I. Al-Fuqaha 26 7 0 01 Mar 2021
M6: A Chinese Multimodal Pretrainer Junyang Lin Rui Men An Yang Chan Zhou Ming Ding ... Yong Li Wei Lin Jingren Zhou J. Tang Hongxia Yang VLM MoE 37 133 0 01 Mar 2021
A Primer on Contrastive Pretraining in Language Processing: Methods, Lessons Learned and Perspectives Nils Rethmeier Isabelle Augenstein SSL VLM 96 91 0 25 Feb 2021
ZJUKLAB at SemEval-2021 Task 4: Negative Augmentation with Language Model for Reading Comprehension of Abstract Meaning Xin Xie Xiangnan Chen Xiang Chen Yong Wang Ningyu Zhang Shumin Deng Huajun Chen 42 2 0 25 Feb 2021
LazyFormer: Self Attention with Lazy Update Chengxuan Ying Guolin Ke Di He Tie-Yan Liu 25 15 0 25 Feb 2021
Generating Human Readable Transcript for Automatic Speech Recognition with Pre-trained Language Model Junwei Liao Yu Shi Ming Gong Linjun Shou Sefik Emre Eskimez Liyang Lu Hong Qu Michael Zeng 25 9 0 22 Feb 2021
Position Information in Transformers: An Overview Philipp Dufter Martin Schmitt Hinrich Schütze 34 141 0 22 Feb 2021
LogME: Practical Assessment of Pre-trained Models for Transfer Learning Kaichao You Yong Liu Jianmin Wang Mingsheng Long 32 178 0 22 Feb 2021
Better Call the Plumber: Orchestrating Dynamic Information Extraction Pipelines M. Y. Jaradeh Kuldeep Singh M. Stocker A. Both Sören Auer 22 7 0 22 Feb 2021
UniT: Multimodal Multitask Learning with a Unified Transformer Ronghang Hu Amanpreet Singh ViT 25 296 0 22 Feb 2021