RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 4,842 papers shown

Title
ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback Mike Wu Noah D. Goodman Chris Piech Chelsea Finn 40 19 0 23 Jul 2021
Modelling Latent Translations for Cross-Lingual Transfer Edoardo Ponti Julia Kreutzer Ivan Vulić Siva Reddy 37 18 0 23 Jul 2021
Did the Cat Drink the Coffee? Challenging Transformers with Generalized Event Knowledge Paolo Pedinotti Giulia Rambelli Emmanuele Chersoni Enrico Santus Alessandro Lenci P. Blache 27 27 0 22 Jul 2021
Evaluation of contextual embeddings on less-resourced languages Matej Ulvcar Alevs vZagar C. S. Armendariz Andravz Repar Senja Pollak Matthew Purver Marko Robnik-vSikonja 41 11 0 22 Jul 2021
Back-Translated Task Adaptive Pretraining: Improving Accuracy and Robustness on Text Classification Junghoon Lee Jounghee Kim Pilsung Kang VLM 19 5 0 22 Jul 2021
Spinning Sequence-to-Sequence Models with Meta-Backdoors Eugene Bagdasaryan Vitaly Shmatikov SILM AAML 45 8 0 22 Jul 2021
Small-Text: Active Learning for Text Classification in Python Christopher Schröder Lydia Muller A. Niekler Martin Potthast CLIP VLM AI4CE 39 23 0 21 Jul 2021
Improved Text Classification via Contrastive Adversarial Training Lin Pan Chung-Wei Hang Avirup Sil Saloni Potdar AAML 33 86 0 21 Jul 2021
CausalBERT: Injecting Causal Knowledge Into Pre-trained Models with Minimal Supervision Zhongyang Li Xiao Ding Kuo Liao Bing Qin Ting Liu CML 29 17 0 21 Jul 2021
Large-scale graph representation learning with very deep GNNs and self-supervision Ravichandra Addanki Peter W. Battaglia David Budden Andreea Deac Jonathan Godwin ... Wai Lok Sibon Li Alvaro Sanchez-Gonzalez Jacklynn Stott S. Thakoor Petar Velivcković SSL AI4CE 27 25 0 20 Jul 2021
Generative Pretraining for Paraphrase Evaluation J. Weston R. Lenain U. Meepegama E. Fristed AIMat 27 10 0 17 Jul 2021
Overview and Insights from the SciVer Shared Task on Scientific Claim Verification David Wadden Kyle Lo 72 11 0 17 Jul 2021
Are Multilingual Models the Best Choice for Moderately Under-resourced Languages? A Comprehensive Assessment for Catalan Jordi Armengol-Estapé C. Carrino Carlos Rodríguez-Penagos Ona de Gibert Bonet Carme Armentano-Oller Aitor Gonzalez-Agirre Maite Melero Marta Villegas 68 42 0 16 Jul 2021
FewCLUE: A Chinese Few-shot Learning Evaluation Benchmark Liang Xu Xiaojing Lu Chenyang Yuan Xuanwei Zhang Huilin Xu ... Guoao Wei X. Pan Xin Tian Libo Qin Hai Hu ELM 24 57 0 15 Jul 2021
AutoBERT-Zero: Evolving BERT Backbone from Scratch Jiahui Gao Hang Xu Han Shi Xiaozhe Ren Philip L. H. Yu Xiaodan Liang Xin Jiang Zhenguo Li 21 37 0 15 Jul 2021
Trusting RoBERTa over BERT: Insights from CheckListing the Natural Language Inference Task Ishan Tarunesh Somak Aditya Monojit Choudhury 15 17 0 15 Jul 2021
A Multimodal Machine Learning Framework for Teacher Vocal Delivery Evaluation Hang Li Yunxing Kang Y. Hao Wenbiao Ding Zhongqin Wu Zitao Liu 30 4 0 15 Jul 2021
The Benchmark Lottery Mostafa Dehghani Yi Tay A. Gritsenko Zhe Zhao N. Houlsby Fernando Diaz Donald Metzler Oriol Vinyals 44 89 0 14 Jul 2021
Increasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Features Hannah Rashkin David Reitter Gaurav Singh Tomar Dipanjan Das 175 101 0 14 Jul 2021
BERT Fine-Tuning for Sentiment Analysis on Indonesian Mobile Apps Reviews Kuncahyo Setyo Nugroho Anantha Yullian Sukmadewa DW HaftittahWuswilahaken F. A. Bachtiar N. Yudistira 26 43 0 14 Jul 2021
Combiner: Full Attention Transformer with Sparse Computation Cost Hongyu Ren H. Dai Zihang Dai Mengjiao Yang J. Leskovec Dale Schuurmans Bo Dai 87 77 0 12 Jul 2021
Tortured phrases: A dubious writing style emerging in science. Evidence of critical issues affecting established journals G. Cabanac C. Labbé A. Magazinov DeLMO 50 80 0 12 Jul 2021
Accenture at CheckThat! 2021: Interesting claim identification and ranking with contextually sensitive lexical training data augmentation Evan Williams Paul Rodrigues Sieu Tran 139 19 0 12 Jul 2021
Sliding Spectrum Decomposition for Diversified Recommendation Yanhua Huang Weikun Wang Lei Zhang Ruiwen Xu 19 40 0 12 Jul 2021
Noise Stability Regularization for Improving BERT Fine-tuning Hang Hua Xingjian Li Dejing Dou Chengzhong Xu Jiebo Luo 19 44 0 10 Jul 2021
Can Deep Neural Networks Predict Data Correlations from Column Names? Immanuel Trummer 22 8 0 09 Jul 2021
A Survey on Low-Resource Neural Machine Translation Rui Wang Xu Tan Renqian Luo Tao Qin Tie-Yan Liu 3DV 43 58 0 09 Jul 2021
Keep it Simple: Unsupervised Simplification of Multi-Paragraph Text Philippe Laban Tobias Schnabel Paul N. Bennett Marti A. Hearst 23 60 0 07 Jul 2021
Evaluating Large Language Models Trained on Code Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Pondé ... Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever Wojciech Zaremba ELM ALM 86 5,161 0 07 Jul 2021
Learning Vision Transformer with Squeeze and Excitation for Facial Expression Recognition Mouath Aouayeb W. Hamidouche Catherine Soladié K. Kpalma Renaud Séguier ViT 28 58 0 07 Jul 2021
Android Security using NLP Techniques: A Review Sevil Sen Burcu Can AAML 24 4 0 07 Jul 2021
Neural Natural Language Processing for Unstructured Data in Electronic Health Records: a Review Irene Z Li Jessica Pan Jeremy Goldwasser Neha Verma Wai Pan Wong ... Matthew Zhang David Chang R. Taylor H. Krumholz Dragomir R. Radev BDL 31 154 0 07 Jul 2021
Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoning Maxwell Nye Michael Henry Tessler J. Tenenbaum Brenden M. Lake 33 117 0 06 Jul 2021
Sarcasm Detection: A Comparative Study Hamed Yaghoobian H. Arabnia Khaled Rasheed 31 22 0 05 Jul 2021
DeepRapper: Neural Rap Generation with Rhyme and Rhythm Modeling Lanqing Xue Kaitao Song Duocai Wu Xu Tan N. Zhang Tao Qin Weiqiang Zhang Tie-Yan Liu 37 37 0 05 Jul 2021
Doing Good or Doing Right? Exploring the Weakness of Commonsense Causal Reasoning Models Mingyue Han Yinglin Wang LRM 21 11 0 05 Jul 2021
DRIFT: A Toolkit for Diachronic Analysis of Scientific Literature Abheesht Sharma Gunjan Chhablani Harshit Pandey Rajaswa Patil 35 7 0 02 Jul 2021
Learned Token Pruning for Transformers Sehoon Kim Sheng Shen D. Thorsley A. Gholami Woosuk Kwon Joseph Hassoun Kurt Keutzer 17 146 0 02 Jul 2021
He Thinks He Knows Better than the Doctors: BERT for Event Factuality Fails on Pragmatics Nan-Jiang Jiang M. Marneffe 32 21 0 02 Jul 2021
An Investigation of the (In)effectiveness of Counterfactually Augmented Data Nitish Joshi He He OODD 24 46 0 01 Jul 2021
A Primer on Pretrained Multilingual Language Models Sumanth Doddapaneni Gowtham Ramesh Mitesh M. Khapra Anoop Kunchukuttan Pratyush Kumar LRM 59 74 0 01 Jul 2021
Productivity, Portability, Performance: Data-Centric Python Yiheng Wang Yao Zhang Yanzhang Wang Yan Wan Jiao Wang Zhongyuan Wu Yuhao Yang Bowen She 59 95 0 01 Jul 2021
CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understanding Dong Wang Ning Ding Pijian Li Haitao Zheng AAML 39 115 0 01 Jul 2021
Elbert: Fast Albert with Confidence-Window Based Early Exit Keli Xie Siyuan Lu Meiqi Wang Zhongfeng Wang 22 20 0 01 Jul 2021
Controllable Open-ended Question Generation with A New Question Type Ontology Shuyang Cao Lu Wang 28 50 0 01 Jul 2021
Regressing Location on Text for Probabilistic Geocoding Benjamin J. Radford BDL 18 12 0 30 Jun 2021
The MultiBERTs: BERT Reproductions for Robustness Analysis Thibault Sellam Steve Yadlowsky Jason W. Wei Naomi Saphra Alexander DÁmour ... Iulia Turc Jacob Eisenstein Dipanjan Das Ian Tenney Ellie Pavlick 24 93 0 30 Jun 2021
ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information Zijun Sun Xiaoya Li Xiaofei Sun Yuxian Meng Xiang Ao Qing He Fei Wu Jiwei Li SSeg 57 184 0 30 Jun 2021
What can linear interpolation of neural network loss landscapes tell us? Tiffany J. Vlaar Jonathan Frankle MoMe 30 27 0 30 Jun 2021
Automatically Select Emotion for Response via Personality-affected Emotion Transition Zhiyuan Wen Jiannong Cao Ruosong Yang Shuaiqi Liu Jiaxing Shen 37 25 0 30 Jun 2021