RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 4,639 papers shown

Title
Sparsifying Transformer Models with Trainable Representation Pooling Michal Pietruszka Łukasz Borchmann Lukasz Garncarek 23 10 0 10 Sep 2020
Rank over Class: The Untapped Potential of Ranking in Natural Language Processing Amir Atapour-Abarghouei Stephen Bonner A. Mcgough 10 4 0 10 Sep 2020
Investigating Gender Bias in BERT Rishabh Bhardwaj Navonil Majumder Soujanya Poria 33 106 0 10 Sep 2020
Brain2Word: Decoding Brain Activity for Language Generation Nicolas Affolter Béni Egressy Damian Pascual Roger Wattenhofer 11 22 0 10 Sep 2020
Exploiting Multi-Modal Features From Pre-trained Networks for Alzheimer's Dementia Recognition Junghyun Koo Jie Hwan Lee Jaewoo Pyo Yujin Jo Kyogu Lee 27 58 0 09 Sep 2020
kk2018 at SemEval-2020 Task 9: Adversarial Training for Code-Mixing Sentiment Classification Jiaxiang Liu Xuyi Chen Shikun Feng Shuohuan Wang Ouyang Xuan Yu Sun Zhengjie Huang Weiyue Su 30 19 0 08 Sep 2020
Accenture at CheckThat! 2020: If you say so: Post-hoc fact-checking of claims using transformer-based models Evan Williams Paul Rodrigues Valerie Novak 42 42 0 05 Sep 2020
Attention Flows: Analyzing and Comparing Attention Mechanisms in Language Models Joseph F DeRose Jiayao Wang M. Berger 17 83 0 03 Sep 2020
A Primer on Motion Capture with Deep Learning: Principles, Pitfalls and Perspectives Alexander Mathis Steffen Schneider Jessy Lauer Mackenzie W. Mathis 35 165 0 01 Sep 2020
A Framework For Contrastive Self-Supervised Learning And Designing A New Approach William Falcon Kyunghyun Cho SSL 21 103 0 31 Aug 2020
Zero-Resource Knowledge-Grounded Dialogue Generation Linxiao Li Can Xu Wei Wu Yufan Zhao Xueliang Zhao Chongyang Tao 36 70 0 29 Aug 2020
A Survey of Evaluation Metrics Used for NLG Systems Ananya B. Sai Akash Kumar Mohankumar Mitesh M. Khapra ELM 33 230 0 27 Aug 2020
Analysis and Evaluation of Language Models for Word Sense Disambiguation Daniel Loureiro Kiamehr Rezaee Mohammad Taher Pilehvar Jose Camacho-Collados 33 13 0 26 Aug 2020
Multi-Label Sentiment Analysis on 100 Languages with Dynamic Weighting for Label Imbalance Selim F. Yilmaz E. Kaynak Aykut Koç H. Dibeklioğlu Suleyman Serdar Kozat 37 26 0 26 Aug 2020
Conceptualized Representation Learning for Chinese Biomedical Text Mining Ningyu Zhang Qianghuai Jia Kangping Yin Liang Dong Feng Gao Nengwei Hua OOD 39 65 0 25 Aug 2020
How Have We Reacted To The COVID-19 Pandemic? Analyzing Changing Indian Emotions Through The Lens of Twitter Rajdeep Mukherjee S. Poddar Atharva Naik Soham Dasgupta 17 5 0 20 Aug 2020
Language Models as Few-Shot Learner for Task-Oriented Dialogue Systems Andrea Madotto Zihan Liu Zhaojiang Lin Pascale Fung 43 58 0 14 Aug 2020
Hybrid Ranking Network for Text-to-SQL Qin Lyu K. Chakrabarti Shobhit Hathi Souvik Kundu Jianwen Zhang Zheng Chen AIMat 22 83 0 11 Aug 2020
FireBERT: Hardening BERT-based classifiers against adversarial attack Gunnar Mein Kevin Hartman Andrew Morris SILM AAML 16 0 0 10 Aug 2020
KR-BERT: A Small-Scale Korean-Specific Language Model Sangah Lee Hansol Jang Yunmee Baik Suzi Park Hyopil Shin 30 51 0 10 Aug 2020
Distilling the Knowledge of BERT for Sequence-to-Sequence ASR Hayato Futami Hirofumi Inaguma Sei Ueno Masato Mimura S. Sakai Tatsuya Kawahara 24 50 0 09 Aug 2020
ConvBERT: Improving BERT with Span-based Dynamic Convolution Zihang Jiang Weihao Yu Daquan Zhou Yunpeng Chen Jiashi Feng Shuicheng Yan 43 157 0 06 Aug 2020
Aligning AI With Shared Human Values Dan Hendrycks Collin Burns Steven Basart Andrew Critch Jingkai Li D. Song Jacob Steinhardt 63 519 0 05 Aug 2020
Multilingual Translation with Extensible Multilingual Pretraining and Finetuning Y. Tang C. Tran Xian Li Peng-Jen Chen Naman Goyal Vishrav Chaudhary Jiatao Gu Angela Fan CLL 55 447 0 02 Aug 2020
TweepFake: about Detecting Deepfake Tweets T. Fagni Fabrizio Falchi Margherita Gambini Antonio Martella Maurizio Tesconi 21 218 0 31 Jul 2020
MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering Shayne Longpre Yi Lu Joachim Daiber ELM HILM 41 152 0 30 Jul 2020
Public Sentiment Toward Solar Energy: Opinion Mining of Twitter Using a Transformer-Based Language Model Serena Y Kim K. Ganesan P. Dickens S. Panda 28 58 0 27 Jul 2020
Representation Learning with Video Deep InfoMax R. Devon Hjelm Philip Bachman SSL MDE 26 28 0 27 Jul 2020
Reed at SemEval-2020 Task 9: Fine-Tuning and Bag-of-Words Approaches to Code-Mixed Sentiment Analysis Vinay Gopalan Mark Hopkins 33 6 0 26 Jul 2020
Named entity recognition in chemical patents using ensemble of contextual language models J. Copara Nona Naderi J. Knafou Patrick Ruch Douglas Teodoro 27 23 0 24 Jul 2020
FiSSA at SemEval-2020 Task 9: Fine-tuned For Feelings Bertelt Braaksma R. Scholtens Stan van Suijlekom Remy Wang Ahmet Üstün 23 3 0 24 Jul 2020
Clustering of Social Media Messages for Humanitarian Aid Response during Crisis Swati Padhee T. K. Saha Joel R. Tetreault A. Jaimes 22 6 0 23 Jul 2020
FTRANS: Energy-Efficient Acceleration of Transformers using FPGA Bingbing Li Santosh Pandey Haowen Fang Yanjun Lyv Ji Li Jieyang Chen Mimi Xie Lipeng Wan Hang Liu Caiwen Ding AI4CE 16 170 0 16 Jul 2020
Investigating Pretrained Language Models for Graph-to-Text Generation Leonardo F. R. Ribeiro Martin Schmitt Hinrich Schütze Iryna Gurevych 19 215 0 16 Jul 2020
LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning Jian Liu Leyang Cui Hanmeng Liu Dandan Huang Yile Wang Yue Zhang RALM 22 339 0 16 Jul 2020
Fighting the COVID-19 Infodemic in Social Media: A Holistic Perspective and a Call to Arms Firoj Alam Fahim Dalvi Shaden Shaar Nadir Durrani Hamdy Mubarak ... Giovanni Da San Martino Ahmed Abdelali Hassan Sajjad Kareem Darwish Preslav Nakov 22 102 0 15 Jul 2020
Fine-Tune Longformer for Jointly Predicting Rumor Stance and Veracity Anant Khandelwal 21 22 0 15 Jul 2020
Deep learning models for representing out-of-vocabulary words Johannes V. Lochter Renato M. Silva Tiago A. Almeida 18 15 0 14 Jul 2020
An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models Lifu Tu Garima Lalwani Spandana Gella He He LRM 33 184 0 14 Jul 2020
Learning Reasoning Strategies in End-to-End Differentiable Proving Pasquale Minervini Sebastian Riedel Pontus Stenetorp Edward Grefenstette Tim Rocktaschel LRM 45 96 0 13 Jul 2020
TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech Andy T. Liu Shang-Wen Li Hung-yi Lee SSL 62 356 0 12 Jul 2020
Unsupervised Text Generation by Learning from Search Jingjing Li Zichao Li Lili Mou Xin Jiang Michael R. Lyu Irwin King 27 55 0 09 Jul 2020
Learning Speech Representations from Raw Audio by Joint Audiovisual Self-Supervision Abhinav Shukla Stavros Petridis M. Pantic SSL 32 16 0 08 Jul 2020
Robust Prediction of Punctuation and Truecasing for Medical ASR Monica Sunkara S. Ronanki Kalpit Dixit S. Bodapati Katrin Kirchhoff 11 33 0 04 Jul 2020
Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval Lee Xiong Chenyan Xiong Ye Li Kwok-Fung Tang Jialin Liu Paul N. Bennett Junaid Ahmed Arnold Overwijk 16 1,181 0 01 Jul 2020
SemEval-2020 Task 4: Commonsense Validation and Explanation Cunxiang Wang Shuailong Liang Yili Jin Yilong Wang Xiao-Dan Zhu Yue Zhang LRM 25 98 0 01 Jul 2020
Data Movement Is All You Need: A Case Study on Optimizing Transformers A. Ivanov Nikoli Dryden Tal Ben-Nun Shigang Li Torsten Hoefler 36 131 0 30 Jun 2020
Learning Sparse Prototypes for Text Generation Junxian He Taylor Berg-Kirkpatrick Graham Neubig 27 23 0 29 Jun 2020
Improving Sequence Tagging for Vietnamese Text Using Transformer-based Neural Models Viet The Bui Oanh T. K. Tran Hong Phuong Le 25 38 0 29 Jun 2020
Video-Grounded Dialogues with Pretrained Generation Language Models Hung Le Guosheng Lin 34 28 0 27 Jun 2020