RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 4,752 papers shown

Title
Multilingual Answer Sentence Reranking via Automatically Translated Data Thuy Vu Alessandro Moschitti 30 5 0 20 Feb 2021
Entity Structure Within and Throughout: Modeling Mention Dependencies for Document-Level Relation Extraction Benfeng Xu Quan Wang Yajuan Lyu Yong Zhu Zhendong Mao 27 166 0 20 Feb 2021
Scaling up DNA digital data storage by efficiently predicting DNA hybridisation using deep learning David Buterez 19 7 0 19 Feb 2021
Towards Emotion Recognition in Hindi-English Code-Mixed Data: A Transformer Based Approach Anshul Wadhawan Akshita Aggarwal 22 29 0 19 Feb 2021
MUDES: Multilingual Detection of Offensive Spans Tharindu Ranasinghe Marcos Zampieri 27 41 0 18 Feb 2021
Training Large-Scale News Recommenders with Pretrained Language Models in the Loop Shitao Xiao Zheng Liu Yingxia Shao Tao Di Xing Xie VLM AIFin 127 41 0 18 Feb 2021
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 317 1,086 0 17 Feb 2021
Towards generalisable hate speech detection: a review on obstacles and solutions Wenjie Yin A. Zubiaga 117 164 0 17 Feb 2021
Open-Retrieval Conversational Machine Reading Yifan Gao Jingjing Li Chien-Sheng Wu M. Lyu Irwin King 51 17 0 17 Feb 2021
COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining Yu Meng Chenyan Xiong Payal Bajaj Saurabh Tiwary Paul N. Bennett Jiawei Han Xia Song 127 203 0 16 Feb 2021
Boosting Low-Resource Biomedical QA via Entity-Aware Masking Strategies Gabriele Pergola E. Kochkina Lin Gui Maria Liakata Yulan He 88 31 0 16 Feb 2021
NoiseQA: Challenge Set Evaluation for User-Centric Question Answering Abhilasha Ravichander Siddharth Dalmia Maria Ryskina Florian Metze Eduard H. Hovy A. Black ELM 29 32 0 16 Feb 2021
GradInit: Learning to Initialize Neural Networks for Stable and Efficient Training Chen Zhu Renkun Ni Zheng Xu Kezhi Kong Wenjie Huang Tom Goldstein ODL 41 54 0 16 Feb 2021
Exploring Transformers in Natural Language Generation: GPT, BERT, and XLNet M. O. Topal Anil Bas Imke van Heerden LLMAG AI4CE 26 88 0 16 Feb 2021
DOBF: A Deobfuscation Pre-Training Objective for Programming Languages Baptiste Roziere Marie-Anne Lachaux Marc Szafraniec Guillaume Lample AI4CE 52 138 0 15 Feb 2021
indicnlp@kgp at DravidianLangTech-EACL2021: Offensive Language Identification in Dravidian Languages Kushal Kedia Abhilash Nandy 29 23 0 14 Feb 2021
CATE: Computation-aware Neural Architecture Encoding with Transformers Shen Yan Kaiqiang Song Z. Feng Mi Zhang 27 24 0 14 Feb 2021
PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them Patrick Lewis Yuxiang Wu Linqing Liu Pasquale Minervini Heinrich Küttler Aleksandra Piktus Pontus Stenetorp Sebastian Riedel RALM 42 229 0 13 Feb 2021
Optimizing Inference Performance of Transformers on CPUs D. Dice Alex Kogan 19 15 0 12 Feb 2021
Neural Inverse Text Normalization Monica Sunkara Chaitanya P. Shivade S. Bodapati Katrin Kirchhoff 49 31 0 12 Feb 2021
Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling Jie Lei Linjie Li Luowei Zhou Zhe Gan Tamara L. Berg Joey Tianyi Zhou Jingjing Liu CLIP 46 648 0 11 Feb 2021
Multi-Task Reinforcement Learning with Context-based Representations Shagun Sodhani Amy Zhang Joelle Pineau 34 182 0 11 Feb 2021
Multi-turn Dialogue Reading Comprehension with Pivot Turns and Knowledge Zhuosheng Zhang Junlong Li Hai Zhao 40 23 0 10 Feb 2021
Biomedical Question Answering: A Survey of Approaches and Challenges Qiao Jin Zheng Yuan Guangzhi Xiong Qian Yu Huaiyuan Ying Chuanqi Tan Mosha Chen Songfang Huang Xiaozhong Liu Sheng Yu 34 96 0 10 Feb 2021
SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networks Bahare Fatemi Layla El Asri Seyed Mehran Kazemi GNN SSL 22 160 0 09 Feb 2021
Unifying Vision-and-Language Tasks via Text Generation Jaemin Cho Jie Lei Hao Tan Joey Tianyi Zhou MLLM 277 525 0 04 Feb 2021
When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data Peter Hase Joey Tianyi Zhou XAI 30 87 0 03 Feb 2021
Mind the Gap: Assessing Temporal Generalization in Neural Language Models Angeliki Lazaridou A. Kuncoro E. Gribovskaya Devang Agrawal Adam Liska ... Sebastian Ruder Dani Yogatama Kris Cao Susannah Young Phil Blunsom VLM 43 207 0 03 Feb 2021
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics Sebastian Gehrmann Tosin Adewumi Karmanya Aggarwal Pawan Sasanka Ammanamanchi Aremu Anuoluwapo ... Nishant Subramani Wei Xu Diyi Yang Akhila Yerukola Jiawei Zhou VLM 260 285 0 02 Feb 2021
MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers Krishna Pillutla Swabha Swayamdipta Rowan Zellers John Thickstun Sean Welleck Yejin Choi Zaïd Harchaoui 48 343 0 02 Feb 2021
Measuring and Improving Consistency in Pretrained Language Models Yanai Elazar Nora Kassner Shauli Ravfogel Abhilasha Ravichander Eduard H. Hovy Hinrich Schütze Yoav Goldberg HILM 272 347 0 01 Feb 2021
Speech Recognition by Simply Fine-tuning BERT Wen-Chin Huang Chia-Hua Wu Shang-Bao Luo Kuan-Yu Chen Hsin-Min Wang Tomoki Toda 74 28 0 30 Jan 2021
CD2CR: Co-reference Resolution Across Documents and Domains James Ravenscroft Arie Cattan A. Clare Ido Dagan Maria Liakata 90 8 0 29 Jan 2021
A transformer based approach for fighting COVID-19 fake news S. M. S. Shifath Mohammad Faiyaz Khan Md. Saiful Islam MedIm 34 23 0 28 Jan 2021
Identifying COVID-19 Fake News in Social Media Tathagata Raha Vijayasaradhi Indurthi Aayush Upadhyaya Jeevesh Kataria Pramud Bommakanti Vikram Keswani Vasudeva Varma GNN MedIm 25 12 0 28 Jan 2021
Bottleneck Transformers for Visual Recognition A. Srinivas Nayeon Lee Niki Parmar Jonathon Shlens Pieter Abbeel Ashish Vaswani SLR 290 981 0 27 Jan 2021
Muppet: Massive Multi-task Representations with Pre-Finetuning Armen Aghajanyan Anchit Gupta Akshat Shrivastava Xilun Chen Luke Zettlemoyer Sonal Gupta 33 266 0 26 Jan 2021
A Case Study of Deep Learning Based Multi-Modal Methods for Predicting the Age-Suitability Rating of Movie Trailers Mahsa Shafaei C. Smailis I. Kakadiaris Thamar Solorio 215 1 0 26 Jan 2021
Debiasing Pre-trained Contextualised Embeddings Masahiro Kaneko Danushka Bollegala 218 137 0 23 Jan 2021
PalmTree: Learning an Assembly Language Model for Instruction Embedding Xuezixiang Li Qu Yu Heng Yin 24 144 0 21 Jan 2021
Word Alignment by Fine-tuning Embeddings on Parallel Corpora Zi-Yi Dou Graham Neubig 100 258 0 20 Jan 2021
Open-Domain Conversational Search Assistant with Transformers Rafael Ferreira Mariana Leite David Semedo João Magalhães 18 11 0 20 Jan 2021
Learning to Augment for Data-Scarce Domain BERT Knowledge Distillation Lingyun Feng Minghui Qiu Yaliang Li Haitao Zheng Ying Shen 46 10 0 20 Jan 2021
WeChat AI & ICT's Submission for DSTC9 Interactive Dialogue Evaluation Track Zekang Li Zongjia Li Jinchao Zhang Yang Feng Jie Zhou 24 1 0 20 Jan 2021
Towards Facilitating Empathic Conversations in Online Mental Health Support: A Reinforcement Learning Approach Ashish Sharma Inna Wanyin Lin Adam S. Miner David C. Atkins Tim Althoff AI4MH 30 140 0 19 Jan 2021
Transformer-Based Models for Question Answering on COVID19 Hillary Ngai Yoona Park John Chen Mahboobeh Parsapoor OOD 27 21 0 16 Jan 2021
Match-Ignition: Plugging PageRank into Transformer for Long-form Text Matching Liang Pang Yanyan Lan Xueqi Cheng 24 19 0 16 Jan 2021
Grid Search Hyperparameter Benchmarking of BERT, ALBERT, and LongFormer on DuoRC Alex John Quijano Sam Nguyen Juanita Ordoñez 29 7 0 15 Jan 2021
TextGNN: Improving Text Encoder via Graph Neural Network in Sponsored Search Jason Zhu Yanling Cui Yuming Liu Hao Sun Xue Li Markus Pelger Tianqi Yan Liangjie Zhang Ruofei Zhang Huasha Zhao AI4CE 72 75 0 15 Jan 2021
LIME: Learning Inductive Bias for Primitives of Mathematical Reasoning Yuhuai Wu M. Rabe Wenda Li Jimmy Ba Roger C. Grosse Christian Szegedy AIMat LRM 82 53 0 15 Jan 2021