RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 4,752 papers shown

Title
K-PLUG: Knowledge-injected Pre-trained Language Model for Natural Language Understanding and Generation in E-Commerce Song Xu Haoran Li Peng Yuan Yujia Wang Youzheng Wu Xiaodong He Ying Liu Bowen Zhou KELM 38 24 0 14 Apr 2021
Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little Koustuv Sinha Robin Jia Dieuwke Hupkes J. Pineau Adina Williams Douwe Kiela 45 245 0 14 Apr 2021
AR-LSAT: Investigating Analytical Reasoning of Text Wanjun Zhong Siyuan Wang Duyu Tang Zenan Xu Daya Guo Jiahai Wang Jian Yin Ming Zhou Nan Duan ELM 27 41 0 14 Apr 2021
MultiModalQA: Complex Question Answering over Text, Tables and Images Alon Talmor Ori Yoran Amnon Catav Dan Lahav Yizhong Wang Akari Asai Gabriel Ilharco Hannaneh Hajishirzi Jonathan Berant LMTD 32 150 0 13 Apr 2021
CLEVR_HYP: A Challenge Dataset and Baselines for Visual Question Answering with Hypothetical Actions over Images Shailaja Keyur Sampat Akshay Kumar Yezhou Yang Chitta Baral 29 26 0 13 Apr 2021
"Subverting the Jewtocracy": Online Antisemitism Detection Using Multimodal Deep Learning Mohit Chandra D. Pailla Himanshu Bhatia AadilMehdi J. Sanchawala Manish Gupta Manish Shrivastava Ponnurangam Kumaraguru 27 38 0 13 Apr 2021
Semantic maps and metrics for science Semantic maps and metrics for science using deep transformer encoders Brendan Chambers James A. Evans MedIm 13 0 0 13 Apr 2021
DirectProbe: Studying Representations without Classifiers Yichu Zhou Vivek Srikumar 34 27 0 13 Apr 2021
Evaluating Pre-Trained Models for User Feedback Analysis in Software Engineering: A Study on Classification of App-Reviews M. Hadi Fatemeh H. Fard 26 30 0 12 Apr 2021
Relational World Knowledge Representation in Contextual Language Models: A Review Tara Safavi Danai Koutra KELM 43 51 0 12 Apr 2021
Fighting the COVID-19 Infodemic with a Holistic BERT Ensemble Georgios Tziafas Konstantinos Kogkalidis Tommaso Caselli 34 9 0 12 Apr 2021
Escaping the Big Data Paradigm with Compact Transformers Ali Hassani Steven Walton Nikhil Shah Abulikemu Abuduweili Jiachen Li Humphrey Shi 79 462 0 12 Apr 2021
FUDGE: Controlled Text Generation With Future Discriminators Kevin Kaichuang Yang Dan Klein 45 315 0 12 Apr 2021
UniDrop: A Simple yet Effective Technique to Improve Transformer without Extra Cost Zhen Wu Lijun Wu Qi Meng Yingce Xia Shufang Xie Tao Qin Xinyu Dai Tie-Yan Liu 18 22 0 11 Apr 2021
Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach Simiao Zuo Chen Liang Haoming Jiang Xiaodong Liu Pengcheng He Jianfeng Gao Weizhu Chen T. Zhao 68 9 0 11 Apr 2021
NLI Data Sanity Check: Assessing the Effect of Data Corruption on Model Performance Aarne Talman Marianna Apidianaki S. Chatzikyriakidis Jörg Tiedemann 33 10 0 10 Apr 2021
Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections Ruiqi Zhong Kristy Lee Zheng-Wei Zhang Dan Klein 44 166 0 10 Apr 2021
UPB at SemEval-2021 Task 8: Extracting Semantic Information on Measurements as Multi-Turn Question Answering Andrei-Marius Avram George-Eduard Zaharia Dumitru-Clementin Cercel M. Dascalu 27 2 0 09 Apr 2021
Deep Indexed Active Learning for Matching Heterogeneous Entity Representations Arjit Jain Sunita Sarawagi Prithviraj Sen 30 24 0 08 Apr 2021
COVID-19 Named Entity Recognition for Vietnamese Thinh Hung Truong M. Dao Dat Quoc Nguyen 29 38 0 08 Apr 2021
Layer Reduction: Accelerating Conformer-Based Self-Supervised Model via Layer Consistency Jinchuan Tian Rongzhi Gu Helin Wang Yuexian Zou 26 0 0 08 Apr 2021
Dynabench: Rethinking Benchmarking in NLP Douwe Kiela Max Bartolo Yixin Nie Divyansh Kaushik Atticus Geiger ... Pontus Stenetorp Robin Jia Joey Tianyi Zhou Christopher Potts Adina Williams 29 392 0 07 Apr 2021
HumAID: Human-Annotated Disaster Incidents Data from Twitter with Deep Learning Benchmarks Firoj Alam U. Qazi Muhammad Imran Ferda Ofli 25 65 0 07 Apr 2021
Personalized Entity Resolution with Dynamic Heterogeneous Knowledge Graph Representations Ying Lin H. Wang Jiangning Chen Tong Wang Yue Liu Heng Ji Yang Liu Premkumar Natarajan 26 10 0 06 Apr 2021
What Will it Take to Fix Benchmarking in Natural Language Understanding? Samuel R. Bowman George E. Dahl ELM ALM 30 157 0 05 Apr 2021
Semantic Distance: A New Metric for ASR Performance Analysis Towards Spoken Language Understanding Suyoun Kim Abhinav Arora Duc Le Ching-Feng Yeh Christian Fuegen Ozlem Kalinli M. Seltzer 30 25 0 05 Apr 2021
Intent Detection and Slot Filling for Vietnamese M. Dao Thinh Hung Truong Dat Quoc Nguyen VLM 37 35 0 05 Apr 2021
A Heuristic-driven Uncertainty based Ensemble Framework for Fake News Detection in Tweets and News Articles Sourya Dipta Das Ayan Basak S. Dutta 47 47 0 05 Apr 2021
BBAEG: Towards BERT-based Biomedical Adversarial Example Generation for Text Classification Ishani Mondal AAML SILM MedIm 19 14 0 05 Apr 2021
A New Approach to Overgenerating and Scoring Abstractive Summaries Kaiqiang Song Bingqing Wang Z. Feng Fei Liu 22 17 0 05 Apr 2021
Inference Time Style Control for Summarization Shuyang Cao Lu Wang AI4TS 32 15 0 05 Apr 2021
Recommending Metamodel Concepts during Modeling Activities with Pre-Trained Language Models Martin Weyssow H. Sahraoui Eugene Syriani 18 50 0 04 Apr 2021
Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems Derek Chen Howard Chen Yi Yang A. Lin Zhou Yu 25 65 0 01 Apr 2021
Group-Free 3D Object Detection via Transformers Ze Liu Zheng-Wei Zhang Yue Cao Han Hu Xin Tong ViT 3DPC 50 310 0 01 Apr 2021
Going deeper with Image Transformers Hugo Touvron Matthieu Cord Alexandre Sablayrolles Gabriel Synnaeve Hervé Jégou ViT 27 990 0 31 Mar 2021
Deep Neural Approaches to Relation Triplets Extraction: A Comprehensive Survey Tapas Nayak Navonil Majumder Pawan Goyal Soujanya Poria ViT 21 49 0 31 Mar 2021
BASE Layers: Simplifying Training of Large, Sparse Models M. Lewis Shruti Bhosale Tim Dettmers Naman Goyal Luke Zettlemoyer MoE 53 274 0 30 Mar 2021
Retraining DistilBERT for a Voice Shopping Assistant by Using Universal Dependencies P. Jayarao Arpit Sharma 21 2 0 29 Mar 2021
On the Adversarial Robustness of Vision Transformers Rulin Shao Zhouxing Shi Jinfeng Yi Pin-Yu Chen Cho-Jui Hsieh ViT 37 138 0 29 Mar 2021
Efficient Explanations from Empirical Explainers Robert Schwarzenberg Nils Feldhus Sebastian Möller FAtt 37 9 0 29 Mar 2021
Machine Learning Meets Natural Language Processing -- The story so far N. Galanis P. Vafiadis K.-G. Mirzaev G. Papakostas 43 7 0 27 Mar 2021
Bertinho: Galician BERT Representations David Vilares Marcos Garcia Carlos Gómez-Rodríguez 70 22 0 25 Mar 2021
Vision Transformers for Dense Prediction René Ranftl Alexey Bochkovskiy V. Koltun ViT MDE 45 1,670 0 24 Mar 2021
FastMoE: A Fast Mixture-of-Expert Training System Jiaao He J. Qiu Aohan Zeng Zhilin Yang Jidong Zhai Jie Tang ALM MoE 45 94 0 24 Mar 2021
Finetuning Pretrained Transformers into RNNs Jungo Kasai Hao Peng Yizhe Zhang Dani Yogatama Gabriel Ilharco Nikolaos Pappas Yi Mao Weizhu Chen Noah A. Smith 46 63 0 24 Mar 2021
Thinking Aloud: Dynamic Context Generation Improves Zero-Shot Reasoning Performance of GPT-2 Gregor Betz Kyle Richardson Christian Voigt ReLM LRM 24 30 0 24 Mar 2021
Czert -- Czech BERT-like Model for Language Representation Jakub Sido O. Pražák P. Pribán Jan Pasek Michal Seják Miloslav Konopík 31 43 0 24 Mar 2021
UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmark Nicholas Lourie Ronan Le Bras Chandra Bhagavatula Yejin Choi LRM 30 137 0 24 Mar 2021
The NLP Cookbook: Modern Recipes for Transformer based Deep Learning Architectures Sushant Singh A. Mahmood AI4TS 60 94 0 23 Mar 2021
A Pseudo-Metric between Probability Distributions based on Depth-Trimmed Regions Guillaume Staerman Pavlo Mozharovskyi Pierre Colombo Stéphan Clémenccon Florence dÁlché-Buc OOD 69 17 0 23 Mar 2021