RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 4,746 papers shown

Title
Multi-Head Self-Attention with Role-Guided Masks Dongsheng Wang Casper Hansen Lucas Chaves Lima Christian B. Hansen Maria Maistro J. Simonsen Christina Lioma 26 1 0 22 Dec 2020
g2tmn at Constraint@AAAI2021: Exploiting CT-BERT and Ensembling Learning for COVID-19 Fake News Detection Anna Glazkova Maksim Glazkov T. Trifonov 32 58 0 22 Dec 2020
Recognizing Emotion Cause in Conversations Soujanya Poria Navonil Majumder Devamanyu Hazarika Deepanway Ghosal Rishabh Bhardwaj ... Romila Ghosh Abhinaba Roy Niyati Chhaya Alexander Gelbukh Rada Mihalcea 54 124 0 22 Dec 2020
Mention Extraction and Linking for SQL Query Generation Jianqiang Ma Zeyu Yan Shuai Pang Yang Zhang Jianping Shen 37 29 0 18 Dec 2020
Exploring Fluent Query Reformulations with Text-to-Text Transformers and Reinforcement Learning Jerry Zikun Chen S. Yu Haoran Wang 226 5 0 18 Dec 2020
Can Transformers Reason About Effects of Actions? Pratyay Banerjee Chitta Baral Man Luo Arindam Mitra Kuntal Kumar Pal Tran Cao Son Neeraj Varshney LRM AI4CE 29 10 0 17 Dec 2020
Transformer Interpretability Beyond Attention Visualization Hila Chefer Shir Gur Lior Wolf 45 645 0 17 Dec 2020
BERT Goes Shopping: Comparing Distributional Models for Product Representations Federico Bianchi Bingqing Yu Jacopo Tagliabue 28 15 0 17 Dec 2020
Assessing COVID-19 Impacts on College Students via Automated Processing of Free-form Text Ravi Sharma Srivyshnavi Pagadala Pratool Bharti Sriram Chellappan Trine Schmidt Raj Goyal 26 7 0 17 Dec 2020
MELINDA: A Multimodal Dataset for Biomedical Experiment Method Classification Te-Lin Wu Shikhar Singh S. Paul Gully A. Burns Nanyun Peng 30 18 0 16 Dec 2020
LIREx: Augmenting Language Inference with Relevant Explanation Xinyan Zhao V. Vydiswaran LRM 33 38 0 16 Dec 2020
DialogXL: All-in-One XLNet for Multi-Party Conversation Emotion Recognition Weizhou Shen Junqing Chen Xiaojun Quan Zhixiang Xie 22 201 0 16 Dec 2020
Objective-Based Hierarchical Clustering of Deep Embedding Vectors Stanislav A. Naumov G. Yaroslavtsev Dmitrii Avdiukhin 22 23 0 15 Dec 2020
*-CFQ: Analyzing the Scalability of Machine Learning on a Compositional Task Dmitry Tsarkov Tibor Tihon Nathan Scales Nikola Momchev Danila Sinopalnikov Nathanael Scharli 18 17 0 15 Dec 2020
Learning to Rationalize for Nonmonotonic Reasoning with Distant Supervision Faeze Brahman Vered Shwartz Rachel Rudinger Yejin Choi LRM 15 42 0 14 Dec 2020
Parameter-Efficient Transfer Learning with Diff Pruning Demi Guo Alexander M. Rush Yoon Kim 13 386 0 14 Dec 2020
Yelp Review Rating Prediction: Machine Learning and Deep Learning Models Zefang Liu VLM 16 15 0 12 Dec 2020
Morphology Matters: A Multilingual Language Modeling Analysis Hyunji Hayley Park Katherine J. Zhang Coleman Haley K. Steimel Han Liu Lane Schwartz 53 47 0 11 Dec 2020
Reinforced Multi-Teacher Selection for Knowledge Distillation Fei Yuan Linjun Shou J. Pei Wutao Lin Ming Gong Yan Fu Daxin Jiang 15 121 0 11 Dec 2020
Infusing Finetuning with Semantic Dependencies Zhaofeng Wu Hao Peng Noah A. Smith 25 37 0 10 Dec 2020
Discourse Parsing of Contentious, Non-Convergent Online Discussions S. Zakharov Omri Hadar Tovit Hakak Dina Grossman Y. Kolikant Oren Tsur 27 8 0 08 Dec 2020
Benchmarking Commercial Intent Detection Services with Practice-Driven Evaluations Haode Qi Lin Pan Atin Sood Abhishek Shah L. Kunc Mo Yu Saloni Potdar VLM 33 16 0 07 Dec 2020
KgPLM: Knowledge-guided Language Model Pre-training via Generative and Discriminative Learning Bin He Xin Jiang Jinghui Xiao Qun Liu 37 18 0 07 Dec 2020
People Still Care About Facts: Twitter Users Engage More with Factual Discourse than Misinformation--A Comparison Between COVID and General Narratives on Twitter Mirela Silva Fabrício Ceschin P. Shrestha Christopher Brant Shlok Gilda Juliana Fernandes Catia S. Silva André Grégio Daniela Oliveira Luiz H. F. Giovanini 31 1 0 03 Dec 2020
Self-Explaining Structures Improve NLP Models Zijun Sun Chun Fan Qinghong Han Xiaofei Sun Yuxian Meng Fei Wu Jiwei Li MILM XAI LRM FAtt 46 38 0 03 Dec 2020
End-to-End QA on COVID-19: Domain Adaptation with Synthetic Training R. Reddy Bhavani Iyer Md Arafat Sultan Rong Zhang Avirup Sil Vittorio Castelli Radu Florian Salim Roukos OOD 20 19 0 02 Dec 2020
Depression Status Estimation by Deep Learning based Hybrid Multi-Modal Fusion Model H. Shalu P. Harikrishnan C. Harisankar Akash Das Saptarshi Majumder Arnhav Datar MS SubinMathew Anugyan Das Juned Kadiwala 13 4 0 30 Nov 2020
A Novel Sentiment Analysis Engine for Preliminary Depression Status Estimation on Social Media S. Suman H. Shalu Lakshya A Agrawal Archit Agrawal Juned Kadiwala 21 6 0 29 Nov 2020
Progressively Stacking 2.0: A Multi-stage Layerwise Training Method for BERT Training Speedup Cheng Yang Shengnan Wang Chao Yang Yuechuan Li Ru He Jingqiao Zhang 32 25 0 27 Nov 2020
GLGE: A New General Language Generation Evaluation Benchmark Dayiheng Liu Yu Yan Yeyun Gong Weizhen Qi Hang Zhang ... Jiancheng Lv Ruofei Zhang Winnie Wu Ming Zhou Nan Duan ELM 40 66 0 24 Nov 2020
Detection and Classification of mental illnesses on social media using RoBERTa Ankit Murarka Balaji Radhakrishnan S. Ravichandran AI4MH 22 45 0 23 Nov 2020
Evaluating Semantic Accuracy of Data-to-Text Generation with Natural Language Inference Ondrej Dusek Zdeněk Kasner 30 66 0 21 Nov 2020
Persuasive Dialogue Understanding: the Baselines and Negative Results Hui Chen Deepanway Ghosal Navonil Majumder Amir Hussain Soujanya Poria 29 8 0 19 Nov 2020
EasyTransfer -- A Simple and Scalable Deep Transfer Learning Platform for NLP Applications Minghui Qiu Peng Li Chengyu Wang Hanjie Pan Yaliang Li ... Jun Yang Yaliang Li Jun Huang Deng Cai Wei Lin VLM SyDa 39 20 0 18 Nov 2020
WikiAsp: A Dataset for Multi-domain Aspect-based Summarization Hiroaki Hayashi Prashant Budania Peng Wang Chris Ackerson Raj Neervannan Graham Neubig 34 64 0 16 Nov 2020
DORB: Dynamically Optimizing Multiple Rewards with Bandits Ramakanth Pasunuru Han Guo Joey Tianyi Zhou OffRL 32 6 0 15 Nov 2020
FLERT: Document-Level Features for Named Entity Recognition Stefan Schweter Alan Akbik 27 111 0 13 Nov 2020
Hurricane Forecasting: A Novel Multimodal Machine Learning Framework L. Boussioux C. Zeng Théo Guénais Dimitris Bertsimas 16 38 0 11 Nov 2020
CalibreNet: Calibration Networks for Multilingual Sequence Labeling Shining Liang Linjun Shou J. Pei Ming Gong Wanli Zuo Daxin Jiang 31 8 0 11 Nov 2020
NIT COVID-19 at WNUT-2020 Task 2: Deep Learning Model RoBERTa for Identify Informative COVID-19 English Tweets Jagadeesh M S Alphonse P J A 11 12 0 11 Nov 2020
Multimodal Pretraining for Dense Video Captioning Gabriel Huang Bo Pang Zhenhai Zhu Clara E. Rivera Radu Soricut 21 81 0 10 Nov 2020
When Do You Need Billions of Words of Pretraining Data? Yian Zhang Alex Warstadt Haau-Sing Li Samuel R. Bowman 29 136 0 10 Nov 2020
Multi-document Summarization via Deep Learning Techniques: A Survey Congbo Ma W. Zhang Mingyu Guo Hu Wang Quan Z. Sheng 13 126 0 10 Nov 2020
Adversarial Semantic Collisions Congzheng Song Alexander M. Rush Vitaly Shmatikov AAML 22 52 0 09 Nov 2020
Bangla Text Classification using Transformers Tanvirul Alam A. Khan Firoj Alam 31 34 0 09 Nov 2020
Towards Domain-Agnostic Contrastive Learning Vikas Verma Minh-Thang Luong Kenji Kawaguchi Hieu H. Pham Quoc V. Le SSL 15 116 0 09 Nov 2020
Positional Artefacts Propagate Through Masked Language Model Embeddings Ziyang Luo Artur Kulmizev Xiaoxi Mao 29 41 0 09 Nov 2020
Know What You Don't Need: Single-Shot Meta-Pruning for Attention Heads Zhengyan Zhang Fanchao Qi Zhiyuan Liu Qun Liu Maosong Sun VLM 46 30 0 07 Nov 2020
Underspecification Presents Challenges for Credibility in Modern Machine Learning Alexander DÁmour Katherine A. Heller D. Moldovan Ben Adlam B. Alipanahi ... Kellie Webster Steve Yadlowsky T. Yun Xiaohua Zhai D. Sculley OffRL 77 671 0 06 Nov 2020
Improving Machine Reading Comprehension with Single-choice Decision and Transfer Learning Yufan Jiang Shuangzhi Wu Jing Gong Yahui Cheng Peng Meng Weiliang Lin Zhibo Chen Mu Li FaML 24 11 0 06 Nov 2020