RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 4,659 papers shown

Title
Cold-start Active Learning through Self-supervised Language Modeling Michelle Yuan Hsuan-Tien Lin Jordan L. Boyd-Graber 116 180 0 19 Oct 2020
Towards Interpreting BERT for Reading Comprehension Based QA Sahana Ramnath Preksha Nema Deep Sahni Mitesh M. Khapra 42 30 0 18 Oct 2020
HABERTOR: An Efficient and Effective Deep Hatespeech Detector T. Tran Yifan Hu Changwei Hu Kevin Yen Fei Tan Kyumin Lee Serim Park VLM 34 32 0 17 Oct 2020
TweetBERT: A Pretrained Language Representation Model for Twitter Text Analysis Mohiuddin Md Abdul Qudar Vijay K. Mago SSeg 28 35 0 17 Oct 2020
Mischief: A Simple Black-Box Attack Against Transformer Architectures Adrian de Wynter AAML 42 1 0 16 Oct 2020
WNUT-2020 Task 2: Identification of Informative COVID-19 English Tweets Dat Quoc Nguyen Thanh Tien Vu A. Rahimi M. Dao L. T. Nguyen Long Doan 17 74 0 16 Oct 2020
RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering Yingqi Qu Yuchen Ding Jing Liu Kai Liu Ruiyang Ren Xin Zhao Daxiang Dong Hua Wu Haifeng Wang RALM OffRL 214 595 0 16 Oct 2020
What is More Likely to Happen Next? Video-and-Language Future Event Prediction Jie Lei Licheng Yu Tamara L. Berg Joey Tianyi Zhou 33 72 0 15 Oct 2020
TopicBERT for Energy Efficient Document Classification Yatin Chaudhary Pankaj Gupta Khushbu Saxena Vivek Kulkarni Thomas Runkler Hinrich Schütze 10 21 0 15 Oct 2020
Text Classification Using Label Names Only: A Language Model Self-Training Approach Yu Meng Yunyi Zhang Jiaxin Huang Chenyan Xiong Heng Ji Chao Zhang Jiawei Han VLM 55 75 0 14 Oct 2020
Neural Databases James Thorne Majid Yazdani Marzieh Saeidi Fabrizio Silvestri Sebastian Riedel A. Halevy NAI 34 9 0 14 Oct 2020
Pretrained Transformers for Text Ranking: BERT and Beyond Jimmy J. Lin Rodrigo Nogueira Andrew Yates VLM 244 612 0 13 Oct 2020
CAPT: Contrastive Pre-Training for Learning Denoised Sequence Representations Fuli Luo Pengcheng Yang Shicheng Li Xuancheng Ren Xu Sun VLM SSL 21 16 0 13 Oct 2020
Humane Visual AI: Telling the Stories Behind a Medical Condition Wonyoung So Edyta P. Bogucka S. Šćepanović Sagar Joglekar Ke Zhou Daniele Quercia 14 13 0 13 Oct 2020
X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models Zhengbao Jiang Antonios Anastasopoulos Jun Araki Haibo Ding Graham Neubig HILM KELM 21 138 0 13 Oct 2020
Are Some Words Worth More than Others? Shiran Dudy Steven Bedrick 18 14 0 12 Oct 2020
Zero-shot Entity Linking with Efficient Long Range Sequence Modeling Zonghai Yao Liangliang Cao Huapu Pan VLM 31 21 0 12 Oct 2020
BioMegatron: Larger Biomedical Domain Language Model Hoo-Chang Shin Yang Zhang Evelina Bakhturina Raul Puri M. Patwary M. Shoeybi Raghav Mani AI4CE 27 144 0 12 Oct 2020
Webly Supervised Image Classification with Metadata: Automatic Noisy Label Correction via Visual-Semantic Graph Jingkang Yang Weirong Chen Xue Jiang Xiaopeng Yan Huabin Zheng Wayne Zhang NoLa 33 13 0 12 Oct 2020
Probing Pretrained Language Models for Lexical Semantics Ivan Vulić Edoardo Ponti Robert Litschko Goran Glavaš Anna Korhonen KELM 33 233 0 12 Oct 2020
Reformulating Unsupervised Style Transfer as Paraphrase Generation Kalpesh Krishna John Wieting Mohit Iyyer 30 238 0 12 Oct 2020
Counterfactual Variable Control for Robust and Interpretable Question Answering S. Yu Yulei Niu Shuohang Wang Jing Jiang Qianru Sun AAML OOD 44 9 0 12 Oct 2020
Neural, Symbolic and Neural-Symbolic Reasoning on Knowledge Graphs Jing Zhang Bo Chen Lingxi Zhang Xirui Ke Haipeng Ding NAI 40 3 0 12 Oct 2020
A BERT-based Distractor Generation Scheme with Multi-tasking and Negative Answer Training Strategies Ho-Lam Chung Ying-Hong Chan Yao-Chung Fan 39 41 0 12 Oct 2020
Quantitative Argument Summarization and Beyond: Cross-Domain Key Point Analysis Roy Bar-Haim Yoav Kantor Lilach Eden Roni Friedman Dan Lahav Noam Slonim 34 43 0 11 Oct 2020
Neural Machine Translation Doesn't Translate Gender Coreference Right Unless You Make It Danielle Saunders Rosie Sallis Bill Byrne 27 63 0 11 Oct 2020
SMYRF: Efficient Attention using Asymmetric Clustering Giannis Daras Nikita Kitaev Augustus Odena A. Dimakis 31 44 0 11 Oct 2020
SJTU-NICT's Supervised and Unsupervised Neural Machine Translation Systems for the WMT20 News Translation Task Z. Li Hai Zhao Rui Wang Kehai Chen Masao Utiyama Eiichiro Sumita 36 15 0 11 Oct 2020
On the Importance of Adaptive Data Collection for Extremely Imbalanced Pairwise Tasks Stephen Mussmann Robin Jia Percy Liang 29 15 0 10 Oct 2020
Automated Concatenation of Embeddings for Structured Prediction Xinyu Wang Yong-jia Jiang Nguyen Bach Tao Wang Zhongqiang Huang Fei Huang Kewei Tu 35 172 0 10 Oct 2020
What Do Position Embeddings Learn? An Empirical Study of Pre-Trained Language Model Positional Encoding Yu-An Wang Yun-Nung Chen SSL 12 94 0 10 Oct 2020
Counterfactually-Augmented SNLI Training Data Does Not Yield Better Generalization Than Unaugmented Data William Huang Haokun Liu Samuel R. Bowman 24 37 0 09 Oct 2020
Denoising Multi-Source Weak Supervision for Neural Text Classification Wendi Ren Yinghao Li Hanting Su David Kartchner Cassie S. Mitchell Chao Zhang NoLa 36 70 0 09 Oct 2020
Precise Task Formalization Matters in Winograd Schema Evaluations Haokun Liu William Huang Dhara Mungra Samuel R. Bowman ReLM 22 12 0 08 Oct 2020
Two are Better than One: Joint Entity and Relation Extraction with Table-Sequence Encoders Jue Wang Wei Lu 26 225 0 08 Oct 2020
Infusing Disease Knowledge into BERT for Health Question Answering, Medical Inference and Disease Name Recognition Yun He Ziwei Zhu Yin Zhang Qin Chen James Caverlee AI4MH 36 108 0 08 Oct 2020
Exposing Shallow Heuristics of Relation Extraction Models with Challenge Data Shachar Rosenman Alon Jacovi Yoav Goldberg 16 28 0 07 Oct 2020
A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks Nikunj Saunshi Sadhika Malladi Sanjeev Arora 31 87 0 07 Oct 2020
Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsing Xilun Chen Asish Ghoshal Yashar Mehdad Luke Zettlemoyer S. Gupta 41 89 0 07 Oct 2020
What Can We Learn from Collective Human Opinions on Natural Language Inference Data? Yixin Nie Xiang Zhou Joey Tianyi Zhou 29 129 0 07 Oct 2020
Why do you think that? Exploring Faithful Sentence-Level Rationales Without Supervision Max Glockner Ivan Habernal Iryna Gurevych LRM 27 25 0 07 Oct 2020
Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction M. Chen Tao Ge Xingxing Zhang Furu Wei M. Zhou 27 46 0 07 Oct 2020
Like hiking? You probably enjoy nature: Persona-grounded Dialog with Commonsense Expansions Bodhisattwa Prasad Majumder Harsh Jhamtani Taylor Berg-Kirkpatrick Julian McAuley 30 85 0 07 Oct 2020
Program Enhanced Fact Verification with Verbalization and Graph Attention Network Xiaoyu Yang Feng Nie Yufei Feng Quan Liu Zhigang Chen Xiao-Dan Zhu 26 52 0 06 Oct 2020
PRover: Proof Generation for Interpretable Reasoning over Rules Swarnadeep Saha Sayan Ghosh Shashank Srivastava Joey Tianyi Zhou ReLM LRM 34 77 0 06 Oct 2020
Poison Attacks against Text Datasets with Conditional Adversarially Regularized Autoencoder Alvin Chan Yi Tay Yew-Soon Ong Aston Zhang SILM 23 56 0 06 Oct 2020
InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective Wei Ping Shuohang Wang Yu Cheng Zhe Gan R. Jia Bo-wen Li Jingjing Liu AAML 46 113 0 05 Oct 2020
A Pilot Study of Text-to-SQL Semantic Parsing for Vietnamese A. Nguyen M. Dao Dat Quoc Nguyen 13 54 0 05 Oct 2020
PMI-Masking: Principled masking of correlated spans Yoav Levine Barak Lenz Opher Lieber Omri Abend Kevin Leyton-Brown Moshe Tennenholtz Y. Shoham 22 72 0 05 Oct 2020
How Effective is Task-Agnostic Data Augmentation for Pretrained Transformers? Shayne Longpre Yu Wang Christopher DuBois ViT 19 83 0 05 Oct 2020