RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 10,862 papers shown

Title
Event-Centric Question Answering via Contrastive Learning and Invertible Event Transformation Junru Lu Xingwei Tan Gabriele Pergola Lin Gui Yulan He 98 11 0 24 Oct 2022
Knowledge Transfer from Answer Ranking to Answer Generation Matteo Gabburo Rik Koncel-Kedziorski Siddhant Garg Luca Soldaini Alessandro Moschitti 69 8 0 23 Oct 2022
Bootstrapping meaning through listening: Unsupervised learning of spoken sentence embeddings Jian Zhu Zuoyu Tian Yadong Liu Cong Zhang Chia-wen Lo SSL 90 2 0 23 Oct 2022
EUREKA: EUphemism Recognition Enhanced through Knn-based methods and Augmentation Sedrick Scott Keh Rohit K Bharadwaj Emmy Liu Simone Tedeschi Varun Gangal Roberto Navigli 64 7 0 23 Oct 2022
Data Augmentation for Automated Essay Scoring using Transformer Models Kshitij Gupta 98 4 0 23 Oct 2022
Discriminative Language Model as Semantic Consistency Scorer for Prompt-based Few-Shot Text Classification Zhipeng Xie Yahe Li 54 0 0 23 Oct 2022
BotsTalk: Machine-sourced Framework for Automatic Curation of Large-scale Multi-skill Dialogue Datasets Minju Kim Chaehyeong Kim Yongho Song Seung-won Hwang Jinyoung Yeo 148 14 0 23 Oct 2022
ComFact: A Benchmark for Linking Contextual Commonsense Knowledge Silin Gao Jena D. Hwang Saya Kanno Hiromi Wakaki Yuki Mitsufuji Antoine Bosselut HILM 96 16 0 23 Oct 2022
Lexical Generalization Improves with Larger Models and Longer Training Elron Bandel Yoav Goldberg Yanai Elazar 113 7 0 23 Oct 2022
Extending Phrase Grounding with Pronouns in Visual Dialogues Panzhong Lu Xin Zhang Meishan Zhang Min Zhang ObjD 84 4 0 23 Oct 2022
A BERT-based Deep Learning Approach for Reputation Analysis in Social Media Mohammad Wali Ur Rahman Sicong Shao Pratik Satam Salim Hariri Chris Padilla Zoe Taylor C. Nevarez 56 5 0 23 Oct 2022
Model ensemble instead of prompt fusion: a sample-specific knowledge transfer method for few-shot prompt tuning Xiangyu Peng Chen Xing Prafulla Kumar Choubey Chien-Sheng Wu Caiming Xiong VLM 137 12 0 23 Oct 2022
Language Model Pre-Training with Sparse Latent Typing Liliang Ren Zixuan Zhang H. Wang Clare R. Voss Chengxiang Zhai Heng Ji 101 3 0 23 Oct 2022
The Curious Case of Absolute Position Embeddings Koustuv Sinha Amirhossein Kazemnejad Siva Reddy J. Pineau Dieuwke Hupkes Adina Williams 150 15 0 23 Oct 2022
A Visual Tour Of Current Challenges In Multimodal Language Models Shashank Sonkar Naiming Liu Richard G. Baraniuk DiffM 50 2 0 22 Oct 2022
On the Limitations of Reference-Free Evaluations of Generated Text Daniel Deutsch Rotem Dror Dan Roth 127 48 0 22 Oct 2022
Exploring The Landscape of Distributional Robustness for Question Answering Models Anas Awadalla Mitchell Wortsman Gabriel Ilharco Sewon Min Ian H. Magnusson Hannaneh Hajishirzi Ludwig Schmidt ELM OOD KELM 124 21 0 22 Oct 2022
Training Dynamics for Curriculum Learning: A Study on Monolingual and Cross-lingual NLU Fenia Christopoulou Gerasimos Lampouras Ignacio Iacobacci 105 4 0 22 Oct 2022
DiscoSense: Commonsense Reasoning with Discourse Connectives Prajjwal Bhargava Vincent Ng LRM 345 4 0 22 Oct 2022
Spectrum-BERT: Pre-training of Deep Bidirectional Transformers for Spectral Classification of Chinese Liquors Yansong Wang Yundong Sun Yan-Jiao Fu Dongjie Zhu Zhaoshuo Tian 46 6 0 22 Oct 2022
PATS: Sensitivity-aware Noisy Learning for Pretrained Language Models Yupeng Zhang Hongzhi Zhang Sirui Wang Wei Wu Zhoujun Li AAML 104 1 0 22 Oct 2022
NeuroCounterfactuals: Beyond Minimal-Edit Counterfactuals for Richer Data Augmentation Phillip Howard Gadi Singer Vasudev Lal Yejin Choi Swabha Swayamdipta CML 121 25 0 22 Oct 2022
FCGEC: Fine-Grained Corpus for Chinese Grammatical Error Correction Lvxiaowei Xu Jian Wu Jiawei Peng Jiayu Fu Ming Cai 114 16 0 22 Oct 2022
EnDex: Evaluation of Dialogue Engagingness at Scale Guangxuan Xu Ruibo Liu Fabrice Harel-Canada Nischal Reddy Chandra Nanyun Peng 63 5 0 22 Oct 2022
P $^3$ LM: Probabilistically Permuted Prophet Language Modeling for Generative Pre-Training Junwei Bao Yifan Wang Jiangyong Ying Yeyun Gong Jing Zhao Youzheng Wu Xiaodong He 74 1 0 22 Oct 2022
R $^2$ F: A General Retrieval, Reading and Fusion Framework for Document-level Natural Language Inference Hao Wang Yixin Cao Yangguang Li Zhen Huang Kun Wang Jing Shao FedML 73 0 0 22 Oct 2022
Z-LaVI: Zero-Shot Language Solver Fueled by Visual Imagination Yue Yang Wenlin Yao Hongming Zhang Xiaoyang Wang Dong Yu Jianshu Chen VLM 101 22 0 21 Oct 2022
Enhancing Tabular Reasoning with Pattern Exploiting Training Abhilash Shankarampeta Vivek Gupta Shuo Zhang LMTD RALM ReLM 143 6 0 21 Oct 2022
TCAB: A Large-Scale Text Classification Attack Benchmark Kalyani Asthana Zhouhang Xie Wencong You Adam Noack Jonathan Brophy Sameer Singh Daniel Lowd 125 3 0 21 Oct 2022
SpaBERT: A Pretrained Language Model from Geographic Data for Geo-Entity Representation Zekun Li Jina Kim Yao-Yi Chiang Muhao Chen 133 32 0 21 Oct 2022
WikiWhy: Answering and Explaining Cause-and-Effect Questions Matthew Ho Aditya Sharma Justin Chang Michael Stephen Saxon Sharon Levy Yujie Lu William Yang Wang ReLM KELM LRM 170 19 0 21 Oct 2022
Experiencer-Specific Emotion and Appraisal Prediction Maximilian Wegge Enrica Troiano Laura Oberländer Roman Klinger 101 7 0 21 Oct 2022
Optimizing text representations to capture (dis)similarity between political parties Tanise Ceron Nico Blokker Sebastian Padó 52 6 0 21 Oct 2022
Shift-Reduce Task-Oriented Semantic Parsing with Stack-Transformers Daniel Fernández-González 77 0 0 21 Oct 2022
Multimodal Model with Text and Drug Embeddings for Adverse Drug Reaction Classification Andrey Sakhovskiy E. Tutubalina 61 21 0 21 Oct 2022
STAR: SQL Guided Pre-Training for Context-dependent Text-to-SQL Parsing Zefeng Cai Xiangyu Li Binyuan Hui Min Yang Bowen Li ... Zhen Cao Weijie Li Fei Huang Luo Si Yongbin Li 75 32 0 21 Oct 2022
LittleBird: Efficient Faster & Longer Transformer for Question Answering Minchul Lee Kijong Han M. Shin VLM 122 6 0 21 Oct 2022
Robustifying Sentiment Classification by Maximally Exploiting Few Counterfactuals Maarten De Raedt Fréderic Godin Chris Develder Thomas Demeester 54 1 0 21 Oct 2022
Diffuser: Efficient Transformers with Multi-hop Attention Diffusion for Long Sequences Aosong Feng Irene Li Yuang Jiang Rex Ying 86 18 0 21 Oct 2022
Modeling Document-level Temporal Structures for Building Temporal Dependency Graphs Prafulla Kumar Choubey Ruihong Huang 47 3 0 21 Oct 2022
InforMask: Unsupervised Informative Masking for Language Model Pretraining Nafis Sadeq Canwen Xu Julian McAuley 111 13 0 21 Oct 2022
CEFR-Based Sentence Difficulty Annotation and Assessment Yuki Arase Satoru Uchida Tomoyuki Kajiwara 73 25 0 21 Oct 2022
Dissecting Deep Metric Learning Losses for Image-Text Retrieval Hong Xuan Xi Chen 75 2 0 21 Oct 2022
Metric-guided Distillation: Distilling Knowledge from the Metric to Ranker and Retriever for Generative Commonsense Reasoning Xingwei He Yeyun Gong Alex Jin Weizhen Qi Hang Zhang Jian Jiao Bartuer Zhou Biao Cheng Sm Yiu Nan Duan 78 11 0 21 Oct 2022
Multi-View Reasoning: Consistent Contrastive Learning for Math Word Problem Wenqi Zhang Yongliang Shen Yanna Ma Xiaoxia Cheng Zeqi Tan Qingpeng Nong Weiming Lu 85 23 0 21 Oct 2022
Finding Dataset Shortcuts with Grammar Induction Dan Friedman Alexander Wettig Danqi Chen 112 16 0 20 Oct 2022
Unsupervised Text Deidentification John X. Morris Justin T. Chiu Ramin Zabih Alexander M. Rush 72 7 0 20 Oct 2022
Balanced Adversarial Training: Balancing Tradeoffs between Fickleness and Obstinacy in NLP Models Hannah Chen Yangfeng Ji David Evans SILM AAML 87 4 0 20 Oct 2022
Choose Your Lenses: Flaws in Gender Bias Evaluation Hadas Orgad Yonatan Belinkov 84 37 0 20 Oct 2022
Scaling Instruction-Finetuned Language Models Hyung Won Chung Le Hou Shayne Longpre Barret Zoph Yi Tay ... Jacob Devlin Adam Roberts Denny Zhou Quoc V. Le Jason W. Wei ReLM LRM 371 3,180 0 20 Oct 2022