RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 10,835 papers shown

Title
CREPE: Can Vision-Language Foundation Models Reason Compositionally? Zixian Ma Jerry Hong Mustafa Omer Gul Mona Gandhi Irena Gao Ranjay Krishna CoGe 100 143 0 13 Dec 2022
ERNIE-Code: Beyond English-Centric Cross-lingual Pretraining for Programming Languages Yekun Chai Shuohuan Wang Chao Pang Yu Sun Hao Tian Hua Wu 93 38 0 13 Dec 2022
Distantly-Supervised Named Entity Recognition with Adaptive Teacher Learning and Fine-grained Student Ensemble Xiaoye Qu Jun Zeng Daizong Liu Zhefeng Wang Baoxing Huai Pan Zhou 76 22 0 13 Dec 2022
Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining Florian Tramèr Gautam Kamath Nicholas Carlini SILM 133 72 0 13 Dec 2022
TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities Zhe Zhao Yudong Li Cheng-An Hou Jing-xin Zhao Rong Tian ... Xingwu Sun Zhanhui Kang Xiaoyong Du Linlin Shen Kimmo Yan VLM 109 24 0 13 Dec 2022
Robust and Explainable Identification of Logical Fallacies in Natural Language Arguments Zhivar Sourati Vishnu Priya Prasanna Venkatesh D. Deshpande Himanshu Rawlani Filip Ilievski Hông-Ân Sandlin Alain Mermoud AAML 92 21 0 12 Dec 2022
Who Evaluates the Evaluators? On Automatic Metrics for Assessing AI-based Offensive Code Generators Pietro Liguori Cristina Improta R. Natella B. Cukic Domenico Cotroneo ELM 128 18 0 12 Dec 2022
Effective Seed-Guided Topic Discovery by Integrating Multiple Types of Contexts Yu Zhang Yunyi Zhang Martin Michalski Yucheng Jiang Yu Meng Jiawei Han 106 19 0 12 Dec 2022
Continuation KD: Improved Knowledge Distillation through the Lens of Continuation Optimization A. Jafari I. Kobyzev Mehdi Rezagholizadeh Pascal Poupart A. Ghodsi VLM 77 5 0 12 Dec 2022
DexBERT: Effective, Task-Agnostic and Fine-grained Representation Learning of Android Bytecode Tiezhu Sun Kevin Allix Kisub Kim Xin Zhou Dongsun Kim David Lo Tegawende F. Bissyande Jacques Klein 119 14 0 12 Dec 2022
Federated Few-Shot Learning for Mobile NLP Dongqi Cai Shangguang Wang Yaozong Wu F. Lin Mengwei Xu FedML 93 12 0 12 Dec 2022
Improving Generalization of Pre-trained Language Models via Stochastic Weight Averaging Peng Lu I. Kobyzev Mehdi Rezagholizadeh Ahmad Rashid A. Ghodsi Philippe Langlais MoMe 108 11 0 12 Dec 2022
MaNLP@SMM4H22: BERT for Classification of Twitter Posts Keshav Kapur Rajitha Harikrishnan 53 3 0 12 Dec 2022
Automated ICD Coding using Extreme Multi-label Long Text Transformer-based Models Leibo Liu O. Perez-Concha Anthony N. Nguyen Vicki Bennett Louisa R Jorm 84 19 0 12 Dec 2022
Domain Adaptation of Transformer-Based Models using Unlabeled Data for Relevance and Polarity Classification of German Customer Feedback Ahmad Idrissi-Yaghir Henning Schafer Nadja Bauer Christoph M. Friedrich 80 6 0 12 Dec 2022
Ensembling Transformers for Cross-domain Automatic Term Extraction T. Hanh Matej Martinc Andraz Pelicon Antoine Doucet Senja Pollak 52 6 0 12 Dec 2022
Multimodal and Explainable Internet Meme Classification A. Thakur Filip Ilievski Hông-Ân Sandlin Zhivar Sourati Luca Luceri Riccardo Tommasini Alain Mermoud 78 6 0 11 Dec 2022
Associations Between Natural Language Processing (NLP) Enriched Social Determinants of Health and Suicide Death among US Veterans Avijit Mitra R. Pradhan R. Melamed Kun Chen D. Hoaglin ... J. Reisman Zhichao Yang Weisong Liu J. Tsai Hongfeng Yu 66 32 0 11 Dec 2022
Extending TrOCR for Text Localization-Free OCR of Full-Page Scanned Receipt Images Hongkuan Zhang Edward Whittaker I. Kitagishi 75 4 0 11 Dec 2022
FastClass: A Time-Efficient Approach to Weakly-Supervised Text Classification Tingyu Xia Yue Wang Yuan Tian Yi-Ju Chang 55 2 0 11 Dec 2022
MORTY: Structured Summarization for Targeted Information Extraction from Scholarly Articles M. Y. Jaradeh M. Stocker Sören Auer 67 1 0 11 Dec 2022
Topic-Aware Response Generation in Task-Oriented Dialogue with Unstructured Knowledge Access Yue Feng Gerasimos Lampouras Ignacio Iacobacci 56 4 0 10 Dec 2022
Punctuation Restoration for Singaporean Spoken Languages: English, Malay, and Mandarin Abhinav Rao Ho Thi-Nga Chng Eng Siong 77 3 0 10 Dec 2022
Multi-task Learning for Personal Health Mention Detection on Social Media O. Aduragba Jialin Yu Alexandra I. Cristea 41 0 0 09 Dec 2022
The Turing Deception David Noever Matt Ciolino DeLMO ELM LRM 151 9 0 09 Dec 2022
Selective Amnesia: On Efficient, High-Fidelity and Blind Suppression of Backdoor Effects in Trojaned Machine Learning Models Rui Zhu Di Tang Siyuan Tang Wenyuan Xu Haixu Tang AAML FedML 101 14 0 09 Dec 2022
Masked Video Distillation: Rethinking Masked Feature Modeling for Self-supervised Video Representation Learning Rui Wang Dongdong Chen Zuxuan Wu Yinpeng Chen Xiyang Dai Mengchen Liu Lu Yuan Yu-Gang Jiang VGen 131 94 0 08 Dec 2022
OFASys: A Multi-Modal Multi-Task Learning System for Building Generalist Models Jinze Bai Rui Men Han Yang Xuancheng Ren Kai Dang ... Wenhang Ge Jianxin Ma Junyang Lin Jingren Zhou Chang Zhou 88 16 0 08 Dec 2022
BEVBert: Multimodal Map Pre-training for Language-guided Navigation Dongyan An Yuankai Qi Yangguang Li Yan Huang Liangsheng Wang Tieniu Tan Jing Shao 99 64 0 08 Dec 2022
Momentum Calibration for Text Generation Xingxing Zhang Yiran Liu Xun Wang Pengcheng He Yang Yu Si-Qing Chen Wayne Xiong Furu Wei 144 9 0 08 Dec 2022
DDSupport: Language Learning Support System that Displays Differences and Distances from Model Speech Kazuki Kawamura Jun Rekimoto 96 0 0 08 Dec 2022
LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models Chan Hee Song Jiaman Wu Clay Washington Brian M Sadler Wei-Lun Chao Yu-Chuan Su LLMAG LM&Ro 208 425 0 08 Dec 2022
Discovering Latent Knowledge in Language Models Without Supervision Collin Burns Haotian Ye Dan Klein Jacob Steinhardt 176 386 0 07 Dec 2022
Pivotal Role of Language Modeling in Recommender Systems: Enriching Task-specific and Task-agnostic Representation Learning Kyuyong Shin Hanock Kwak Wonjae Kim Jisu Jeong Seungjae Jung KyungHyun Kim Jung-Woo Ha Sang-Woo Lee 105 4 0 07 Dec 2022
Memorization of Named Entities in Fine-tuned BERT Models Andor Diera N. Lell Aygul Garifullina A. Scherp 68 0 0 07 Dec 2022
G-MAP: General Memory-Augmented Pre-trained Language Model for Domain Tasks Zhongwei Wan Yichun Yin Wei Zhang Jiaxin Shi Lifeng Shang Guangyong Chen Xin Jiang Qun Liu VLM CLL 127 18 0 07 Dec 2022
Hierarchical multimodal transformers for Multi-Page DocVQA Rubèn Pérez Tito Dimosthenis Karatzas Ernest Valveny 103 61 0 07 Dec 2022
Text Embeddings by Weakly-Supervised Contrastive Pre-training Liang Wang Nan Yang Xiaolong Huang Binxing Jiao Linjun Yang Daxin Jiang Rangan Majumder Furu Wei VLM 287 625 0 07 Dec 2022
A Generative Approach for Script Event Prediction via Contrastive Fine-tuning Fangqi Zhu Jun Gao Changlong Yu Wei Wang Cheng-Xian Xu Xin Mu Min Yang Ruifeng Xu 95 13 0 07 Dec 2022
JamPatoisNLI: A Jamaican Patois Natural Language Inference Dataset Ruth-Ann Armstrong John Hewitt Christopher D. Manning 94 16 0 07 Dec 2022
Improved Deep Neural Network Generalization Using m-Sharpness-Aware Minimization Kayhan Behdin Qingquan Song Aman Gupta D. Durfee Ayan Acharya S. Keerthi Rahul Mazumder AAML 55 5 0 07 Dec 2022
Counterfactual reasoning: Do language models need world knowledge for causal understanding? Jiaxuan Li Lang-Chi Yu Allyson Ettinger CML LRM 45 2 0 06 Dec 2022
LawngNLI: A Long-Premise Benchmark for In-Domain Generalization from Short to Long Contexts and for Implication-Based Retrieval William F. Bruno Dan Roth ELM AILaw 56 7 0 06 Dec 2022
SODA: A Natural Language Processing Package to Extract Social Determinants of Health for Cancer Studies Zehao Yu Xi Yang Chong Dang P. Adekkanattu Braja Gopal Patra ... T. George W. Hogan Yi Guo Jiang Bian Yonghui Wu 38 15 0 06 Dec 2022
Knowledge-Bridged Causal Interaction Network for Causal Emotion Entailment Weixiang Zhao Yanyan Zhao Zhuojun Li Bing Qin 80 34 0 06 Dec 2022
CySecBERT: A Domain-Adapted Language Model for the Cybersecurity Domain Markus Bayer Philip D. . Kuehn Ramin Shanehsaz Christian A. Reuter 73 50 0 06 Dec 2022
Modern French Poetry Generation with RoBERTa and GPT-2 Mika Hämäläinen Khalid Alnajjar Thierry Poibeau BDL 71 10 0 06 Dec 2022
DiSTRICT: Dialogue State Tracking with Retriever Driven In-Context Tuning Praveen Venkateswaran Evelyn Duesterwald Vatche Isahagian 96 9 0 06 Dec 2022
Sources of Noise in Dialogue and How to Deal with Them Derek Chen Zhou Yu 64 2 0 06 Dec 2022
LUNA: Language Understanding with Number Augmentations on Transformers via Number Plugins and Pre-training Hongwei Han Jialiang Xu Mengyuan Zhou Yijia Shao Shi Han Dongmei Zhang LMTD 99 9 0 06 Dec 2022