RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 10,831 papers shown

Title
A Survey of Deep Learning for Mathematical Reasoning Pan Lu Liang Qiu Wenhao Yu Sean Welleck Kai-Wei Chang ReLM LRM 135 150 0 20 Dec 2022
SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Tasks Suwon Shon Siddhant Arora Chyi-Jiunn Lin Ankita Pasad Felix Wu Roshan S. Sharma Wei Wu Hung-yi Lee Karen Livescu Shinji Watanabe ELM 85 33 0 20 Dec 2022
Mini-Model Adaptation: Efficiently Extending Pretrained Models to New Languages via Aligned Shallow Training Kelly Marchisio Patrick Lewis Yihong Chen Mikel Artetxe 92 19 0 20 Dec 2022
Is GPT-3 a Good Data Annotator? Bosheng Ding Chengwei Qin Linlin Liu Yew Ken Chia Shafiq Joty Boyang Albert Li Lidong Bing 97 250 0 20 Dec 2022
Perplexed by Quality: A Perplexity-based Method for Adult and Harmful Content Detection in Multilingual Heterogeneous Web Data Timm Jansen Yangling Tong V. Zevallos Pedro Ortiz Suarez 78 20 0 20 Dec 2022
Fine-Grained Distillation for Long Document Retrieval Yucheng Zhou Tao Shen Xiubo Geng Chongyang Tao Guodong Long Can Xu Daxin Jiang RALM 84 30 0 20 Dec 2022
Towards Reasoning in Large Language Models: A Survey Jie Huang Kevin Chen-Chuan Chang LM&MA ELM LRM 219 645 0 20 Dec 2022
What Are You Token About? Dense Retrieval as Distributions Over the Vocabulary Ori Ram L. Bezalel Adi Zicher Yonatan Belinkov Jonathan Berant Amir Globerson 107 37 0 20 Dec 2022
CoCo: Coherence-Enhanced Machine-Generated Text Detection Under Data Limitation With Contrastive Learning Xiaoming Liu Zhaohan Zhang Yichen Wang Hang Pu Y. Lan Chao Shen 102 41 0 20 Dec 2022
Extrinsic Evaluation of Machine Translation Metrics Nikita Moghe Tom Sherborne Mark Steedman Alexandra Birch ELM 103 20 0 20 Dec 2022
Towards Unsupervised Visual Reasoning: Do Off-The-Shelf Features Know How to Reason? Monika Wysoczañska Tom Monnier Tomasz Trzciñski David Picard ReLM OCL 75 1 0 20 Dec 2022
Pre-trained Language Models for Keyphrase Generation: A Thorough Empirical Study Di Wu Wasi Uddin Ahmad Kai-Wei Chang 96 18 0 20 Dec 2022
GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator Jian Yang Shuming Ma Li Dong Shaohan Huang Haoyang Huang Yuwei Yin Dongdong Zhang Liqun Yang Furu Wei Zhoujun Li SyDa AI4CE 76 25 0 20 Dec 2022
Adam: Dense Retrieval Distillation with Adaptive Dark Examples Chongyang Tao Chang Liu Tao Shen Can Xu Xiubo Geng Binxing Jiao Daxin Jiang 95 5 0 20 Dec 2022
Human-Guided Fair Classification for Natural Language Processing Florian E.Dorner Momchil Peychev Nikola Konstantinov Naman Goel Elliott Ash Martin Vechev FaML 86 4 0 20 Dec 2022
Quirk or Palmer: A Comparative Study of Modal Verb Frameworks with Annotated Datasets Risako Owan Maria Gini Dongyeop Kang 36 1 0 20 Dec 2022
I Cast Detect Thoughts: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragons Pei Zhou Andrew Zhu Jennifer Hu Jay Pujara Xiang Ren Chris Callison-Burch Yejin Choi Prithviraj Ammanabrolu 86 28 0 20 Dec 2022
When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methods Zhuo Zhang Yuanhang Yang Yong Dai Zhuang Li Zenglin Xu FedML 127 85 0 20 Dec 2022
On the Blind Spots of Model-Based Evaluation Metrics for Text Generation Tianxing He Jingyu Zhang Tianle Wang Sachin Kumar Kyunghyun Cho James R. Glass Yulia Tsvetkov 158 45 0 20 Dec 2022
PLUE: Language Understanding Evaluation Benchmark for Privacy Policies in English Jianfeng Chi Wasi Uddin Ahmad Yuan Tian Kai-Wei Chang AILaw ELM 67 11 0 20 Dec 2022
Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table Perturbation Xinyu Pi Bin Wang Yan Gao Jiaqi Guo Zhoujun Li Jian-Guang Lou LMTD 100 32 0 20 Dec 2022
Inducing Character-level Structure in Subword-based Language Models with Type-level Interchange Intervention Training Jing-ling Huang Zhengxuan Wu Kyle Mahowald Christopher Potts 87 14 0 19 Dec 2022
MANTIS at TSAR-2022 Shared Task: Improved Unsupervised Lexical Simplification with Pretrained Encoders Xiaofei Li Daniel Wiechmann Yu Qiao E. Kerz 65 8 0 19 Dec 2022
Dataless Knowledge Fusion by Merging Weights of Language Models Xisen Jin Xiang Ren Daniel Preoţiuc-Pietro Pengxiang Cheng FedML MoMe 122 251 0 19 Dec 2022
Exploring Hybrid and Ensemble Models for Multiclass Prediction of Mental Health Status on Social Media S. Zanwar Daniel Wiechmann Yu Qiao E. Kerz AI4MH 71 5 0 19 Dec 2022
What to Read in a Contract? Party-Specific Summarization of Legal Obligations, Entitlements, and Prohibitions Abhilasha Sancheti Aparna Garimella Balaji Vasan Srinivasan Rachel Rudinger AILaw 92 3 0 19 Dec 2022
Do CoNLL-2003 Named Entity Taggers Still Work Well in 2023? Shuheng Liu Alan Ritter AI4TS 95 13 0 19 Dec 2022
LENS: A Learnable Evaluation Metric for Text Simplification Mounica Maddela Yao Dou David Heineman Wei Xu 62 65 0 19 Dec 2022
Improving Faithfulness of Abstractive Summarization by Controlling Confounding Effect of Irrelevant Sentences Asish Ghoshal Arash Einolghozati A. Arun Haoran Li L. Yu Vera Gor Yashar Mehdad Scott Yih Asli Celikyilmaz HILM 71 1 0 19 Dec 2022
MANER: Mask Augmented Named Entity Recognition for Extreme Low-Resource Languages Shashank Sonkar Zichao Wang Richard G. Baraniuk 63 1 0 19 Dec 2022
Human-in-the-loop Evaluation for Early Misinformation Detection: A Case Study of COVID-19 Treatments Ethan Mendes Yang Chen Wei Xu Alan Ritter 101 16 0 19 Dec 2022
StyleFlow: Disentangle Latent Representations via Normalizing Flow for Unsupervised Text Style Transfer Kangchen Zhu Zhiliang Tian Ruifeng Luo Xiaoguang Mao OOD 107 3 0 19 Dec 2022
Norm of Word Embedding Encodes Information Gain Momose Oyama Sho Yokoi Hidetoshi Shimodaira 76 12 0 19 Dec 2022
MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derendering Fangyu Liu Francesco Piccinno Syrine Krichene Chenxi Pang Kenton Lee Mandar Joshi Yasemin Altun Nigel Collier Julian Martin Eisenschlos VLM LRM 61 102 0 19 Dec 2022
Wukong-Reader: Multi-modal Pre-training for Fine-grained Visual Document Understanding Haoli Bai Zhiguang Liu Xiaojun Meng Wentao Li Shuangning Liu ... Liangwei Wang Lu Hou Jiansheng Wei Xin Jiang Qun Liu ViT 84 13 0 19 Dec 2022
Optimizing Prompts for Text-to-Image Generation Y. Hao Zewen Chi Li Dong Furu Wei 128 153 0 19 Dec 2022
Query-as-context Pre-training for Dense Passage Retrieval Xing Wu Guangyuan Ma Wanhui Qian Zijia Lin Songlin Hu 97 9 0 19 Dec 2022
Source-Free Domain Adaptation for Question Answering with Masked Self-training M. Yin B. Wang Yue Dong Charles Ling OOD 100 4 0 19 Dec 2022
Rethinking Label Smoothing on Multi-hop Question Answering Zhangyue Yin Yuxin Wang Xiannian Hu Yiguang Wu Hang Yan Xinyu Zhang Bo Zhao Xuanjing Huang Xipeng Qiu 78 11 0 19 Dec 2022
Improving the Generalizability of Text-Based Emotion Detection by Leveraging Transformers with Psycholinguistic Features S. Zanwar Daniel Wiechmann Yu Qiao E. Kerz 66 3 0 19 Dec 2022
Human in the loop: How to effectively create coherent topics by manually labeling only a few documents per class Anton Thielmann Christoph Weisser Benjamin Säfken 63 3 0 19 Dec 2022
Enriching Relation Extraction with OpenIE Alessandro Temperoni M. Biryukov Martin Theobald 56 1 0 19 Dec 2022
Bridging The Gap: Entailment Fused-T5 for Open-retrieval Conversational Machine Reading Comprehension Xiao Zhang Heyan Huang Zewen Chi Xian-Ling Mao 78 1 0 19 Dec 2022
APOLLO: A Simple Approach for Adaptive Pretraining of Language Models for Logical Reasoning Soumya Sanyal Yichong Xu Shuohang Wang Ziyi Yang Reid Pryzant Wenhao Yu Chenguang Zhu Xiang Ren ReLM LRM 106 10 0 19 Dec 2022
Statistical Dataset Evaluation: Reliability, Difficulty, and Validity Chengwen Wang Qingxiu Dong Xiaochen Wang Haitao Wang Zhifang Sui XAI 64 3 0 19 Dec 2022
PromptBoosting: Black-Box Text Classification with Ten Forward Passes Bairu Hou J. O'Connor Jacob Andreas Shiyu Chang Yang Zhang VLM 61 44 0 19 Dec 2022
TextGrad: Advancing Robustness Evaluation in NLP by Gradient-Driven Optimization Bairu Hou Jinghan Jia Yihua Zhang Guanhua Zhang Yang Zhang Sijia Liu Shiyu Chang SILM AAML 69 24 0 19 Dec 2022
I2D2: Inductive Knowledge Distillation with NeuroLogic and Self-Imitation Chandra Bhagavatula Jena D. Hwang Doug Downey Ronan Le Bras Ximing Lu Lianhui Qin Keisuke Sakaguchi Swabha Swayamdipta Peter West Yejin Choi 103 34 0 19 Dec 2022
Estimating the Adversarial Robustness of Attributions in Text with Transformers Adam Ivankay Mattia Rigotti Ivan Girardi Chiara Marchiori P. Frossard 68 1 0 18 Dec 2022
Recall, Expand and Multi-Candidate Cross-Encode: Fast and Accurate Ultra-Fine Entity Typing Chengyue Jiang Wenyang Hui Yong Jiang Xiaobin Wang Pengjun Xie Kewei Tu 86 4 0 18 Dec 2022