RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 10,779 papers shown

Title
Cross-Lingual Knowledge Distillation for Answer Sentence Selection in Low-Resource Languages Shivanshu Gupta Yoshitomo Matsubara Ankita N. Chadha Alessandro Moschitti 97 2 0 25 May 2023
Uncovering and Categorizing Social Biases in Text-to-SQL Yang Liu Yan Gao Zhe Su Xiaokang Chen Elliott Ash Jian-Guang Lou 102 6 0 25 May 2023
SING: A Plug-and-Play DNN Learning Technique Adrien Courtois Damien Scieur Jean-Michel Morel Pablo Arias Thomas Eboli 68 0 0 25 May 2023
Sequential Integrated Gradients: a simple but effective method for explaining language models Joseph Enguehard 79 44 0 25 May 2023
UniTRec: A Unified Text-to-Text Transformer and Joint Contrastive Learning Framework for Text-based Recommendation Zhiming Mao Huimin Wang Yiming Du Kam-Fai Wong 100 27 0 25 May 2023
Comparative Study of Pre-Trained BERT Models for Code-Mixed Hindi-English Data Aryan Patil Varad Patwardhan Abhishek Phaltankar Gauri Takawane Raviraj Joshi 83 12 0 25 May 2023
Zero-shot Approach to Overcome Perturbation Sensitivity of Prompts Mohna Chakraborty Adithya Kulkarni Qi Li VLM 76 10 0 25 May 2023
BUCA: A Binary Classification Approach to Unsupervised Commonsense Question Answering Jie He U. SimonChiLok Víctor Gutiérrez-Basulto Jeff Z. Pan 178 10 0 25 May 2023
Text-Augmented Open Knowledge Graph Completion via Pre-Trained Language Models Pengcheng Jiang Shivam Agarwal Bowen Jin Xuan Wang Jimeng Sun Jiawei Han VLM RALM 52 21 0 24 May 2023
Flocks of Stochastic Parrots: Differentially Private Prompt Learning for Large Language Models Haonan Duan Adam Dziedzic Nicolas Papernot Franziska Boenisch AAML 84 67 0 24 May 2023
Automated Refugee Case Analysis: An NLP Pipeline for Supporting Legal Practitioners Claire Barale Michael Rovatsos Nehal Bhuta AILaw 35 7 0 24 May 2023
Exploring Automatically Perturbed Natural Language Explanations in Relation Extraction Wanyun Cui Xingran Chen LRM AAML 63 0 0 24 May 2023
Deriving Language Models from Masked Language Models Lucas Torroba Hennigen Yoon Kim 71 12 0 24 May 2023
Balancing the Picture: Debiasing Vision-Language Datasets with Synthetic Contrast Sets Brandon Smith Miguel Farinha Elizaveta Semenova Hannah Rose Kirk Aleksandar Shtedritski Max Bain 90 19 0 24 May 2023
Uncovering and Quantifying Social Biases in Code Generation Yang Liu Xiaokang Chen Yan Gao Zhe Su Fengji Zhang Daoguang Zan Jian-Guang Lou Pin-Yu Chen Tsung-Yi Ho 92 20 0 24 May 2023
Context-Aware Transformer Pre-Training for Answer Sentence Selection Luca Di Liello Siddhant Garg Alessandro Moschitti 71 4 0 24 May 2023
READ: Recurrent Adaptation of Large Transformers Sida I. Wang John Nguyen Ke Li Carole-Jean Wu 55 11 0 24 May 2023
Multi-Modal Mutual Attention and Iterative Interaction for Referring Image Segmentation Chang Liu Henghui Ding Yulun Zhang Xudong Jiang 107 50 0 24 May 2023
Self-Evolution Learning for Discriminative Language Model Pretraining Qihuang Zhong Liang Ding Juhua Liu Bo Du Dacheng Tao 102 12 0 24 May 2023
Revisiting Token Dropping Strategy in Efficient BERT Pretraining Qihuang Zhong Liang Ding Juhua Liu Xuebo Liu Min Zhang Bo Du Dacheng Tao VLM 75 10 0 24 May 2023
EvEval: A Comprehensive Evaluation of Event Semantics for Large Language Models Zhengwei Tao Zhi Jin Xiaoying Bai Haiyan Zhao Yanlin Feng Jia Li Wenpeng Hu 88 5 0 24 May 2023
Cross-lingual QA: A Key to Unlocking In-context Cross-lingual Performance Sunkyoung Kim Dayeon Ki Yireun Kim Jinsik Lee LRM 63 3 0 24 May 2023
Multi-modal Machine Learning for Vehicle Rating Predictions Using Image, Text, and Parametric Data Hanqi Su Binyang Song Faez Ahmed 56 6 0 24 May 2023
Towards Adaptive Prefix Tuning for Parameter-Efficient Language Model Fine-tuning Zhen-Ru Zhang Chuanqi Tan Haiyang Xu Chengyu Wang Jun Huang Songfang Huang 73 38 0 24 May 2023
Pre-training Multi-party Dialogue Models with Latent Discourse Inference Yiyang Li Xinting Huang Wei Bi Hai Zhao 72 6 0 24 May 2023
Dynamic Masking Rate Schedules for MLM Pretraining Zachary Ankner Naomi Saphra Davis W. Blalock Jonathan Frankle Matthew L. Leavitt 101 8 0 24 May 2023
C-STS: Conditional Semantic Textual Similarity Ameet Deshpande Carlos E. Jimenez Howard Chen Vishvak Murahari Victoria Graf Tanmay Rajpurohit Ashwin Kalyan Danqi Chen Karthik Narasimhan 61 3 0 24 May 2023
STAR: Boosting Low-Resource Information Extraction by Structure-to-Text Data Generation with Large Language Models Mingyu Derek Ma Xiaoxu Wang Po-Nien Kung P. Brantingham Nanyun Peng Wei Wang SyDa 100 5 0 24 May 2023
Detecting Check-Worthy Claims in Political Debates, Speeches, and Interviews Using Audio Data Petar Ivanov Ivan Koychev Momchil Hardalov Preslav Nakov 52 4 0 24 May 2023
Lawyer LLaMA Technical Report Quzhe Huang Mingxu Tao Chen Zhang Zhenwei An Cong Jiang Zhibin Chen Zirui Wu Yansong Feng ELM ALM AILaw 131 54 0 24 May 2023
Who Wrote this Code? Watermarking for Code Generation Taehyun Lee Seokhee Hong Jaewoo Ahn Ilgee Hong Hwaran Lee Sangdoo Yun Jamin Shin Gunhee Kim WaLM 69 98 0 24 May 2023
Reasoning over Hierarchical Question Decomposition Tree for Explainable Question Answering Jiajie Zhang S. Cao Tingjia Zhang Xin Lv Jiaxin Shi Qingwen Tian Juanzi Li Lei Hou 90 13 0 24 May 2023
SETI: Systematicity Evaluation of Textual Inference Xiyan Fu Anette Frank LRM 50 5 0 24 May 2023
How to Distill your BERT: An Empirical Study on the Impact of Weight Initialisation and Distillation Objectives Xinpeng Wang Leonie Weissweiler Hinrich Schütze Barbara Plank 66 8 0 24 May 2023
An Efficient Multilingual Language Model Compression through Vocabulary Trimming Asahi Ushio Yi Zhou Jose Camacho-Collados 128 8 0 24 May 2023
LAraBench: Benchmarking Arabic AI with Large Language Models Ahmed Abdelali Hamdy Mubarak Shammur A. Chowdhury Maram Hasanain Basel Mousi ... Yousseif Elshahawy Ahmed M. Ali Nadir Durrani Natasa Milic-Frayling Firoj Alam ELM LM&MA 70 21 0 24 May 2023
GPTAraEval: A Comprehensive Evaluation of ChatGPT on Arabic NLP Md. Tawkat Islam Khondaker Abdul Waheed El Moatez Billah Nagoudi Muhammad Abdul-Mageed ELM LM&MA 93 70 0 24 May 2023
Getting Sick After Seeing a Doctor? Diagnosing and Mitigating Knowledge Conflicts in Event Temporal Reasoning Tianqing Fang Zhaowei Wang Wenxuan Zhou Hongming Zhang Yangqiu Song Muhao Chen 74 15 0 24 May 2023
Detecting Multidimensional Political Incivility on Social Media Sagi Pendzel Nir Lotan Alon Zoizner Einat Minkov 29 1 0 24 May 2023
PESCO: Prompt-enhanced Self Contrastive Learning for Zero-shot Text Classification Yau-Shian Wang Ta-Chung Chi Ruohong Zhang Yiming Yang VLM 53 13 0 24 May 2023
Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmark Minje Choi Jiaxin Pei Sagar Kumar Chang Shu David Jurgens ALM LLMAG 131 72 0 24 May 2023
Towards Reliable Misinformation Mitigation: Generalization, Uncertainty, and GPT-4 Kellin Pelrine Anne Imouza Camille Thibault Meilina Reksoprodjo Caleb Gupta J. Christoph Jean-François Godbout Reihaneh Rabbany UQLM AI4CE 123 42 0 24 May 2023
Structural Ambiguity and its Disambiguation in Language Model Based Parsers: the Case of Dutch Clause Relativization G. Wijnholds M. Moortgat 50 3 0 24 May 2023
M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection Yuxia Wang Jonibek Mansurov Petar Ivanov Jinyan Su Artem Shelmanov ... Thomas Arnold Alham Fikri Aji Nizar Habash Iryna Gurevych Preslav Nakov DeLMO 92 127 0 24 May 2023
Text encoders bottleneck compositionality in contrastive vision-language models Amita Kamath Jack Hessel Kai-Wei Chang CoGe CLIP VLM 92 21 0 24 May 2023
Extracting Psychological Indicators Using Question Answering Luka Pavlović 21 0 0 24 May 2023
CAR: Conceptualization-Augmented Reasoner for Zero-Shot Commonsense Question Answering Weiqi Wang Tianqing Fang Wenxuan Ding Baixuan Xu Xin Liu Yangqiu Song Antoine Bosselut ReLM LRM 73 43 0 24 May 2023
Sparse Weight Averaging with Multiple Particles for Iterative Magnitude Pruning Moonseok Choi Hyungi Lee G. Nam Juho Lee 78 2 0 24 May 2023
Drafting Event Schemas using Language Models Anisha Gunjal Greg Durrett AI4TS 114 6 0 24 May 2023
Towards Few-shot Entity Recognition in Document Images: A Graph Neural Network Approach Robust to Image Manipulation Prashant Krishnan Zilong Wang Yangkun Wang Jingbo Shang 67 3 0 24 May 2023