v1v2 (latest)

A Predictive Factor Analysis of Social Biases and Task-Performance in Pretrained Masked Language Models

19 October 2023

Yi Zhou

Jose Camacho-Collados

Danushka Bollegala

ArXiv (abs)PDF HTML

Papers citing "A Predictive Factor Analysis of Social Biases and Task-Performance in Pretrained Masked Language Models"

37 / 37 papers shown

Title
BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languages Junho Myung Nayeon Lee Yi Zhou Jiho Jin Rifki Afina Putri ... Seid Muhie Yimam Mohammad Taher Pilehvar N. Ousidhoum Jose Camacho-Collados Alice Oh 150 52 0 17 Jan 2025
Fewer Errors, but More Stereotypes? The Effect of Model Size on Gender Bias Yarden Tal Inbal Magar Roy Schwartz 62 35 0 20 Jun 2022
Legal Transformer Models May Not Always Help Sakbo Geng R. Lebret Karl Aberer VLM AILaw ELM 25 12 0 14 Sep 2021
Mitigating Language-Dependent Ethnic Bias in BERT Jaimeen Ahn Alice Oh 211 101 0 13 Sep 2021
Sustainable Modular Debiasing of Language Models Anne Lauscher Tobias Lüken Goran Glavaš 114 122 0 08 Sep 2021
Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies Sunipa Dev Masoud Monajatipoor Anaelia Ovalle Arjun Subramonian J. M. Phillips Kai-Wei Chang 114 170 0 27 Aug 2021
XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and Beyond Francesco Barbieri Luis Espinosa Anke Jose Camacho-Collados 216 221 0 25 Apr 2021
Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP Timo Schick Sahana Udupa Hinrich Schütze 306 386 0 28 Feb 2021
Debiasing Pre-trained Contextualised Embeddings Masahiro Kaneko Danushka Bollegala 238 142 0 23 Jan 2021
How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models Phillip Rust Jonas Pfeiffer Ivan Vulić Sebastian Ruder Iryna Gurevych 128 254 0 31 Dec 2020
HateBERT: Retraining BERT for Abusive Language Detection in English Tommaso Caselli Valerio Basile Jelena Mitrović Michael Granitzer 77 373 0 23 Oct 2020
TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification Francesco Barbieri Jose Camacho-Collados Leonardo Neves Luis Espinosa-Anke VLM 84 723 0 23 Oct 2020
Measuring and Reducing Gendered Correlations in Pre-trained Models Kellie Webster Xuezhi Wang Ian Tenney Alex Beutel Emily Pitler Ellie Pavlick Jilin Chen Ed Chi Slav Petrov FaML 79 260 0 12 Oct 2020
An Empirical Study of Tokenization Strategies for Various Korean NLP Tasks Kyubyong Park Joohong Lee Seongbo Jang Dawoon Jung 48 65 0 06 Oct 2020
BERTweet: A pre-trained language model for English Tweets Dat Quoc Nguyen Thanh Tien Vu A. Nguyen VLM 96 919 0 20 May 2020
Don't Stop Pretraining: Adapt Language Models to Domains and Tasks Suchin Gururangan Ana Marasović Swabha Swayamdipta Kyle Lo Iz Beltagy Doug Downey Noah A. Smith VLM AI4CE CLL 155 2,428 0 23 Apr 2020
Coreferential Reasoning Learning for Language Representation Deming Ye Yankai Lin Jiaju Du Zhenghao Liu Peng Li Maosong Sun Zhiyuan Liu 77 178 0 15 Apr 2020
SemEval-2017 Task 4: Sentiment Analysis in Twitter Sara Rosenthal N. Farra Preslav Nakov VLM 92 798 0 02 Dec 2019
Unsupervised Cross-lingual Representation Learning at Scale Alexis Conneau Kartikay Khandelwal Naman Goyal Vishrav Chaudhary Guillaume Wenzek Francisco Guzmán Edouard Grave Myle Ott Luke Zettlemoyer Veselin Stoyanov 223 6,565 0 05 Nov 2019
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter Victor Sanh Lysandre Debut Julien Chaumond Thomas Wolf 234 7,520 0 02 Oct 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 371 6,455 0 26 Sep 2019
On Measuring and Mitigating Biased Inferences of Word Embeddings Sunipa Dev Tao Li J. M. Phillips Vivek Srikumar 76 174 0 25 Aug 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 665 24,528 0 26 Jul 2019
XLNet: Generalized Autoregressive Pretraining for Language Understanding Zhilin Yang Zihang Dai Yiming Yang J. Carbonell Ruslan Salakhutdinov Quoc V. Le AI4CE 232 8,433 0 19 Jun 2019
Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets Yifan Peng Shankai Yan Zhiyong Lu LM&MA AI4MH 71 845 0 13 Jun 2019
Publicly Available Clinical BERT Embeddings Emily Alsentzer John R. Murphy Willie Boag W. Weng Di Jin Tristan Naumann Matthew B. A. McDermott AI4MH 158 1,979 0 06 Apr 2019
SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval) Marcos Zampieri S. Malmasi Preslav Nakov Sara Rosenthal N. Farra Ritesh Kumar 74 795 0 19 Mar 2019
Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting Maria De-Arteaga Alexey Romanov Hanna M. Wallach J. Chayes C. Borgs Alexandra Chouldechova S. Geyik K. Kenthapadi Adam Tauman Kalai 191 458 0 27 Jan 2019
BioBERT: a pre-trained biomedical language representation model for biomedical text mining Jinhyuk Lee Wonjin Yoon Sungdong Kim Donghyeon Kim Sunkyu Kim Chan Ho So Jaewoo Kang OOD 167 5,667 0 25 Jan 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 250 3,730 0 09 Jan 2019
Neural Network Acceptability Judgments Alex Warstadt Amanpreet Singh Samuel R. Bowman 233 1,411 0 31 May 2018
Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems S. Kiritchenko Saif M. Mohammad FaML 86 439 0 11 May 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,182 0 20 Apr 2018
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference Adina Williams Nikita Nangia Samuel R. Bowman 524 4,492 0 18 Apr 2017
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Zhiwen Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 903 6,796 0 26 Sep 2016
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 289 8,160 0 16 Jun 2016
Neural Machine Translation of Rare Words with Subword Units Rico Sennrich Barry Haddow Alexandra Birch 224 7,745 0 31 Aug 2015