CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models

30 September 2020

Papers citing "CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models"

50 / 139 papers shown

Title
Bridging AI and Carbon Capture: A Dataset for LLMs in Ionic Liquids and CBE Research Gaurab Sarkar Sougata Saha 30 0 0 11 May 2025
Developing A Framework to Support Human Evaluation of Bias in Generated Free Response Text Jennifer Healey Laurie Byrum Md Nadeem Akhtar Surabhi Bhargava Moumita Sinha 34 0 0 05 May 2025
Emotions in the Loop: A Survey of Affective Computing for Emotional Support Karishma Hegde Hemadri Jayalath 32 1 0 02 May 2025
BiasGuard: A Reasoning-enhanced Bias Detection Tool For Large Language Models Zhiting Fan Ruizhe Chen Zuozhu Liu 44 0 0 30 Apr 2025
Bias Beyond English: Evaluating Social Bias and Debiasing Methods in a Low-Resource Setting Ej Zhou Weiming Lu 31 0 0 15 Apr 2025
Gender and content bias in Large Language Models: a case study on Google Gemini 2.0 Flash Experimental Roberto Balestri 49 0 0 18 Mar 2025
Towards Large Language Models that Benefit for All: Benchmarking Group Fairness in Reward Models Kefan Song Jin Yao Runnan Jiang Rohan Chandra Shangtong Zhang ALM 46 0 0 10 Mar 2025
Gender Encoding Patterns in Pretrained Language Model Representations Mahdi Zakizadeh Mohammad Taher Pilehvar 48 0 0 09 Mar 2025
Cross-Lingual Transfer of Debiasing and Detoxification in Multilingual LLMs: An Extensive Investigation Vera Neplenbroek Arianna Bisazza Raquel Fernández 107 0 0 17 Feb 2025
Fairness through Difference Awareness: Measuring Desired Group Discrimination in LLMs Angelina Wang Michelle Phan Daniel E. Ho Sanmi Koyejo 54 2 0 04 Feb 2025
Understanding and Mitigating Gender Bias in LLMs via Interpretable Neuron Editing Zeping Yu Sophia Ananiadou KELM 47 1 0 24 Jan 2025
Characterization of GPU TEE Overheads in Distributed Data Parallel ML Training Jonghytun Lee Yongqin Wang Rachit Rajat M. Annavaram 55 0 0 20 Jan 2025
BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languages Junho Myung Nayeon Lee Yi Zhou Jiho Jin Rifki Afina Putri ... Seid Muhie Yimam Mohammad Taher Pilehvar N. Ousidhoum Jose Camacho-Collados Alice Oh 92 35 0 17 Jan 2025
Foundation Models at Work: Fine-Tuning for Fairness in Algorithmic Hiring Buse Sibel Korkmaz Rahul Nair Elizabeth M. Daly Evangelos Anagnostopoulos Christos Varytimidis Antonio del Rio Chanona 40 0 0 13 Jan 2025
Bridging the Fairness Gap: Enhancing Pre-trained Models with LLM-Generated Sentences Liu Yu Ludie Guo Ping Kuang Fan Zhou 44 0 0 12 Jan 2025
ChineseSafe: A Chinese Benchmark for Evaluating Safety in Large Language Models Han Zhang Hongfu Gao Qiang Hu Guanhua Chen L. Yang Bingyi Jing Hongxin Wei Bing Wang Haifeng Bai Lei Yang AILaw ELM 49 2 0 24 Oct 2024
LLMScan: Causal Scan for LLM Misbehavior Detection Mengdi Zhang Kai Kiat Goh Peixin Zhang Jun Sun Rose Lin Xin Hongyu Zhang 25 0 0 22 Oct 2024
Bias Similarity Across Large Language Models Hyejun Jeong Shiqing Ma Amir Houmansadr 54 0 0 15 Oct 2024
ELICIT: LLM Augmentation via External In-Context Capability Futing Wang Jianhao Yan Yue Zhang Tao Lin 44 0 0 12 Oct 2024
No Free Lunch: Retrieval-Augmented Generation Undermines Fairness in LLMs, Even for Vigilant Users Mengxuan Hu Hongyi Wu Zihan Guan Ronghang Zhu Dongliang Guo Daiqing Qi Sheng Li SILM 41 3 0 10 Oct 2024
STOP! Benchmarking Large Language Models with Sensitivity Testing on Offensive Progressions Robert D Morabito Sangmitra Madhusudan Tyler McDonald Ali Emami 31 0 0 20 Sep 2024
AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuances Dhruv Agarwal Mor Naaman Aditya Vashistha 36 16 0 17 Sep 2024
Acceptable Use Policies for Foundation Models Kevin Klyman 36 14 0 29 Aug 2024
Bi-Factorial Preference Optimization: Balancing Safety-Helpfulness in Language Models Wenxuan Zhang Philip Torr Mohamed Elhoseiny Adel Bibi 88 10 0 27 Aug 2024
From 'Showgirls' to 'Performers': Fine-tuning with Gender-inclusive Language for Bias Reduction in LLMs Marion Bartl Susan Leavy 43 8 0 05 Jul 2024
CEB: Compositional Evaluation Benchmark for Fairness in Large Language Models Song Wang Peng Wang Tong Zhou Yushun Dong Zhen Tan Jundong Li CoGe 56 7 0 02 Jul 2024
Raising the Bar: Investigating the Values of Large Language Models via Generative Evolving Testing Han Jiang Xiaoyuan Yi Zhihua Wei Shu Wang Xing Xie Xing Xie ALM ELM 54 5 0 20 Jun 2024
Do Large Language Models Discriminate in Hiring Decisions on the Basis of Race, Ethnicity, and Gender? Haozhe An Christabel Acquaye Colin Wang Zongxia Li Rachel Rudinger 36 12 0 15 Jun 2024
Benchmark Data Contamination of Large Language Models: A Survey Cheng Xu Shuhao Guan Derek Greene Mohand-Tahar Kechadi ELM ALM 38 39 0 06 Jun 2024
Ask LLMs Directly, "What shapes your bias?": Measuring Social Bias in Large Language Models Jisu Shin Hoyun Song Huije Lee Soyeong Jeong Jong C. Park 38 6 0 06 Jun 2024
Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Art Chen Cecilia Liu Iryna Gurevych Anna Korhonen 33 5 0 06 Jun 2024
Uncovering Bias in Large Vision-Language Models at Scale with Counterfactuals Phillip Howard Kathleen C. Fraser Anahita Bhiwandiwalla S. Kiritchenko 52 9 0 30 May 2024
Aligning to Thousands of Preferences via System Message Generalization Seongyun Lee Sue Hyun Park Seungone Kim Minjoon Seo ALM 44 38 0 28 May 2024
GPT is Not an Annotator: The Necessity of Human Annotation in Fairness Benchmark Construction Virginia K. Felkner Jennifer A. Thompson Jonathan May 49 9 0 24 May 2024
Quite Good, but Not Enough: Nationality Bias in Large Language Models -- A Case Study of ChatGPT Shucheng Zhu Weikang Wang Ying Liu 37 5 0 11 May 2024
Are Models Biased on Text without Gender-related Language? Catarina G Belém P. Seshadri Yasaman Razeghi Sameer Singh 41 8 0 01 May 2024
SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety Paul Röttger Fabio Pernisi Bertie Vidgen Dirk Hovy ELM KELM 60 32 0 08 Apr 2024
Specification Overfitting in Artificial Intelligence Benjamin Roth Pedro Henrique Luz de Araujo Yuxi Xia Saskia Kaltenbrunner Christoph Korab 58 0 0 13 Mar 2024
Angry Men, Sad Women: Large Language Models Reflect Gendered Stereotypes in Emotion Attribution Flor Miriam Plaza del Arco Amanda Cercas Curry Alba Curry Gavin Abercrombie Dirk Hovy 37 24 0 05 Mar 2024
LLM-Assisted Content Conditional Debiasing for Fair Text Embedding Wenlong Deng Blair Chen Beidi Zhao Chiyu Zhang Xiaoxiao Li Christos Thrampoulidis 35 0 0 22 Feb 2024
COBIAS: Assessing the Contextual Reliability of Bias Benchmarks for Language Models Priyanshul Govil Hemang Jain Vamshi Krishna Bonagiri Aman Chadha Ponnurangam Kumaraguru Manas Gaur Sanorita Dey 55 2 0 22 Feb 2024
Measuring Social Biases in Masked Language Models by Proxy of Prediction Quality Rahul Zalkikar Kanchan Chandra 37 1 0 21 Feb 2024
Bias in Language Models: Beyond Trick Tests and Toward RUTEd Evaluation Kristian Lum Jacy Reese Anthis Chirag Nagpal Alex DÁmour Alexander D’Amour 31 14 0 20 Feb 2024
Explaining Probabilistic Models with Distributional Values Luca Franceschi Michele Donini Cédric Archambeau Matthias Seeger FAtt 37 2 0 15 Feb 2024
CroissantLLM: A Truly Bilingual French-English Language Model Manuel Faysse Patrick Fernandes Nuno M. Guerreiro António Loison Duarte M. Alves ... François Yvon André F.T. Martins Gautier Viaud C´eline Hudelot Pierre Colombo 58 32 0 01 Feb 2024
Evaluating Gender Bias in Large Language Models via Chain-of-Thought Prompting Masahiro Kaneko Danushka Bollegala Naoaki Okazaki Timothy Baldwin LRM 37 27 0 28 Jan 2024
Quantifying Stereotypes in Language Yang Liu 38 1 0 28 Jan 2024
Multilingual large language models leak human stereotypes across language boundaries Yang Trista Cao Anna Sotnikova Jieyu Zhao Linda X. Zou Rachel Rudinger Hal Daumé PILM 33 10 0 12 Dec 2023
Identifying and Adapting Transformer-Components Responsible for Gender Bias in an English Language Model Abhijith Chintam Rahel Beloch Willem H. Zuidema Michael Hanna Oskar van der Wal 28 16 0 19 Oct 2023
Emerging Challenges in Personalized Medicine: Assessing Demographic Effects on Biomedical Question Answering Systems Sagi Shaier Kevin Bennett Lawrence E Hunter K. Wense 29 0 0 16 Oct 2023