BBQ: A Hand-Built Bias Benchmark for Question Answering

15 October 2021

Papers citing "BBQ: A Hand-Built Bias Benchmark for Question Answering"

50 / 64 papers shown

Title
Developing A Framework to Support Human Evaluation of Bias in Generated Free Response Text Jennifer Healey Laurie Byrum Md Nadeem Akhtar Surabhi Bhargava Moumita Sinha 29 0 0 05 May 2025
BiasGuard: A Reasoning-enhanced Bias Detection Tool For Large Language Models Zhiting Fan Ruizhe Chen Zuozhu Liu 44 0 0 30 Apr 2025
$$\texttt{SAGE}$: A Generic Framework for LLM Safety Evaluation$ $\texttt{SAGE}$ : A Generic Framework for LLM Safety Evaluation Madhur Jindal Hari Shrawgi Parag Agrawal Sandipan Dandapat ELM 47 0 0 28 Apr 2025
Values in the Wild: Discovering and Analyzing Values in Real-World Language Model Interactions Saffron Huang Esin Durmus Miles McCain Kunal Handa Alex Tamkin Jerry Hong Michael Stern Arushi Somani Xiuruo Zhang Deep Ganguli VLM 51 1 0 21 Apr 2025
Following the Whispers of Values: Unraveling Neural Mechanisms Behind Value-Oriented Behaviors in LLMs Ling Hu Yuemei Xu Xiaoyang Gu Letao Han 28 0 0 07 Apr 2025
Intent-Aware Self-Correction for Mitigating Social Biases in Large Language Models Panatchakorn Anantaprayoon Masahiro Kaneko Naoaki Okazaki LRM KELM 50 0 0 08 Mar 2025
Evaluating Large Language Models for Public Health Classification and Extraction Tasks Joshua Harris Timothy Laurence Leo Loman Fan Grayson Toby Nonnenmacher ... Hamish Mohammed Thomas Finnie Luke Hounsome Michael Borowitz Steven Riley LM&MA AI4MH 83 5 0 20 Feb 2025
Bias Similarity Across Large Language Models Hyejun Jeong Shiqing Ma Amir Houmansadr 51 0 0 20 Feb 2025
Cross-Lingual Transfer of Debiasing and Detoxification in Multilingual LLMs: An Extensive Investigation Vera Neplenbroek Arianna Bisazza Raquel Fernández 103 0 0 17 Feb 2025
Fairness through Difference Awareness: Measuring Desired Group Discrimination in LLMs Angelina Wang Michelle Phan Daniel E. Ho Sanmi Koyejo 49 2 0 04 Feb 2025
Smaller Large Language Models Can Do Moral Self-Correction Guangliang Liu Zhiyu Xue Rongrong Wang K. Johnson Kristen Marie Johnson LRM 29 0 0 30 Oct 2024
Focus On This, Not That! Steering LLMs With Adaptive Feature Specification Tom A. Lamb Adam Davies Alasdair Paren Philip H. S. Torr Francesco Pinto 47 0 0 30 Oct 2024
ChineseSafe: A Chinese Benchmark for Evaluating Safety in Large Language Models H. Zhang Hongfu Gao Qiang Hu Guanhua Chen L. Yang Bingyi Jing Hongxin Wei Bing Wang Haifeng Bai Lei Yang AILaw ELM 49 2 0 24 Oct 2024
ELICIT: LLM Augmentation via External In-Context Capability Futing Wang Jianhao Yan Yue Zhang Tao Lin 39 0 0 12 Oct 2024
No Free Lunch: Retrieval-Augmented Generation Undermines Fairness in LLMs, Even for Vigilant Users Mengxuan Hu Hongyi Wu Zihan Guan Ronghang Zhu Dongliang Guo Daiqing Qi Sheng Li SILM 35 3 0 10 Oct 2024
COMPL-AI Framework: A Technical Interpretation and LLM Benchmarking Suite for the EU Artificial Intelligence Act Philipp Guldimann Alexander Spiridonov Robin Staab Nikola Jovanović Mark Vero ... Mislav Balunović Nikola Konstantinov Pavol Bielik Petar Tsankov Martin Vechev ELM 45 4 0 10 Oct 2024
Does RAG Introduce Unfairness in LLMs? Evaluating Fairness in Retrieval-Augmented Generation Systems Xuyang Wu Shuowei Li Hsin-Tai Wu Zhiqiang Tao Yi Fang 117 8 0 29 Sep 2024
Evaluating Gender, Racial, and Age Biases in Large Language Models: A Comparative Analysis of Occupational and Crime Scenarios Vishal Mirza Rahul Kulkarni Aakanksha Jadhav 49 2 0 22 Sep 2024
Edu-Values: Towards Evaluating the Chinese Education Values of Large Language Models Peiyi Zhang Yazhou Zhang Bo Wang Lu Rong Jing Qin Jing Qin AI4Ed ELM 47 1 0 19 Sep 2024
AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuances Dhruv Agarwal Mor Naaman Aditya Vashistha 36 16 0 17 Sep 2024
Bi-Factorial Preference Optimization: Balancing Safety-Helpfulness in Language Models Wenxuan Zhang Philip H. S. Torr Mohamed Elhoseiny Adel Bibi 77 9 0 27 Aug 2024
Bringing AI Participation Down to Scale: A Comment on Open AIs Democratic Inputs to AI Project David Moats Chandrima Ganguly VLM 38 0 0 16 Jul 2024
OffsetBias: Leveraging Debiased Data for Tuning Evaluators Junsoo Park Seungyeon Jwa Meiying Ren Daeyoung Kim Sanghyuk Choi ALM 34 31 0 09 Jul 2024
Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compression Zhichao Xu Ashim Gupta Tao Li Oliver Bentham Vivek Srikumar 44 8 0 06 Jul 2024
CEB: Compositional Evaluation Benchmark for Fairness in Large Language Models Song Wang Peng Wang Tong Zhou Yushun Dong Zhen Tan Jundong Li CoGe 51 6 0 02 Jul 2024
Monitoring Latent World States in Language Models with Propositional Probes Jiahai Feng Stuart Russell Jacob Steinhardt HILM 46 6 0 27 Jun 2024
SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal Tinghao Xie Xiangyu Qi Yi Zeng Yangsibo Huang Udari Madhushani Sehwag ... Bo Li Kai Li Danqi Chen Peter Henderson Prateek Mittal ALM ELM 50 51 0 20 Jun 2024
People will agree what I think: Investigating LLM's False Consensus Effect Junhyuk Choi Yeseon Hong Bugeun Kim 54 0 0 16 Jun 2024
Do Large Language Models Discriminate in Hiring Decisions on the Basis of Race, Ethnicity, and Gender? Haozhe An Christabel Acquaye Colin Wang Zongxia Li Rachel Rudinger 36 12 0 15 Jun 2024
Aligning to Thousands of Preferences via System Message Generalization Seongyun Lee Sue Hyun Park Seungone Kim Minjoon Seo ALM 41 37 0 28 May 2024
Quite Good, but Not Enough: Nationality Bias in Large Language Models -- A Case Study of ChatGPT Shucheng Zhu Weikang Wang Ying Liu 29 5 0 11 May 2024
Natural Language Processing RELIES on Linguistics Juri Opitz Shira Wein Nathan Schneider AI4CE 52 7 0 09 May 2024
SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety Paul Röttger Fabio Pernisi Bertie Vidgen Dirk Hovy ELM KELM 58 30 0 08 Apr 2024
A Comprehensive Evaluation of Quantization Strategies for Large Language Models Renren Jin Jiangcun Du Wuwei Huang Wei Liu Jian Luan Bin Wang Deyi Xiong MQ 30 31 0 26 Feb 2024
Evaluating Gender Bias in Large Language Models via Chain-of-Thought Prompting Masahiro Kaneko Danushka Bollegala Naoaki Okazaki Timothy Baldwin LRM 34 27 0 28 Jan 2024
Aligning Large Language Models with Counterfactual DPO Bradley Butcher ALM 21 1 0 17 Jan 2024
Evaluating and Mitigating Discrimination in Language Model Decisions Alex Tamkin Amanda Askell Liane Lovitt Esin Durmus Nicholas Joseph Shauna Kravec Karina Nguyen Jared Kaplan Deep Ganguli 38 66 0 06 Dec 2023
FlexModel: A Framework for Interpretability of Distributed Large Language Models Matthew Choi Muhammad Adil Asif John Willes David Emerson AI4CE ALM 24 1 0 05 Dec 2023
From Classification to Clinical Insights: Towards Analyzing and Reasoning About Mobile and Behavioral Health Data With Large Language Models Zachary Englhardt Chengqian Ma Margaret E. Morris X. Xu Chun-Cheng Chang Lianhui Qin Daniel J. McDuff Xin Liu Shwetak N. Patel Vikram Iyer AI4MH 44 11 0 21 Nov 2023
Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable Tasks Rahul Ramesh Ekdeep Singh Lubana Mikail Khona Robert P. Dick Hidenori Tanaka CoGe 33 6 0 21 Nov 2023
Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization Zhexin Zhang Junxiao Yang Pei Ke Fei Mi Hongning Wang Minlie Huang AAML 26 113 0 15 Nov 2023
Foundation Metrics for Evaluating Effectiveness of Healthcare Conversations Powered by Generative AI Mahyar Abbasian Elahe Khatibi Iman Azimi David Oniani Zahra Shakeri Hossein Abad ... Bryant Lin Olivier Gevaert Li-Jia Li Ramesh C. Jain Amir M. Rahmani LM&MA ELM AI4MH 37 66 0 21 Sep 2023
LatEval: An Interactive LLMs Evaluation Benchmark with Incomplete Information from Lateral Thinking Puzzles Shulin Huang Shirong Ma Yinghui Li Mengzuo Huang Wuhe Zou Weidong Zhang Haitao Zheng LLMAG LRM 28 26 0 21 Aug 2023
FairMonitor: A Four-Stage Automatic Framework for Detecting Stereotypes and Biases in Large Language Models Yanhong Bai Jiabao Zhao Jinxin Shi Tingjiang Wei Xingjiao Wu Liangbo He 33 0 0 21 Aug 2023
A Survey on Fairness in Large Language Models Yingji Li Mengnan Du Rui Song Xin Wang Ying Wang ALM 49 59 0 20 Aug 2023
Uncovering and Categorizing Social Biases in Text-to-SQL Y. Liu Yan Gao Zhe Su Xiaokang Chen Elliott Ash Jian-Guang Lou 58 6 0 25 May 2023
Uncovering and Quantifying Social Biases in Code Generation Y. Liu Xiaokang Chen Yan Gao Zhe Su Fengji Zhang Daoguang Zan Jian-Guang Lou Pin-Yu Chen Tsung-Yi Ho 36 19 0 24 May 2023
Learning from Mistakes via Cooperative Study Assistant for Large Language Models Danqing Wang Lei Li 28 6 0 23 May 2023
PaLM 2 Technical Report Rohan Anil Andrew M. Dai Orhan Firat Melvin Johnson Dmitry Lepikhin ... Ce Zheng Wei Zhou Denny Zhou Slav Petrov Yonghui Wu ReLM LRM 80 1,147 0 17 May 2023
Language Model Tokenizers Introduce Unfairness Between Languages Aleksandar Petrov Emanuele La Malfa Philip H. S. Torr Adel Bibi 16 96 0 17 May 2023