v1v2 (latest)

Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support

10 December 2021

Michael A. Madaio

Lisa Egede

Hariharan Subramonyam

Jennifer Wortman Vaughan

Hanna M. Wallach

ArXiv (abs)PDF HTML

Papers citing "Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support"

50 / 60 papers shown

Title
Can we Debias Social Stereotypes in AI-Generated Images? Examining Text-to-Image Outputs and User Perceptions Saharsh Barve Andy Mao Jiayue Melissa Shi Prerna Juneja Koustuv Saha 27 0 0 27 May 2025
Fairness Practices in Industry: A Case Study in Machine Learning Teams Building Recommender Systems Jing Nathan Yan Junxiong Wang Jeffrey M. Rzeszotarski Allison Koenecke FaML 84 0 0 26 May 2025
AI LEGO: Scaffolding Cross-Functional Collaboration in Industrial Responsible AI Practices during Early Design Stages Muzhe Wu Yanzhi Zhao Shuyi Han Michael Xieyang Liu Hong Shen 142 1 0 15 May 2025
From Silos to Systems: Process-Oriented Hazard Analysis for AI Systems Shalaleh Rismani Roel Dobbe AJung Moon 66 4 0 29 Oct 2024
Ethics Whitepaper: Whitepaper on Ethical Research into Large Language Models Eddie L. Ungless Nikolas Vitsakis Zeerak Talat James Garforth Bjorn Ross Arno Onken Atoosa Kasirzadeh Alexandra Birch 71 1 0 17 Oct 2024
Minimum Viable Ethics: From Institutionalizing Industry AI Governance to Product Impact Archana Ahlawat Amy Winecoff Jonathan Mayer 80 1 0 11 Sep 2024
A Catalog of Fairness-Aware Practices in Machine Learning Engineering Gianmario Voria Giulia Sellitto Carmine Ferrara Francesco Abate A. Lucia F. Ferrucci Gemma Catolino Fabio Palomba FaML 111 3 0 29 Aug 2024
Supporting Industry Computing Researchers in Assessing, Articulating, and Addressing the Potential Negative Societal Impact of Their Work Wesley Hanwen Deng Solon Barocas Jennifer Wortman Vaughan 77 7 0 02 Aug 2024
The Impact of Responsible AI Research on Innovation and Development Ali Akbar Septiandri Marios Constantinides Daniele Quercia 88 1 0 22 Jul 2024
Representation Bias in Political Sample Simulations with Large Language Models Weihong Qi Hanjia Lyu Jiebo Luo 88 6 0 16 Jul 2024
EARN Fairness: Explaining, Asking, Reviewing, and Negotiating Artificial Intelligence Fairness Metrics Among Stakeholders Lin Luo Yuri Nakao Mathieu Chollet Hiroya Inakoshi Simone Stumpf 110 1 0 16 Jul 2024
A Taxonomy of Challenges to Curating Fair Datasets Dora Zhao M. Scheuerman Pooja Chitre Jerone T. A. Andrews Georgia Panagiotidou Shawn Walker Kathleen H. Pine Alice Xiang 97 2 0 10 Jun 2024
Public Technologies Transforming Work of the Public and the Public Sector Seyun Kim Bonnie Fan Willa Yunqi Yang Jessie Ramey Sarah E Fox Haiyi Zhu John Zimmerman Motahhare Eslami 64 4 0 28 May 2024
The Impossibility of Fair LLMs Jacy Reese Anthis Kristian Lum Michael Ekstrand Avi Feller Alexander D’Amour FaML 123 14 0 28 May 2024
Navigating Fairness: Practitioners' Understanding, Challenges, and Strategies in AI/ML Development Aastha Pant Rashina Hoda Chakkrit Tantithamthavorn Burak Turhan 61 2 0 21 Mar 2024
Analysing and Organising Human Communications for AI Fairness-Related Decisions: Use Cases from the Public Sector Mirthe Dankloff Vanja Skoric Giovanni Sileno Sennay Ghebreab Jacco van Ossenbruggen Emma Beauxis-Aussalet 47 4 0 20 Mar 2024
Farsight: Fostering Responsible AI Awareness During AI Application Prototyping Zijie J. Wang Chinmay Kulkarni Lauren Wilcox Michael Terry Michael A. Madaio 75 50 0 23 Feb 2024
Bias in Language Models: Beyond Trick Tests and Toward RUTEd Evaluation Kristian Lum Jacy Reese Anthis Chirag Nagpal Alex DÁmour Alexander D’Amour 120 17 0 20 Feb 2024
A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations G. Berman Nitesh Goyal Michael A. Madaio ELM 73 23 0 30 Jan 2024
Quantifying Stereotypes in Language Yang Liu 70 1 0 28 Jan 2024
A structured regression approach for evaluating model performance across intersectional subgroups Christine Herlihy Kimberly Truong Alexandra Chouldechova Miroslav Dudik 65 5 0 26 Jan 2024
Towards a Non-Ideal Methodological Framework for Responsible ML Ramaravind Kommiya Mothilal Shion Guha Syed Ishtiaque Ahmed 85 8 0 20 Jan 2024
Fairness Concerns in App Reviews: A Study on AI-based Mobile Apps A. R. Nasab Maedeh Dashti Mojtaba Shahin Mansooreh Zahedi Hourieh Khalajzadeh Chetan Arora Peng Liang 63 6 0 16 Jan 2024
Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine Harsha Nori Yin Tat Lee Sheng Zhang Dean Carignan Richard Edgar ... Hoifung Poon Tao Qin Naoto Usuyama Chris White Eric Horvitz LM&MA AI4MH MedIm ELM 101 327 0 28 Nov 2023
Model Compression in Practice: Lessons Learned from Practitioners Creating On-device Machine Learning Experiences Fred Hohman Mary Beth Kery Donghao Ren Dominik Moritz 101 19 0 06 Oct 2023
The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice Fernando Delgado Stephen Yang Michael A. Madaio Qian Yang 126 121 0 02 Oct 2023
On the Vulnerability of Fairness Constrained Learning to Malicious Noise Avrim Blum Princewill Okoroafor Aadirupa Saha Kevin Stangl 67 2 0 21 Jul 2023
Beyond the ML Model: Applying Safety Engineering Frameworks to Text-to-Image Development Shalaleh Rismani Renee Shelby A. Smart Renelito Delos Santos AJung Moon Negar Rostamzadeh 69 9 0 19 Jul 2023
Ethics in the Age of AI: An Analysis of AI Practitioners' Awareness and Challenges Aastha Pant Rashina Hoda Simone V. Spiegler Chakkrit Tantithamthavorn Burak Turhan 66 29 0 14 Jul 2023
Machine Learning practices and infrastructures G. Berman 69 4 0 13 Jul 2023
Scaling Laws Do Not Scale Fernando Diaz Michael A. Madaio 104 12 0 05 Jul 2023
Investigating Practices and Opportunities for Cross-functional Collaboration around AI Fairness in Industry Practice Wesley Hanwen Deng Nuri Yildirim Monica Chang Motahhare Eslami Kenneth Holstein Michael A. Madaio 108 46 0 10 Jun 2023
AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap Q. V. Liao J. Vaughan 145 168 0 02 Jun 2023
Survey of Trustworthy AI: A Meta Decision of AI Caesar Wu Yuan-Fang Li Pascal Bouvry 104 3 0 01 Jun 2023
Generating Process-Centric Explanations to Enable Contestability in Algorithmic Decision-Making: Challenges and Opportunities Mireia Yurrita Agathe Balayn U. Gadiraju 64 3 0 01 May 2023
Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoML Hilde J. P. Weerts Florian Pfisterer Matthias Feurer Katharina Eggensperger Eddie Bergman Noor H. Awad Joaquin Vanschoren Mykola Pechenizkiy B. Bischl Frank Hutter FaML 100 19 0 15 Mar 2023
Sensing Wellbeing in the Workplace, Why and For Whom? Envisioning Impacts with Organizational Stakeholders Anna Kawakami Shreyans Chowdhary Shamsi T. Iqbal Q. V. Liao Alexandra Olteanu J. Suh Koustuv Saha 67 35 0 13 Mar 2023
Chasing Fairness Under Distribution Shift: A Model Weight Perturbation Approach Zhimeng Jiang Xiaotian Han Hongye Jin Guanchu Wang Rui Chen Na Zou Helen Zhou 85 14 0 06 Mar 2023
Towards Algorithmic Fairness by means of Instance-level Data Re-weighting based on Shapley Values Adrián Arnaiz-Rodríguez Nuria Oliver FedML TDI 67 2 0 03 Mar 2023
Fairness Evaluation in Text Classification: Machine Learning Practitioner Perspectives of Individual and Group Fairness Zahra Ashktorab Benjamin Hoover Mayank Agarwal Casey Dugan Werner Geyer Han Yang Mikhail Yurochkin FaML 74 18 0 01 Mar 2023
Deliberating with AI: Improving Decision-Making for the Future through Participatory AI Design and Stakeholder Deliberation Angie Zhang Olympia Walker Kaci Nguyen Jiajun Dai Anqing Chen Min Kyung Lee 73 49 0 22 Feb 2023
Designerly Understanding: Information Needs for Model Transparency to Support Design Ideation for AI-Powered User Experience Q. V. Liao Hariharan Subramonyam Jennifer Wang Jennifer Wortman Vaughan HAI 86 60 0 21 Feb 2023
Concrete Safety for ML Problems: System Safety for ML Development and Assessment Edgar W. Jatho L. Mailloux Eugene D. Williams P. McClure Joshua A. Kroll 57 1 0 06 Feb 2023
Out of Context: Investigating the Bias and Fairness Concerns of "Artificial Intelligence as a Service" Kornel Lewicki M. S. Lee Jennifer Cobbe Jatinder Singh 88 22 0 02 Feb 2023
Investigating How Practitioners Use Human-AI Guidelines: A Case Study on the People + AI Guidebook Nur Yildirim Mahima Pushkarna Nitesh Goyal Martin Wattenberg Fernanda Viégas 101 70 0 28 Jan 2023
Seamful XAI: Operationalizing Seamful Design in Explainable AI Upol Ehsan Q. V. Liao Samir Passi Mark O. Riedl Hal Daumé 91 23 0 12 Nov 2022
System Safety Engineering for Social and Ethical ML Risks: A Case Study Edgar W. Jatho L. Mailloux Shalaleh Rismani Eugene D. Williams Joshua A. Kroll 82 2 0 08 Nov 2022
Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice Wesley Hanwen Deng B. Guo Alicia DeVrio Hong Shen Motahhare Eslami Kenneth Holstein MLAU 106 64 0 07 Oct 2022
Weak Proxies are Sufficient and Preferable for Fairness with Missing Sensitive Attributes Zhaowei Zhu Yuanshun Yao Jiankai Sun Hanguang Li Yang Liu 91 24 0 06 Oct 2022
From plane crashes to algorithmic harm: applicability of safety engineering frameworks for responsible ML Shalaleh Rismani Renee Shelby A. Smart Edgar W. Jatho Joshua A. Kroll AJung Moon Negar Rostamzadeh 106 42 0 06 Oct 2022