Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors

6 May 2021

Motahhare Eslami

Papers citing "Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors"

47 / 47 papers shown

Title
AI LEGO: Scaffolding Cross-Functional Collaboration in Industrial Responsible AI Practices during Early Design Stages Muzhe Wu Yanzhi Zhao Shuyi Han Michael Xieyang Liu Hong Shen 32 0 0 15 May 2025
Algorithmic Collective Action with Two Collectives Aditya Karan Nicholas Vincent Karrie Karahalios Hari Sundaram 36 0 0 30 Apr 2025
Toward a Human-Centered Evaluation Framework for Trustworthy LLM-Powered GUI Agents Cen Chen Zhiping Zhang Ibrahim Khalilov Bingcan Guo Simret Araya Gebreegziabher Yanfang Ye Ziang Xiao Yaxing Yao Tianshi Li T. Li LLMAG ELM 89 2 0 24 Apr 2025
Un-Straightening Generative AI: How Queer Artists Surface and Challenge the Normativity of Generative AI Models Jordan Taylor Joel Mire Franchesca Spektor Alicia DeVrio Maarten Sap Haiyi Zhu Sarah E Fox 60 1 0 12 Mar 2025
Testing software for non-discrimination: an updated and extended audit in the Italian car insurance domain Marco Rondina A. Vetrò Riccardo Coppola Oumaima Regragrui Alessandro Fabris Gianmaria Silvello Gian Antonio Susto Juan Carlos De Martin 53 0 0 10 Feb 2025
Building Solidarity Amid Hostility: Experiences of Fat People in Online Communities Blakeley H. Payne Jordan Taylor Katta Spiel Casey Fiesler 43 0 0 06 Oct 2024
AI Policy Projector: Grounding LLM Policy Design in Iterative Mapmaking Michelle S. Lam Fred Hohman Dominik Moritz Jeffrey P. Bigham Kenneth Holstein Mary Beth Kery 40 1 0 26 Sep 2024
Designing an Intervention Tool for End-User Algorithm Audits in Personalized Recommendation Systems Qunfang Wu Lu Xian MLAU 52 0 0 20 Sep 2024
Let's Influence Algorithms Together: How Millions of Fans Build Collective Understanding of Algorithms and Organize Coordinated Algorithmic Actions Qing Xiao Yuhang Zheng Xianzhe Fan Bingbing Zhang Zhicong Lu 25 0 0 16 Sep 2024
End User Authoring of Personalized Content Classifiers: Comparing Example Labeling, Rule Writing, and LLM Prompting Leijie Wang Kathryn Yurechko Pranati Dani Quan Ze Chen Amy X. Zhang 55 3 0 05 Sep 2024
User-Driven Value Alignment: Understanding Users' Perceptions and Strategies for Addressing Biased and Discriminatory Statements in AI Companions Xianzhe Fan Qing Xiao Xuhui Zhou Jiaxin Pei Maarten Sap Zhicong Lu Hong Shen 71 7 0 01 Sep 2024
Misfitting With AI: How Blind People Verify and Contest AI Errors Rahaf Alharbi P. Lor Jaylin Herskovitz S. Schoenebeck Robin Brewer 38 10 0 13 Aug 2024
Experiences of Censorship on TikTok Across Marginalised Identities Eddie L. Ungless Nina Markl Bjorn Ross MLAU 34 2 0 19 Jul 2024
Auditing of AI: Legal, Ethical and Technical Approaches Jakob Mokander 59 38 0 07 Jul 2024
Generative AI Search Engines as Arbiters of Public Knowledge: An Audit of Bias and Authority Alice Li Luanne Sinnamon 38 3 0 22 May 2024
Exploring Subjectivity for more Human-Centric Assessment of Social Biases in Large Language Models Paula Akemi Aoyagui Sharon Ferguson Anastasia Kuzminykh 62 0 0 17 May 2024
The Fall of an Algorithm: Characterizing the Dynamics Toward Abandonment Nari Johnson Sanika Moharana Christina Harrington Nazanin Andalibi Hoda Heidari Motahhare Eslami 31 8 0 21 Apr 2024
Youth as Peer Auditors: Engaging Teenagers with Algorithm Auditing of Machine Learning Applications Luis Morales-Navarro Yasmin B. Kafai Vedya Konda D. Metaxa MLAU 36 3 0 08 Apr 2024
Law and the Emerging Political Economy of Algorithmic Audits P. Terzis Michael Veale Noëlle Gaumann MLAU 48 7 0 03 Apr 2024
(Beyond) Reasonable Doubt: Challenges that Public Defenders Face in Scrutinizing AI in Court Angela Jin Niloufar Salehi ELM 41 2 0 13 Mar 2024
Wikibench: Community-Driven Data Curation for AI Evaluation on Wikipedia Tzu-Sheng Kuo Aaron L Halfaker Zirui Cheng Jiwoo Kim Meng-Hsin Wu Tongshuang Wu Kenneth Holstein Haiyi Zhu 67 21 0 21 Feb 2024
Cruising Queer HCI on the DL: A Literature Review of LGBTQ+ People in HCI Jordan Taylor Ellen Simpson Anh-Ton Tran Jed R. Brubaker Sarah E Fox Haiyi Zhu 43 24 0 12 Feb 2024
Clarify: Improving Model Robustness With Natural Language Corrections Yoonho Lee Michelle S. Lam Helena Vasconcelos Michael S. Bernstein Chelsea Finn 40 6 0 06 Feb 2024
A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations G. Berman Nitesh Goyal Michael A. Madaio ELM 50 20 0 30 Jan 2024
Sociotechnical Audits: Broadening the Algorithm Auditing Lens to Investigate Targeted Advertising Michelle S. Lam Ayush Pandit Colin H. Kalicki Rachit Gupta Poonam Sahoo Danaë Metaxa MLAU 23 26 0 30 Aug 2023
Bound by the Bounty: Collaboratively Shaping Evaluation Processes for Queer AI Harms Organizers of QueerInAI Nathaniel Dennler Anaelia Ovalle Ashwin Singh Luca Soldaini ... Kyra Yee Irene Font Peradejordi Zeerak Talat Mayra Russo Jessica de Jesus de Pinho Pinhal 28 15 0 15 Jul 2023
Investigating Practices and Opportunities for Cross-functional Collaboration around AI Fairness in Industry Practice Wesley Hanwen Deng Nuri Yildirim Monica Chang Motahhare Eslami Kenneth Holstein Michael A. Madaio 42 43 0 10 Jun 2023
Stereotypes and Smut: The (Mis)representation of Non-cisgender Identities by Text-to-Image Models Eddie L. Ungless Bjorn Ross Anne Lauscher 41 32 0 26 May 2023
Supporting Human-AI Collaboration in Auditing LLMs with LLMs Charvi Rastogi Marco Tulio Ribeiro Nicholas King Harsha Nori Saleema Amershi ALM 40 68 0 19 Apr 2023
Angler: Helping Machine Translation Practitioners Prioritize Model Improvements Samantha Robertson Zijie J. Wang Dominik Moritz Mary Beth Kery Fred Hohman 43 15 0 12 Apr 2023
Participation and Division of Labor in User-Driven Algorithm Audits: How Do Everyday Users Work together to Surface Algorithmic Harms? Ren-de Li Sara Kingsley Chelsea Fan Proteeti Sinha Nora Wai Jaimie Lee Hong Shen Motahhare Eslami Jason I. Hong MLAU 29 16 0 04 Apr 2023
A Systematic Literature Review of Human-Centered, Ethical, and Responsible AI Mohammad Tahaei Marios Constantinides Daniele Quercia Michael J. Muller AI4TS 57 8 0 10 Feb 2023
Out of Context: Investigating the Bias and Fairness Concerns of "Artificial Intelligence as a Service" Kornel Lewicki M. S. Lee Jennifer Cobbe Jatinder Singh 41 21 0 02 Feb 2023
Auditing YouTube's Recommendation Algorithm for Misinformation Filter Bubbles Ivan Srba Robert Moro M. Tomlein Branislav Pecher Jakub Simko ... Michal Kompan Andrea Hrckova Juraj Podroužek Adrian Gavornik Maria Bielikova MLAU 44 33 0 18 Oct 2022
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction Renee Shelby Shalaleh Rismani Kathryn Henne AJung Moon Negar Rostamzadeh ... N'Mah Yilla-Akbari Jess Gallegos A. Smart Emilio Garcia Gurleen Virk 47 188 0 11 Oct 2022
A Keyword Based Approach to Understanding the Overpenalization of Marginalized Groups by English Marginal Abuse Models on Twitter Kyra Yee Alice Schoenauer Sebag Olivia Redfield Emily Sheng Matthias Eck Luca Belli 30 2 0 07 Oct 2022
Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice Wesley Hanwen Deng B. Guo Alicia DeVrio Hong Shen Motahhare Eslami Kenneth Holstein MLAU 31 58 0 07 Oct 2022
From plane crashes to algorithmic harm: applicability of safety engineering frameworks for responsible ML Shalaleh Rismani Renee Shelby A. Smart Edgar W. Jatho Joshua A. Kroll AJung Moon Negar Rostamzadeh 51 36 0 06 Oct 2022
The Algorithmic Imprint Upol Ehsan Ranjit Singh Jacob Metcalf Mark O. Riedl FaML 44 32 0 03 Jun 2022
Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits Wesley Hanwen Deng Manish Nagireddy M. S. Lee Jatinder Singh Zhiwei Steven Wu Kenneth Holstein Haiyi Zhu 45 89 0 13 May 2022
Towards a multi-stakeholder value-based assessment framework for algorithmic systems Mireia Yurrita Dave Murray-Rust Agathe Balayn A. Bozzon MLAU 36 29 0 09 May 2022
Interactive Model Cards: A Human-Centered Approach to Model Documentation Anamaria Crisan Margaret Drouhard Jesse Vig Nazneen Rajani HAI 45 87 0 05 May 2022
Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support Anna Kawakami Venkatesh Sivaraman H. Cheng Logan Stapleton Yanghuidi Cheng Diana Qing Adam Perer Zhiwei Steven Wu Haiyi Zhu Kenneth Holstein 44 107 0 05 Apr 2022
System Cards for AI-Based Decision-Making for Public Policy Furkan Gursoy I. Kakadiaris MLAU 34 14 0 01 Mar 2022
Seeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics Richmond Y. Wong Michael A. Madaio Nick Merrill 43 80 0 17 Feb 2022
Discovering and Validating AI Errors With Crowdsourced Failure Reports Ángel Alexander Cabrera Abraham J. Druck Jason I. Hong Adam Perer HAI 68 54 0 23 Sep 2021
Improving fairness in machine learning systems: What do industry practitioners need? Kenneth Holstein Jennifer Wortman Vaughan Hal Daumé Miroslav Dudík Hanna M. Wallach FaML HAI 195 742 0 13 Dec 2018