As our understanding of autism and ableism continues to increase, so does our understanding of ableist language towards autistic people. Such language poses a significant challenge in NLP research due to its subtle and context-dependent nature. Yet, detecting anti-autistic ableist language remains underexplored, with existing NLP tools often failing to capture its nuanced expressions. We present AUTALIC, the first benchmark dataset dedicated to the detection of anti-autistic ableist language in context, addressing a significant gap in the field. The dataset comprises 2,400 autism-related sentences collected from Reddit, accompanied by surrounding context, and is annotated by trained experts with backgrounds in neurodiversity. Our comprehensive evaluation reveals that current language models, including state-of-the-art LLMs, struggle to reliably identify anti-autistic ableism and align with human judgments, underscoring their limitations in this domain. We publicly release AUTALIC along with the individual annotations which serve as a valuable resource to researchers working on ableism, neurodiversity, and also studying disagreements in annotation tasks. This dataset serves as a crucial step towards developing more inclusive and context-aware NLP systems that better reflect diverse perspectives.
View on arXiv@article{rizvi2025_2410.16520,
title={ AUTALIC: A Dataset for Anti-AUTistic Ableist Language In Context },
author={ Naba Rizvi and Harper Strickland and Daniel Gitelman and Tristan Cooper and Alexis Morales-Flores and Michael Golden and Aekta Kallepalli and Akshat Alurkar and Haaset Owens and Saleha Ahmedi and Isha Khirwadkar and Imani Munyaka and Nedjma Ousidhoum },
journal={arXiv preprint arXiv:2410.16520},
year={ 2025 }
}