Prior studies show that adopting the annotation diversity shaped by different backgrounds and life experiences and incorporating them into the model learning, i.e. multi-perspective approach, contribute to the development of more responsible models. Thus, in this paper we propose a new framework for designing and further evaluating perspective-aware models on stance detection task,in which multiple annotators assign stances based on a controversial topic. We also share a new dataset established through obtaining both human and LLM annotations. Results show that the multi-perspective approach yields better classification performance (higher F1-scores), outperforming the traditional approaches that use a single ground-truth, while displaying lower model confidence scores, probably due to the high level of subjectivity of the stance detection task.
View on arXiv@article{muscato2025_2503.00489,
title={ Embracing Diversity: A Multi-Perspective Approach with Soft Labels },
author={ Benedetta Muscato and Praveen Bushipaka and Gizem Gezici and Lucia Passaro and Fosca Giannotti and Tommaso Cucinotta },
journal={arXiv preprint arXiv:2503.00489},
year={ 2025 }
}