Graph-based Online Monitoring of Train Driver States via Facial and Skeletal Features

Driver fatigue poses a significant challenge to railway safety, with traditional systems like the dead-man switch offering limited and basic alertness checks. This study presents an online behavior-based monitoring system utilizing a customised Directed-Graph Neural Network (DGNN) to classify train driver's states into three categories: alert, not alert, and pathological. To optimize input representations for the model, an ablation study was performed, comparing three feature configurations: skeletal-only, facial-only, and a combination of both. Experimental results show that combining facial and skeletal features yields the highest accuracy (80.88%) in the three-class model, outperforming models using only facial or skeletal features. Furthermore, this combination achieves over 99% accuracy in the binary alertness classification. Additionally, we introduced a novel dataset that, for the first time, incorporates simulated pathological conditions into train driver monitoring, broadening the scope for assessing risks related to fatigue and health. This work represents a step forward in enhancing railway safety through advanced online monitoring using vision-based technologies.

@article{nocentini2025_2505.08800,
  title={ Graph-based Online Monitoring of Train Driver States via Facial and Skeletal Features },
  author={ Olivia Nocentini and Marta Lagomarsino and Gokhan Solak and Younggeol Cho and Qiyi Tong and Marta Lorenzini and Arash Ajoudani },
  journal={arXiv preprint arXiv:2505.08800},
  year={ 2025 }
}