Recent Vision-Language Models (VLMs) have demonstrated impressive multimodal comprehension and reasoning capabilities, yet they often struggle with trivially simple visual tasks. In this work, we focus on the domain of basic 2D Euclidean geometry and systematically categorize the fundamental, indivisible visual perception skills, which we refer to as atomic visual skills. We then introduce the Atomic Visual Skills Dataset (AVSD) for evaluating VLMs on the atomic visual skills. Using AVSD, we benchmark state-of-the-art VLMs and find that they struggle with these tasks, despite being trivial for adult humans. Our findings highlight the need for purpose-built datasets to train and evaluate VLMs on atomic, rather than composite, visual perception tasks.
View on arXiv@article{chae2025_2505.20021,
title={ Decomposing Complex Visual Comprehension into Atomic Visual Skills for Vision Language Models },
author={ Hyunsik Chae and Seungwoo Yoon and Jaden Park and Chloe Yewon Chun and Yongin Cho and Mu Cai and Yong Jae Lee and Ernest K. Ryu },
journal={arXiv preprint arXiv:2505.20021},
year={ 2025 }
}