Investigating Active Sampling for Hardness Classification with Vision-Based Tactile Sensors

One of the most important object properties that humans and robots perceive through touch is hardness. This paper investigates information-theoretic active sampling strategies for sample-efficient hardness classification with vision-based tactile sensors. We evaluate three probabilistic classifier models and two model-uncertainty-based sampling strategies on a robotic setup as well as on a previously published dataset of samples collected by human testers. Our findings indicate that the active sampling approaches, driven by uncertainty metrics, surpass a random sampling baseline in terms of accuracy and stability. Additionally, while in our human study, the participants achieve an average accuracy of 48.00%, our best approach achieves an average accuracy of 88.78% on the same set of objects, demonstrating the effectiveness of vision-based tactile sensors for object hardness classification.

@article{chen2025_2505.13231,
  title={ Investigating Active Sampling for Hardness Classification with Vision-Based Tactile Sensors },
  author={ Junyi Chen and Alap Kshirsagar and Frederik Heller and Mario Gómez Andreu and Boris Belousov and Tim Schneider and Lisa P. Y. Lin and Katja Doerschner and Knut Drewing and Jan Peters },
  journal={arXiv preprint arXiv:2505.13231},
  year={ 2025 }
}