Machine learning-based interatomic potentials and force fields depend critically on accurate atomic structures, yet such data are scarce due to the limited availability of experimentally resolved crystals. Although atomic-resolution electron microscopy offers a potential source of structural data, converting these images into simulation-ready formats remains labor-intensive and error-prone, creating a bottleneck for model training and validation. We introduce AutoMat, an end-to-end, agent-assisted pipeline that automatically transforms scanning transmission electron microscopy (STEM) images into atomic crystal structures and predicts their physical properties. AutoMat combines pattern-adaptive denoising, physics-guided template retrieval, symmetry-aware atomic reconstruction, fast relaxation and property prediction via MatterSim, and coordinated orchestration across all stages. We propose the first dedicated STEM2Mat-Bench for this task and evaluate performance using lattice RMSD, formation energy MAE, and structure-matching success rate. By orchestrating external tool calls, AutoMat enables a text-only LLM to outperform vision-language models in this domain, achieving closed-loop reasoning throughout the pipeline. In large-scale experiments over 450 structure samples, AutoMat substantially outperforms existing multimodal large language models and tools. These results validate both AutoMat and STEM2Mat-Bench, marking a key step toward bridging microscopy and atomistic simulation in materialsthis http URLcode and dataset are publicly available atthis https URLandthis https URL.
View on arXiv@article{yang2025_2505.12650,
title={ AutoMat: Enabling Automated Crystal Structure Reconstruction from Microscopy via Agentic Tool Use },
author={ Yaotian Yang and Yiwen Tang and Yizhe Chen and Xiao Chen and Jiangjie Qiu and Hao Xiong and Haoyu Yin and Zhiyao Luo and Yifei Zhang and Sijia Tao and Wentao Li and Qinghua Zhang and Yuqiang Li and Wanli Ouyang and Bin Zhao and Xiaonan Wang and Fei Wei },
journal={arXiv preprint arXiv:2505.12650},
year={ 2025 }
}