This paper presents a detailed evaluation of a Retrieval-Augmented Generation (RAG) system that integrates large language models (LLMs) to enhance information retrieval and instruction generation for maintenance personnel across diverse data formats. We assessed the performance of eight LLMs, emphasizing key metrics such as response speed and accuracy, which were quantified using BLEU and METEOR scores. Our findings reveal that advanced models like GPT-4 and GPT-4o-mini significantly outperform their counterparts, particularly when addressing complex queries requiring multi-format data integration. The results validate the system's ability to deliver timely and accurate responses, highlighting the potential of RAG frameworks to optimize maintenance operations. Future research will focus on refining retrieval techniques for these models and enhancing response generation, particularly for intricate scenarios, ultimately improving the system's practical applicability in dynamic real-world environments.
View on arXiv@article{nagy2025_2502.15604,
title={ Cross-Format Retrieval-Augmented Generation in XR with LLMs for Context-Aware Maintenance Assistance },
author={ Akos Nagy and Yannis Spyridis and Vasileios Argyriou },
journal={arXiv preprint arXiv:2502.15604},
year={ 2025 }
}