Evaluating Knowledge Graph Based Retrieval Augmented Generation Methods under Knowledge Incompleteness

Knowledge Graph based Retrieval-Augmented Generation (KG-RAG) is a technique that enhances Large Language Model (LLM) inference in tasks like Question Answering (QA) by retrieving relevant information from knowledge graphs (KGs). However, real-world KGs are often incomplete, meaning that essential information for answering questions may be missing. Existing benchmarks do not adequately capture the impact of KG incompleteness on KG-RAG performance. In this paper, we systematically evaluate KG-RAG methods under incomplete KGs by removing triples using different methods and analyzing the resulting effects. We demonstrate that KG-RAG methods are sensitive to KG incompleteness, highlighting the need for more robust approaches in realistic settings.

@article{zhou2025_2504.05163,
  title={ Evaluating Knowledge Graph Based Retrieval Augmented Generation Methods under Knowledge Incompleteness },
  author={ Dongzhuoran Zhou and Yuqicheng Zhu and Yuan He and Jiaoyan Chen and Evgeny Kharlamov and Steffen Staab },
  journal={arXiv preprint arXiv:2504.05163},
  year={ 2025 }
}