7
0

Lightweight Relevance Grader in RAG

Main:5 Pages
6 Figures
Bibliography:1 Pages
5 Tables
Appendix:1 Pages
Abstract

Retrieval-Augmented Generation (RAG) addresses limitations of large language models (LLMs) by leveraging a vector database to provide more accurate and up-to-date information. When a user submits a query, RAG executes a vector search to find relevant documents, which are then used to generate a response. However, ensuring the relevance of retrieved documents with a query would be a big challenge. To address this, a secondary model, known as a relevant grader, can be served to verify its relevance. To reduce computational requirements of a relevant grader, a lightweight small language model is preferred. In this work, we finetuned llama-3.2-1b as a relevant grader and achieved a significant increase in precision from 0.1301 to 0.7750. Its precision is comparable to that of llama-3.1-70b. Our code is available atthis https URL.

View on arXiv
@article{jeong2025_2506.14084,
  title={ Lightweight Relevance Grader in RAG },
  author={ Taehee Jeong },
  journal={arXiv preprint arXiv:2506.14084},
  year={ 2025 }
}
Comments on this paper

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.