Can we teach Large Language Models (LLMs) to refrain from hallucinating factual statements? In this paper we present a fine-tuning strategy that we call ConfQA, which can reduce hallucination rate from 20-40% to under 5% across multiple factuality benchmarks. The core idea is simple: when the LLM answers a question correctly, it is trained to continue with the answer; otherwise, it is trained to admit "I am unsure". But there are two key factors that make the training highly effective. First, we introduce a dampening prompt "answer only if you are confident" to explicitly guide the behavior, without which hallucination remains high as 15%-25%. Second, we leverage simple factual statements, specifically attribute values from knowledge graphs, to help LLMs calibrate the confidence, resulting in robust generalization across domains and question types. Building on this insight, we propose the Dual Neural Knowledge framework, which seamlessly select between internally parameterized neural knowledge and externally recorded symbolic knowledge based on ConfQA's confidence. The framework enables potential accuracy gains to beyond 95%, while reducing unnecessary external retrievals by over 30%.
View on arXiv@article{huang2025_2506.07309,
title={ ConfQA: Answer Only If You Are Confident },
author={ Yin Huang and Yifan Ethan Xu and Kai Sun and Vera Yan and Alicia Sun and Haidar Khan and Jimmy Nguyen and Mohammad Kachuee and Zhaojiang Lin and Yue Liu and Aaron Colak and Anuj Kumar and Wen-tau Yih and Xin Luna Dong },
journal={arXiv preprint arXiv:2506.07309},
year={ 2025 }
}