ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2506.09669
74
0

Query-Level Uncertainty in Large Language Models

11 June 2025
Lihu Chen
Gaël Varoquaux
ArXiv (abs)PDFHTML
Main:8 Pages
5 Figures
Bibliography:3 Pages
1 Tables
Abstract

It is important for Large Language Models to be aware of the boundary of their knowledge, the mechanism of identifying known and unknown queries. This type of awareness can help models perform adaptive inference, such as invoking RAG, engaging in slow and deep thinking, or adopting the abstention mechanism, which is beneficial to the development of efficient and trustworthy AI. In this work, we propose a method to detect knowledge boundaries via Query-Level Uncertainty, which aims to determine if the model is able to address a given query without generating any tokens. To this end, we introduce a novel and training-free method called \emph{Internal Confidence}, which leverages self-evaluations across layers and tokens. Empirical results on both factual QA and mathematical reasoning tasks demonstrate that our internal confidence can outperform several baselines. Furthermore, we showcase that our proposed method can be used for efficient RAG and model cascading, which is able to reduce inference costs while maintaining performance.

View on arXiv
@article{chen2025_2506.09669,
  title={ Query-Level Uncertainty in Large Language Models },
  author={ Lihu Chen and Gaël Varoquaux },
  journal={arXiv preprint arXiv:2506.09669},
  year={ 2025 }
}
Comments on this paper