Self-reflective Uncertainties: Do LLMs Know Their Internal Answer Distribution?

26 May 2025

Michael Kirchhof

Luca Füger

Adam Goliñski

Eeshan Gunesh Dhekane

Arno Blaas

Sinead Williamson

ArXiv (abs)PDF HTML

Papers citing "Self-reflective Uncertainties: Do LLMs Know Their Internal Answer Distribution?"

22 / 22 papers shown

Title
Revisiting Uncertainty Quantification Evaluation in Language Models: Spurious Interactions with Response Length Bias Results Andrea Santilli Adam Goliñski Michael Kirchhof Federico Danieli Arno Blaas Miao Xiong Luca Zappella Sinead Williamson 48 3 0 18 Apr 2025
Gemma 3 Technical Report Gemma Team Aishwarya B Kamath Johan Ferret Shreya Pathak Nino Vieillard ... Harshal Tushar Lehri Hussein Hazimeh Ian Ballantyne Idan Szpektor Ivan Nardini VLM 185 133 0 25 Mar 2025
Rethinking Uncertainty Estimation in Natural Language Generation L. Aichberger Kajetan Schweighofer Sepp Hochreiter 75 3 0 19 Dec 2024
Phi-4 Technical Report Marah Abdin J. Aneja Harkirat Singh Behl Sébastien Bubeck Ronen Eldan ... Rachel A. Ward Yue Wu Dingli Yu Cyril Zhang Yi Zhang ALM SyDa 174 147 0 12 Dec 2024
LoGU: Long-form Generation with Uncertainty Expressions Ruihan Yang Caiqi Zhang Zhisong Zhang Xinting Huang Sen Yang Nigel Collier Dong Yu Deqing Yang HILM 204 9 0 18 Oct 2024
On Subjective Uncertainty Quantification and Calibration in Natural Language Generation Ziyu Wang Chris Holmes UQLM 143 7 0 07 Jun 2024
SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales Tianyang Xu Shujin Wu Shizhe Diao Xiaoze Liu Xingyao Wang Yangyi Chen Jing Gao LRM 79 43 0 31 May 2024
Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words? G. Yona Roee Aharoni Mor Geva HILM 83 31 0 27 May 2024
Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantification Ekaterina Fadeeva Aleksandr Rubashevskii Artem Shelmanov Sergey Petrakov Haonan Li ... Gleb Kuzmin Alexander Panchenko Timothy Baldwin Preslav Nakov Maxim Panov HILM 82 56 0 07 Mar 2024
A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methods Hanlei Jin Yang Zhang Dan Meng Jun Wang Jinghua Tan 234 96 0 05 Mar 2024
A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications Pranab Sahoo Ayush Kumar Singh Sriparna Saha Vinija Jain S. Mondal Aman Chadha 120 325 0 05 Feb 2024
LM-Polygraph: Uncertainty Estimation for Language Models Ekaterina Fadeeva Roman Vashurin Akim Tsvigun Artem Vazhentsev Sergey Petrakov ... Elizaveta Goncharova Alexander Panchenko Maxim Panov Timothy Baldwin Artem Shelmanov 53 68 0 13 Nov 2023
Efficient Memory Management for Large Language Model Serving with PagedAttention Woosuk Kwon Zhuohan Li Siyuan Zhuang Ying Sheng Lianmin Zheng Cody Hao Yu Joseph E. Gonzalez Haotong Zhang Ion Stoica VLM 192 2,303 0 12 Sep 2023
Towards General Text Embeddings with Multi-stage Contrastive Learning Zehan Li Xin Zhang Yanzhao Zhang Dingkun Long Pengjun Xie Meishan Zhang 137 414 0 07 Aug 2023
Multi-Dimensional Evaluation of Text Summarization with In-Context Learning Sameer Jain Vaishakh Keshava Swarnashree Mysore Sathyendra Patrick Fernandes Pengfei Liu Graham Neubig Chunting Zhou ELM 117 39 0 01 Jun 2023
A Better Way to Do Masked Language Model Scoring Carina Kauf Anna A. Ivanova 73 27 0 17 May 2023
Teaching Models to Express Their Uncertainty in Words Stephanie C. Lin Jacob Hilton Owain Evans OOD 94 423 0 28 May 2022
Measuring Massive Multitask Language Understanding Dan Hendrycks Collin Burns Steven Basart Andy Zou Mantas Mazeika Basel Alomair Jacob Steinhardt ELM RALM 184 4,553 0 07 Sep 2020
Unsupervised Quality Estimation for Neural Machine Translation M. Fomicheva Shuo Sun Lisa Yankovskaya Frédéric Blain Francisco Guzmán Mark Fishel Nikolaos Aletras Vishrav Chaudhary Lucia Specia UQLM 87 209 0 21 May 2020
Fill in the BLANC: Human-free quality estimation of document summaries Oleg V. Vasilyev Vedant Dharnidharka John Bohannon 3DH 83 119 0 23 Feb 2020
BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model Alex Jinpeng Wang Kyunghyun Cho VLM 88 358 0 11 Feb 2019
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension Mandar Joshi Eunsol Choi Daniel S. Weld Luke Zettlemoyer RALM 222 2,686 0 09 May 2017