Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models

30 May 2023

Jimeng Sun

Papers citing "Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models"

38 / 38 papers shown

Title
Calibrating Uncertainty Quantification of Multi-Modal LLMs using Grounding Trilok Padhi R. Kaur Adam D. Cobb Manoj Acharya Anirban Roy Colin Samplawski Brian Matejek Alexander M. Berenbeim Nathaniel D. Bastian Susmit Jha 28 0 0 30 Apr 2025
Uncertainty Quantification for Language Models: A Suite of Black-Box, White-Box, LLM Judge, and Ensemble Scorers Dylan Bouchard Mohit Singh Chauhan HILM 84 0 0 27 Apr 2025
Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Review Toghrul Abbasli Kentaroh Toyoda Yuan Wang Leon Witt Muhammad Asif Ali Yukai Miao Dan Li Qingsong Wei UQCV 94 0 0 25 Apr 2025
Data-Driven Calibration of Prediction Sets in Large Vision-Language Models Based on Inductive Conformal Prediction Yuanchang Ye Weiyan Wen VLM 65 0 0 24 Apr 2025
Hallucination Detection in LLMs via Topological Divergence on Attention Graphs Alexandra Bazarova Aleksandr Yugay Andrey Shulga A. Ermilova Andrei Volodichev ... Dmitry Simakov M. Savchenko Andrey Savchenko Serguei Barannikov Alexey Zaytsev HILM 35 0 0 14 Apr 2025
TruthPrInt: Mitigating LVLM Object Hallucination Via Latent Truthful-Guided Pre-Intervention Jinhao Duan Fei Kong Hao-Ran Cheng James Diffenderfer B. Kailkhura Lichao Sun Xiaofeng Zhu Xiaoshuang Shi Kaidi Xu 173 0 0 13 Mar 2025
Probabilistic Reasoning with LLMs for k-anonymity Estimation Jonathan Zheng Sauvik Das Alan Ritter Wei-ping Xu 57 0 0 12 Mar 2025
Semantic Volume: Quantifying and Detecting both External and Internal Uncertainty in LLMs Xiaomin Li Zhou Yu Ziji Zhang Yingying Zhuang Shri Kiran Srinivasan Narayanan Sadagopan Anurag Beniwal HILM 60 0 0 28 Feb 2025
Monte Carlo Temperature: a robust sampling strategy for LLM's uncertainty quantification methods Nicola Cecere Andrea Bacciu Ignacio Fernández Tobías Amin Mantrach 66 1 0 25 Feb 2025
Large Language Model Confidence Estimation via Black-Box Access Tejaswini Pedapati Amit Dhurandhar Soumya Ghosh Soham Dan P. Sattigeri 89 3 0 21 Feb 2025
Can Your Uncertainty Scores Detect Hallucinated Entity? Min-Hsuan Yeh Max Kamachee Seongheon Park Yixuan Li HILM 55 1 0 17 Feb 2025
Cost-Saving LLM Cascades with Early Abstention Michael J. Zellinger Rex Liu Matt Thomson 111 0 0 13 Feb 2025
Enhancing Hallucination Detection through Noise Injection Litian Liu Reza Pourreza Sunny Panchal Apratim Bhattacharyya Yao Qin Roland Memisevic HILM 81 3 0 06 Feb 2025
Latent Space Chain-of-Embedding Enables Output-free LLM Self-Evaluation Yiming Wang Pei Zhang Baosong Yang Derek F. Wong Rui-cang Wang LRM 50 5 0 17 Oct 2024
Functional-level Uncertainty Quantification for Calibrated Fine-tuning on LLMs Ruijia Niu D. Wu Rose Yu Yi Ma 33 1 0 09 Oct 2024
Conformal Generative Modeling with Improved Sample Efficiency through Sequential Greedy Filtering Kemal Kurniawan Bernhard Schölkopf Michael Muehlebach 35 0 0 02 Oct 2024
TrustNavGPT: Modeling Uncertainty to Improve Trustworthiness of Audio-Guided LLM-Based Robot Navigation Xingpeng Sun Yiran Zhang Xindi Tang Amrit Singh Bedi Aniket Bera 50 4 0 03 Aug 2024
Cost-Effective Hallucination Detection for LLMs Simon Valentin Jinmiao Fu Gianluca Detommaso Shaoyuan Xu Giovanni Zappella Bryan Wang HILM 42 4 0 31 Jul 2024
Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph Roman Vashurin Ekaterina Fadeeva Artem Vazhentsev Akim Tsvigun Daniil Vasilev ... Timothy Baldwin Timothy Baldwin Maxim Panov Artem Shelmanov Artem Shelmanov HILM 68 9 0 21 Jun 2024
CSS: Contrastive Semantic Similarity for Uncertainty Quantification of LLMs Shuang Ao Stefan Rueger Advaith Siddharthan 33 1 0 05 Jun 2024
Conformal Alignment: Knowing When to Trust Foundation Models with Guarantees Yu Gui Ying Jin Zhimei Ren MedIm 38 18 0 16 May 2024
BIRD: A Trustworthy Bayesian Inference Framework for Large Language Models Yu Feng Ben Zhou Weidong Lin Dan Roth 76 5 0 18 Apr 2024
Confidence Calibration and Rationalization for LLMs via Multi-Agent Deliberation Ruixin Yang Dheeraj Rajagopal S. Hayati Bin Hu Dongyeop Kang LLMAG 43 5 0 14 Apr 2024
Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward Xuan Xie Jiayang Song Zhehua Zhou Yuheng Huang Da Song Lei Ma OffRL 53 6 0 12 Apr 2024
Multicalibration for Confidence Scoring in LLMs Gianluca Detommaso Martín Bertrán Riccardo Fogliato Aaron Roth 35 12 0 06 Apr 2024
Evaluating Text-to-Image Generative Models: An Empirical Study on Human Image Synthesis Mu-Hwa Chen Yi Liu Jian Yi Changran Xu Qiuxia Lai Hongliang Wang Tsung-Yi Ho Qiang Xu EGVM 37 7 0 08 Mar 2024
Methods to Estimate Large Language Model Confidence Maia Kotelanski Robert Gallo Ashwin Nayak Thomas Savage LM&MA 24 6 0 28 Nov 2023
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models Yue Zhang Yafu Li Leyang Cui Deng Cai Lemao Liu ... Longyue Wang A. Luu Wei Bi Freda Shi Shuming Shi RALM LRM HILM 48 522 0 03 Sep 2023
MoT: Memory-of-Thought Enables ChatGPT to Self-Improve Xiaonan Li Xipeng Qiu ReLM KELM LRM AI4MH 26 32 0 09 May 2023
Post-Abstention: Towards Reliably Re-Attempting the Abstained Instances in QA Neeraj Varshney Chitta Baral 39 13 0 02 May 2023
Out-of-Distribution Detection and Selective Generation for Conditional Language Models Jie Jessie Ren Jiaming Luo Yao-Min Zhao Kundan Krishna Mohammad Saleh Balaji Lakshminarayanan Peter J. Liu OODD 75 96 0 30 Sep 2022
Re-Examining Calibration: The Case of Question Answering Chenglei Si Chen Zhao Sewon Min Jordan L. Boyd-Graber 64 30 0 25 May 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 398 8,559 0 28 Jan 2022
DEUP: Direct Epistemic Uncertainty Prediction Salem Lahlou Moksh Jain Hadi Nekoei V. Butoi Paul Bertin Jarrid Rector-Brooks Maksym Korablyov Yoshua Bengio PER UQLM UQCV UD 204 81 0 16 Feb 2021
Reducing conversational agents' overconfidence through linguistic calibration Sabrina J. Mielke Arthur Szlam Emily Dinan Y-Lan Boureau 209 154 0 30 Dec 2020
Calibration of Pre-trained Transformers Shrey Desai Greg Durrett UQLM 243 290 0 17 Mar 2020
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles Balaji Lakshminarayanan Alexander Pritzel Charles Blundell UQCV BDL 276 5,675 0 05 Dec 2016
Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning Y. Gal Zoubin Ghahramani UQCV BDL 285 9,145 0 06 Jun 2015