Unconditional Truthfulness: Learning Conditional Dependency for
Uncertainty Quantification of Large Language Models

Unconditional Truthfulness: Learning Conditional Dependency for Uncertainty Quantification of Large Language Models

20 August 2024

Artem Vazhentsev

Ekaterina Fadeeva

Alexander Panchenko

Preslav Nakov

Timothy Baldwin

Artem Shelmanov

ArXiv (abs)PDF HTML

Papers citing "Unconditional Truthfulness: Learning Conditional Dependency for Uncertainty Quantification of Large Language Models"

16 / 16 papers shown

Title
Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities Alexander Nikitin Jannik Kossen Yarin Gal Pekka Marttinen UQCV 119 43 0 30 May 2024
LM-Polygraph: Uncertainty Estimation for Language Models Ekaterina Fadeeva Roman Vashurin Akim Tsvigun Artem Vazhentsev Sergey Petrakov ... Elizaveta Goncharova Alexander Panchenko Maxim Panov Timothy Baldwin Artem Shelmanov 53 67 0 13 Nov 2023
Density Uncertainty Layers for Reliable Uncertainty Estimation Yookoon Park David M. Blei UQCV BDL 40 2 0 21 Jun 2023
FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation Sewon Min Kalpesh Krishna Xinxi Lyu M. Lewis Wen-tau Yih Pang Wei Koh Mohit Iyyer Luke Zettlemoyer Hannaneh Hajishirzi HILM ALM 128 684 0 23 May 2023
The Internal State of an LLM Knows When It's Lying A. Azaria Tom Michael Mitchell HILM 267 340 0 26 Apr 2023
Mutual Information Alleviates Hallucinations in Abstractive Summarization Liam van der Poel Ryan Cotterell Clara Meister HILM 68 60 0 24 Oct 2022
Out-of-Distribution Detection and Selective Generation for Conditional Language Models Jie Jessie Ren Jiaming Luo Yao-Min Zhao Kundan Krishna Mohammad Saleh Balaji Lakshminarayanan Peter J. Liu OODD 113 110 0 30 Sep 2022
Language Models (Mostly) Know What They Know Saurav Kadavath Tom Conerly Amanda Askell T. Henighan Dawn Drain ... Nicholas Joseph Benjamin Mann Sam McCandlish C. Olah Jared Kaplan ELM 117 826 0 11 Jul 2022
Unsupervised Quality Estimation for Neural Machine Translation M. Fomicheva Shuo Sun Lisa Yankovskaya Frédéric Blain Francisco Guzmán Mark Fishel Nikolaos Aletras Vishrav Chaudhary Lucia Specia UQLM 85 206 0 21 May 2020
SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization Bogdan Gliwa Iwona Mochol M. Biesek A. Wawer 124 631 0 27 Nov 2019
PubMedQA: A Dataset for Biomedical Research Question Answering Qiao Jin Bhuwan Dhingra Zhengping Liu William W. Cohen Xinghua Lu 379 896 0 13 Sep 2019
Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization Shashi Narayan Shay B. Cohen Mirella Lapata AILaw 126 1,676 0 27 Aug 2018
CoQA: A Conversational Question Answering Challenge Siva Reddy Danqi Chen Christopher D. Manning RALM HAI 98 1,205 0 21 Aug 2018
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension Mandar Joshi Eunsol Choi Daniel S. Weld Luke Zettlemoyer RALM 207 2,676 0 09 May 2017
Get To The Point: Summarization with Pointer-Generator Networks A. See Peter J. Liu Christopher D. Manning 3DPC 298 4,019 0 14 Apr 2017
Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning Y. Gal Zoubin Ghahramani UQCV BDL 821 9,318 0 06 Jun 2015