MAQA: Evaluating Uncertainty Quantification in LLMs Regarding Data Uncertainty

13 August 2024

Papers citing "MAQA: Evaluating Uncertainty Quantification in LLMs Regarding Data Uncertainty"

30 / 30 papers shown

Title
Explaining Sources of Uncertainty in Automated Fact-Checking Jingyi Sun Greta Warren Irina Shklovski Isabelle Augenstein 39 1 0 23 May 2025
Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness Khyathi Chandu Linjie Li Anas Awadalla Ximing Lu Jae Sung Park Jack Hessel Lijuan Wang Yejin Choi 79 3 0 02 Jul 2024
UBench: Benchmarking Uncertainty in Large Language Models with Multiple Choice Questions Xunzhi Wang Zhuowei Zhang Qiongyu Li Gaonan Chen Mengting Hu Zhixin Han Bitong Luo Zhiyu li Hang Gao Mengting Hu ELM 73 3 0 18 Jun 2024
Mitigating LLM Hallucinations via Conformal Abstention Yasin Abbasi-Yadkori Ilja Kuzborskij David Stutz András György Adam Fisch ... Wei-Hung Weng Yao-Yuan Yang Csaba Szepesvári A. Cemgil Nenad Tomašev HILM 67 14 0 04 Apr 2024
Calibrating Large Language Models Using Their Generations Only Dennis Ulmer Martin Gubri Hwaran Lee Sangdoo Yun Seong Joon Oh UQLM 468 24 1 09 Mar 2024
Distinguishing the Knowable from the Unknowable with Language Models Gustaf Ahdritz Tian Qin Nikhil Vyas Boaz Barak Benjamin L. Edelman 57 23 0 05 Feb 2024
Benchmarking LLMs via Uncertainty Quantification Fanghua Ye Mingming Yang Jianhui Pang Longyue Wang Derek F. Wong Emine Yilmaz Shuming Shi Zhaopeng Tu ELM 166 53 0 23 Jan 2024
Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling Bairu Hou Yujian Liu Kaizhi Qian Jacob Andreas Shiyu Chang Yang Zhang UD UQCV PER 56 57 0 15 Nov 2023
The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Models Aviv Slobodkin Omer Goldman Avi Caciularu Ido Dagan Shauli Ravfogel HILM LRM 64 27 0 18 Oct 2023
Qwen Technical Report Jinze Bai Shuai Bai Yunfei Chu Zeyu Cui Kai Dang ... Zhenru Zhang Chang Zhou Jingren Zhou Xiaohuan Zhou Tianhang Zhu OSLM 176 1,756 0 28 Sep 2023
Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs Miao Xiong Zhiyuan Hu Xinyang Lu Yifei Li Jie Fu Junxian He Bryan Hooi 155 404 0 22 Jun 2023
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 300 4,186 0 09 Jun 2023
Selectively Answering Ambiguous Questions Jeremy R. Cole Michael J.Q. Zhang D. Gillick Julian Martin Eisenschlos Bhuwan Dhingra Jacob Eisenstein UQLM 60 28 0 24 May 2023
QUEST: A Retrieval Dataset of Entity-Seeking Queries with Implicit Set Operations Chaitanya Malaviya Peter Shaw Ming-Wei Chang Kenton Lee Kristina Toutanova 86 15 0 19 May 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 916 13,788 0 15 Mar 2023
Language Models (Mostly) Know What They Know Saurav Kadavath Tom Conerly Amanda Askell T. Henighan Dawn Drain ... Nicholas Joseph Benjamin Mann Sam McCandlish C. Olah Jared Kaplan ELM 99 787 0 11 Jul 2022
Teaching Models to Express Their Uncertainty in Words Stephanie C. Lin Jacob Hilton Owain Evans OOD 55 401 0 28 May 2022
A Deeper Look into Aleatoric and Epistemic Uncertainty Disentanglement Matias Valdenegro-Toro Daniel Saromo UD PER BDL UQCV 39 81 0 20 Apr 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 732 12,525 0 04 Mar 2022
Training Verifiers to Solve Math Word Problems K. Cobbe V. Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun ... Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse John Schulman ReLM OffRL LRM 214 4,175 0 27 Oct 2021
SituatedQA: Incorporating Extra-Linguistic Contexts into QA Michael J.Q. Zhang Eunsol Choi RALM 71 142 0 13 Sep 2021
TruthfulQA: Measuring How Models Mimic Human Falsehoods Stephanie C. Lin Jacob Hilton Owain Evans HILM 97 1,825 0 08 Sep 2021
Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies Mor Geva Daniel Khashabi Elad Segal Tushar Khot Dan Roth Jonathan Berant RALM 311 697 0 06 Jan 2021
Measuring Massive Multitask Language Understanding Dan Hendrycks Collin Burns Steven Basart Andy Zou Mantas Mazeika D. Song Jacob Steinhardt ELM RALM 155 4,222 0 07 Sep 2020
Unsupervised Quality Estimation for Neural Machine Translation M. Fomicheva Shuo Sun Lisa Yankovskaya Frédéric Blain Francisco Guzmán Mark Fishel Nikolaos Aletras Vishrav Chaudhary Lucia Specia UQLM 80 198 0 21 May 2020
ProtoQA: A Question Answering Dataset for Prototypical Common-Sense Reasoning Michael Boratko Xiang Lorraine Li Rajarshi Das Timothy J. O'Gorman Daniel Le Andrew McCallum 63 56 0 02 May 2020
AmbigQA: Answering Ambiguous Open-domain Questions Sewon Min Julian Michael Hannaneh Hajishirzi Luke Zettlemoyer 86 302 0 22 Apr 2020
Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods Eyke Hüllermeier Willem Waegeman PER UD 191 1,388 0 21 Oct 2019
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension Mandar Joshi Eunsol Choi Daniel S. Weld Luke Zettlemoyer RALM 190 2,610 0 09 May 2017
Reading Wikipedia to Answer Open-Domain Questions Danqi Chen Adam Fisch Jason Weston Antoine Bordes RALM 108 2,003 0 31 Mar 2017