v1v2 (latest)

Revisiting Uncertainty Quantification Evaluation in Language Models: Spurious Interactions with Response Length Bias Results

18 April 2025

Papers citing "Revisiting Uncertainty Quantification Evaluation in Language Models: Spurious Interactions with Response Length Bias Results"

25 / 25 papers shown

Title
Self-reflective Uncertainties: Do LLMs Know Their Internal Answer Distribution? Michael Kirchhof Luca Füger Adam Goliñski Eeshan Gunesh Dhekane Arno Blaas Sinead Williamson 37 1 0 26 May 2025
Measuring short-form factuality in large language models Jason W. Wei Nguyen Karina Hyung Won Chung Yunxin Joy Jiao Spencer Papay Amelia Glaese John Schulman W. Fedus ELM KELM HILM 68 78 0 07 Nov 2024
LoGU: Long-form Generation with Uncertainty Expressions Ruihan Yang Caiqi Zhang Zhisong Zhang Xinting Huang Sen Yang Nigel Collier Dong Yu Deqing Yang HILM 228 9 0 18 Oct 2024
LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks A. Bavaresco Raffaella Bernardi Leonardo Bertolazzi Desmond Elliott Raquel Fernández ... David Schlangen Alessandro Suglia Aditya K Surikuchi Ece Takmaz A. Testoni ALM ELM 141 88 0 26 Jun 2024
Linguistic Calibration of Long-Form Generations Wei He Xuechen Li Tengyu Ma Aline Villavicencio 98 27 0 30 Mar 2024
Humans or LLMs as the Judge? A Study on Judgement Biases Guiming Hardy Chen Shunian Chen Ziche Liu Feng Jiang Benyou Wang 169 112 0 16 Feb 2024
INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection Chao Chen Kai-Chun Liu Ze Chen Yi Gu Yue-bo Wu Mingyuan Tao Zhihang Fu Jieping Ye HILM 126 110 0 06 Feb 2024
LM-Polygraph: Uncertainty Estimation for Language Models Ekaterina Fadeeva Roman Vashurin Akim Tsvigun Artem Vazhentsev Sergey Petrakov ... Elizaveta Goncharova Alexander Panchenko Maxim Panov Timothy Baldwin Artem Shelmanov 57 68 0 13 Nov 2023
Uncertainty in Natural Language Generation: From Theory to Applications Joris Baan Nico Daheim Evgenia Ilia Dennis Ulmer Haau-Sing Li Raquel Fernández Barbara Plank Rico Sennrich Chrysoula Zerva Wilker Aziz UQLM 131 45 0 28 Jul 2023
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 441 4,444 0 09 Jun 2023
Large Language Models are not Fair Evaluators Peiyi Wang Lei Li Liang Chen Zefan Cai Dawei Zhu Binghuai Lin Yunbo Cao Qi Liu Tianyu Liu Zhifang Sui ALM 132 575 0 29 May 2023
AlignScore: Evaluating Factual Consistency with a Unified Alignment Function Yuheng Zha Yichi Yang Ruichen Li Zhiting Hu HILM 90 208 0 26 May 2023
Hallucinations in Large Multilingual Translation Models Nuno M. Guerreiro Duarte M. Alves Jonas Waldendorf Barry Haddow Alexandra Birch Pierre Colombo André F.T. Martins VLM HILM LRM 190 152 0 28 Mar 2023
Evaluate & Evaluation on the Hub: Better Best Practices for Data and Model Measurements Leandro von Werra Lewis Tunstall A. Thakur A. Luccioni Tristan Thrush ... Julien Chaumond Margaret Mitchell Alexander M. Rush Thomas Wolf Douwe Kiela ELM 96 26 0 30 Sep 2022
Language Models (Mostly) Know What They Know Saurav Kadavath Tom Conerly Amanda Askell T. Henighan Dawn Drain ... Nicholas Joseph Benjamin Mann Sam McCandlish C. Olah Jared Kaplan ELM 127 833 0 11 Jul 2022
On Hallucination and Predictive Uncertainty in Conditional Language Generation Yijun Xiao Wenjie Wang HILM 159 192 0 28 Mar 2021
SummEval: Re-evaluating Summarization Evaluation Alexander R. Fabbri Wojciech Kry'sciñski Bryan McCann Caiming Xiong R. Socher Dragomir R. Radev HILM 100 722 0 24 Jul 2020
Unsupervised Quality Estimation for Neural Machine Translation M. Fomicheva Shuo Sun Lisa Yankovskaya Frédéric Blain Francisco Guzmán Mark Fishel Nikolaos Aletras Vishrav Chaudhary Lucia Specia UQLM 87 209 0 21 May 2020
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Nils Reimers Iryna Gurevych 1.3K 12,316 0 27 Aug 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 689 24,557 0 26 Jul 2019
Latent Retrieval for Weakly Supervised Open Domain Question Answering Kenton Lee Ming-Wei Chang Kristina Toutanova RALM 119 1,018 0 01 Jun 2019
BERTScore: Evaluating Text Generation with BERT Tianyi Zhang Varsha Kishore Felix Wu Kilian Q. Weinberger Yoav Artzi 357 5,872 0 21 Apr 2019
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension Mandar Joshi Eunsol Choi Daniel S. Weld Luke Zettlemoyer RALM 237 2,692 0 09 May 2017
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 316 8,174 0 16 Jun 2016
Revisiting Summarization Evaluation for Scientific Articles Arman Cohan Nazli Goharian 59 81 0 01 Apr 2016