Navigating the Grey Area: How Expressions of Uncertainty and
Overconfidence Affect Language Models

v1v2 (latest)

Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models

26 February 2023

Dan Jurafsky

Tatsunori Hashimoto

ArXiv (abs)PDF HTML

Papers citing "Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models"

16 / 16 papers shown

Title
Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinations Ziwei Ji L. Yu Yeskendir Koishekenov Yejin Bang Anthony Hartshorn Alan Schelten Cheng Zhang Pascale Fung Nicola Cancedda 77 5 0 18 Mar 2025
UAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language Models Boyang Xue Fei Mi Qi Zhu Hongru Wang Rui Wang Sheng Wang Erxin Yu Xuming Hu Kam-Fai Wong HILM 187 2 0 16 Dec 2024
Teaching LLMs to Abstain across Languages via Multilingual Feedback Shangbin Feng Weijia Shi Yike Wang Wenxuan Ding Orevaoghene Ahia Shuyue Stella Li Vidhisha Balachandran Sunayana Sitaram Yulia Tsvetkov 111 7 0 22 Jun 2024
"You might think about slightly revising the title": identifying hedges in peer-tutoring interactions Yann Raphalen Chloé Clavel Justine Cassell 23 13 0 18 Jun 2023
Demystifying Prompts in Language Models via Perplexity Estimation Hila Gonen Srini Iyer Terra Blevins Noah A. Smith Luke Zettlemoyer LRM 119 210 0 08 Dec 2022
Calibrated Interpretation: Confidence Estimation in Semantic Parsing Elias Stengel-Eskin Benjamin Van Durme UQLM 66 25 0 14 Nov 2022
Language Models (Mostly) Know What They Know Saurav Kadavath Tom Conerly Amanda Askell T. Henighan Dawn Drain ... Nicholas Joseph Benjamin Mann Sam McCandlish C. Olah Jared Kaplan ELM 117 826 0 11 Jul 2022
Prompt-and-Rerank: A Method for Zero-Shot and Few-Shot Arbitrary Textual Style Transfer with Small Language Models Mirac Suzgun Luke Melas-Kyriazi Dan Jurafsky VLM 109 66 0 23 May 2022
He Thinks He Knows Better than the Doctors: BERT for Event Factuality Fails on Pragmatics Nan-Jiang Jiang M. Marneffe 60 21 0 02 Jul 2021
The Possible, the Plausible, and the Desirable: Event-Based Modality Detection for Language Processing Valentina Pyatkin Shoval Sadde Aynat Rubinstein P. Portner Reut Tsarfaty 64 18 0 15 Jun 2021
Revisiting the Calibration of Modern Neural Networks Matthias Minderer Josip Djolonga Rob Romijnders F. Hubis Xiaohua Zhai N. Houlsby Dustin Tran Mario Lucic UQCV 98 367 0 15 Jun 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 450 2,096 0 31 Dec 2020
On the Inference Calibration of Neural Machine Translation Shuo Wang Zhaopeng Tu Shuming Shi Yang Liu 81 82 0 03 May 2020
Calibration of Pre-trained Transformers Shrey Desai Greg Durrett UQLM 288 300 0 17 Mar 2020
Automatically Neutralizing Subjective Bias in Text Reid Pryzant Richard Diehl Martinez Nathan Dass Sadao Kurohashi Dan Jurafsky Diyi Yang 79 178 0 21 Nov 2019
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension Mandar Joshi Eunsol Choi Daniel S. Weld Luke Zettlemoyer RALM 207 2,676 0 09 May 2017