v1v2 (latest)

Reconfidencing LLMs from the Grouping Loss Perspective

7 February 2024

Lihu Chen

Alexandre Perez-Lebel

Fabian M. Suchanek

Gaël Varoquaux

ArXiv (abs)PDF HTML

Papers citing "Reconfidencing LLMs from the Grouping Loss Perspective"

26 / 26 papers shown

Title
Graph-based Confidence Calibration for Large Language Models Yukun Li Sijia Wang Lifu Huang Li-Ping Liu UQCV 162 2 0 03 Nov 2024
Taming Overconfidence in LLMs: Reward Calibration in RLHF Jixuan Leng Chengsong Huang Banghua Zhu Jiaxin Huang 104 16 0 13 Oct 2024
Uncertainty in Natural Language Generation: From Theory to Applications Joris Baan Nico Daheim Evgenia Ilia Dennis Ulmer Haau-Sing Li Raquel Fernández Barbara Plank Rico Sennrich Chrysoula Zerva Wilker Aziz UQLM 129 45 0 28 Jul 2023
Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs Miao Xiong Zhiyuan Hu Xinyang Lu Yifei Li Jie Fu Junxian He Bryan Hooi 208 447 0 22 Jun 2023
Do Language Models Know When They're Hallucinating References? A. Agrawal Mirac Suzgun Lester W. Mackey Adam Tauman Kalai HILM LRM 112 99 0 29 May 2023
Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback Katherine Tian E. Mitchell Allan Zhou Archit Sharma Rafael Rafailov Huaxiu Yao Chelsea Finn Christopher D. Manning 126 354 0 24 May 2023
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing Zhibin Gou Zhihong Shao Yeyun Gong Yelong Shen Yujiu Yang Nan Duan Weizhu Chen KELM LRM 81 394 0 19 May 2023
The Internal State of an LLM Knows When It's Lying A. Azaria Tom Michael Mitchell HILM 285 344 0 26 Apr 2023
LLaMA: Open and Efficient Foundation Language Models Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux ... Faisal Azhar Aurelien Rodriguez Armand Joulin Edouard Grave Guillaume Lample ALM PILM 1.5K 13,437 0 27 Feb 2023
Beyond calibration: estimating the grouping loss of modern neural networks Alexandre Perez-Lebel Marine Le Morvan Gaël Varoquaux UQCV 55 27 0 28 Oct 2022
Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysis Yuxin Xiao Paul Pu Liang Umang Bhatt Willie Neiswanger Ruslan Salakhutdinov Louis-Philippe Morency 222 96 0 10 Oct 2022
Language Models (Mostly) Know What They Know Saurav Kadavath Tom Conerly Amanda Askell T. Henighan Dawn Drain ... Nicholas Joseph Benjamin Mann Sam McCandlish C. Olah Jared Kaplan ELM 122 830 0 11 Jul 2022
Teaching Models to Express Their Uncertainty in Words Stephanie C. Lin Jacob Hilton Owain Evans OOD 94 423 0 28 May 2022
On the Calibration of Pre-trained Language Models using Mixup Guided by Area Under the Margin and Saliency Seohong Park Cornelia Caragea UQCV 49 36 0 14 Mar 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 886 13,176 0 04 Mar 2022
TruthfulQA: Measuring How Models Mimic Human Falsehoods Stephanie C. Lin Jacob Hilton Owain Evans HILM 149 1,938 0 08 Sep 2021
A Survey of Uncertainty in Deep Neural Networks J. Gawlikowski Cedrique Rovile Njieutcheu Tassi Mohsin Ali Jongseo Lee Matthias Humt ... R. Roscher Muhammad Shahzad Wen Yang R. Bamler Xiaoxiang Zhu BDL UQCV OOD 233 1,155 0 07 Jul 2021
Reducing conversational agents' overconfidence through linguistic calibration Sabrina J. Mielke Arthur Szlam Emily Dinan Y-Lan Boureau 280 169 0 30 Dec 2020
How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering Zhengbao Jiang Jun Araki Haibo Ding Graham Neubig UQCV 60 436 0 02 Dec 2020
Measuring Massive Multitask Language Understanding Dan Hendrycks Collin Burns Steven Basart Andy Zou Mantas Mazeika Basel Alomair Jacob Steinhardt ELM RALM 184 4,553 0 07 Sep 2020
DeBERTa: Decoding-enhanced BERT with Disentangled Attention Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen AAML 165 2,750 0 05 Jun 2020
CoQA: A Conversational Question Answering Challenge Siva Reddy Danqi Chen Christopher D. Manning RALM HAI 114 1,210 0 21 Aug 2018
Crowdsourcing Multiple Choice Science Questions Johannes Welbl Nelson F. Liu Matt Gardner AI4Ed 104 517 0 19 Jul 2017
On Calibration of Modern Neural Networks Chuan Guo Geoff Pleiss Yu Sun Kilian Q. Weinberger UQCV 299 5,862 0 14 Jun 2017
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension Mandar Joshi Eunsol Choi Daniel S. Weld Luke Zettlemoyer RALM 228 2,686 0 09 May 2017
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference Adina Williams Nikita Nangia Samuel R. Bowman 524 4,494 0 18 Apr 2017