Calibrating Language Models with Adaptive Temperature Scaling

Calibrating Language Models with Adaptive Temperature Scaling

29 September 2024

Eric Mitchell

Chelsea Finn

ArXiv (abs)PDF HTML

Papers citing "Calibrating Language Models with Adaptive Temperature Scaling"

16 / 16 papers shown

Title
Large Language Models Do Multi-Label Classification Differently Marcus Ma Georgios Chochlakis Niyantha Maruthu Pandiyan Jesse Thomason Shrikanth Narayanan 96 1 0 23 May 2025
Optimizing Temperature for Language Models with Multi-Sample Inference Weihua Du Yiming Yang Sean Welleck 136 4 0 07 Feb 2025
Optical aberrations in autonomous driving: Physics-informed parameterized temperature scaling for neural network uncertainty calibration D. Wolf Alexander Braun Markus Ulrich 266 0 0 18 Dec 2024
Qwen Technical Report Jinze Bai Shuai Bai Yunfei Chu Zeyu Cui Kai Dang ... Zhenru Zhang Chang Zhou Jingren Zhou Xiaohuan Zhou Tianhang Zhu OSLM 268 1,908 0 28 Sep 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 1.5K 14,761 0 15 Mar 2023
Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysis Yuxin Xiao Paul Pu Liang Umang Bhatt Willie Neiswanger Ruslan Salakhutdinov Louis-Philippe Morency 224 96 0 10 Oct 2022
Language Models (Mostly) Know What They Know Saurav Kadavath Tom Conerly Amanda Askell T. Henighan Dawn Drain ... Nicholas Joseph Benjamin Mann Sam McCandlish C. Olah Jared Kaplan ELM 133 833 0 11 Jul 2022
Teaching Models to Express Their Uncertainty in Words Stephanie C. Lin Jacob Hilton Owain Evans OOD 96 425 0 28 May 2022
Measuring Robustness to Natural Distribution Shifts in Image Classification Rohan Taori Achal Dave Vaishaal Shankar Nicholas Carlini Benjamin Recht Ludwig Schmidt OOD 126 548 0 01 Jul 2020
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization Dan Hendrycks Steven Basart Norman Mu Saurav Kadavath Frank Wang ... Samyak Parajuli Mike Guo Basel Alomair Jacob Steinhardt Justin Gilmer OOD 363 1,757 0 29 Jun 2020
Verified Uncertainty Calibration Ananya Kumar Percy Liang Tengyu Ma 185 359 0 23 Sep 2019
When Does Label Smoothing Help? Rafael Müller Simon Kornblith Geoffrey E. Hinton UQCV 212 1,958 0 06 Jun 2019
Deep Anomaly Detection with Outlier Exposure Dan Hendrycks Mantas Mazeika Thomas G. Dietterich OODD 183 1,487 0 11 Dec 2018
ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness Robert Geirhos Patricia Rubisch Claudio Michaelis Matthias Bethge Felix Wichmann Wieland Brendel 143 2,676 0 29 Nov 2018
On Calibration of Modern Neural Networks Chuan Guo Geoff Pleiss Yu Sun Kilian Q. Weinberger UQCV 299 5,877 0 14 Jun 2017
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension Mandar Joshi Eunsol Choi Daniel S. Weld Luke Zettlemoyer RALM 237 2,692 0 09 May 2017