Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysis

10 October 2022

Louis-Philippe Morency

ArXiv PDF HTML

Papers citing "Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysis"

50 / 57 papers shown

Title
Large Language Model Confidence Estimation via Black-Box Access Tejaswini Pedapati Amit Dhurandhar Soumya Ghosh Soham Dan P. Sattigeri 198 5 0 21 Feb 2025
UAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language Models Boyang Xue Fei Mi Qi Zhu Hongru Wang Rui Wang Sheng Wang Erxin Yu Xuming Hu Kam-Fai Wong HILM 174 2 0 16 Dec 2024
Taming Overconfidence in LLMs: Reward Calibration in RLHF Jixuan Leng Chengsong Huang Banghua Zhu Jiaxin Huang 76 14 0 13 Oct 2024
Functional-level Uncertainty Quantification for Calibrated Fine-tuning on LLMs Ruijia Niu D. Wu Rose Yu Yi-An Ma 65 2 0 09 Oct 2024
Efficient Nearest Neighbor based Uncertainty Estimation for Natural Language Processing Tasks Wataru Hashimoto Hidetaka Kamigaito Taro Watanabe 85 0 0 02 Jul 2024
Uncertainty quantification in fine-tuned LLMs using LoRA ensembles Oleksandr Balabanov Hampus Linander UQCV 88 18 0 19 Feb 2024
Introspective Planning: Aligning Robots' Uncertainty with Inherent Task Ambiguity Kaiqu Liang Zixu Zhang J. F. Fisac LLMAG 94 8 0 09 Feb 2024
Investigating Selective Prediction Approaches Across Several Tasks in IID, OOD, and Adversarial Settings Neeraj Varshney Swaroop Mishra Chitta Baral 68 56 0 01 Mar 2022
Diverse, Global and Amortised Counterfactual Explanations for Uncertainty Estimates Dan Ley Umang Bhatt Adrian Weller UQCV 202 22 0 05 Dec 2021
Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey Bonan Min Hayley L Ross Elior Sulem Amir Pouran Ben Veyseh Thien Huu Nguyen Oscar Sainz Eneko Agirre Ilana Heinz Dan Roth LM&MA VLM AI4CE 107 1,065 0 01 Nov 2021
SAIS: Supervising and Augmenting Intermediate Steps for Document-Level Relation Extraction Yuxin Xiao Zecheng Zhang Yuning Mao Carl Yang Jiawei Han RALM AI4TS 47 47 0 24 Sep 2021
SMedBERT: A Knowledge-Enhanced Pre-trained Language Model with Structured Semantics for Medical Text Mining Taolin Zhang Zerui Cai Chengyu Wang Minghui Qiu Bite Yang Xiaofeng He AI4MH 51 52 0 20 Aug 2021
Revisiting the Calibration of Modern Neural Networks Matthias Minderer Josip Djolonga Rob Romijnders F. Hubis Xiaohua Zhai N. Houlsby Dustin Tran Mario Lucic UQCV 96 365 0 15 Jun 2021
On Calibration and Out-of-domain Generalization Yoav Wald Amir Feder D. Greenfeld Uri Shalit OODD 56 155 0 20 Feb 2021
Improving model calibration with accuracy versus uncertainty optimization R. Krishnan Omesh Tickoo UQCV 230 163 0 14 Dec 2020
Beyond Pinball Loss: Quantile Methods for Calibrated Uncertainty Quantification Youngseog Chung Willie Neiswanger I. Char J. Schneider UQCV 155 88 0 18 Nov 2020
Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty Umang Bhatt Javier Antorán Yunfeng Zhang Q. V. Liao P. Sattigeri ... L. Nachman R. Chunara Madhulika Srikumar Adrian Weller Alice Xiang 62 248 0 15 Nov 2020
A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges Moloud Abdar Farhad Pourpanah Sadiq Hussain Dana Rezazadegan Li Liu ... Xiaochun Cao Abbas Khosravi U. Acharya V. Makarenkov S. Nahavandi BDL UQCV 276 1,914 0 12 Nov 2020
A Survey on Contrastive Self-supervised Learning Ashish Jaiswal Ashwin Ramesh Babu Mohammad Zaki Zadeh Debapriya Banerjee F. Makedon SSL 117 1,389 0 31 Oct 2020
Calibrated Language Model Fine-Tuning for In- and Out-of-Distribution Data Lingkai Kong Haoming Jiang Yuchen Zhuang Jie Lyu T. Zhao Chao Zhang OODD 59 26 0 22 Oct 2020
Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling Wenxuan Zhou Kevin Huang Tengyu Ma Jing Huang 68 276 0 21 Oct 2020
Uncertainty-Aware Semantic Augmentation for Neural Machine Translation Xiangpeng Wei Heng Yu Yue Hu Rongxiang Weng Luxi Xing Weihua Luo UQLM BDL 45 22 0 09 Oct 2020
Towards Improving Selective Prediction Ability of NLP Systems Neeraj Varshney Swaroop Mishra Chitta Baral 37 23 0 21 Aug 2020
Improving Calibration through the Relationship with Adversarial Robustness Yao Qin Xuezhi Wang Alex Beutel Ed H. Chi AAML 54 25 0 29 Jun 2020
Discriminative Jackknife: Quantifying Uncertainty in Deep Learning via Higher-Order Influence Functions Ahmed Alaa M. Schaar UD UQCV BDL TDI 57 53 0 29 Jun 2020
Frequentist Uncertainty in Recurrent Neural Networks via Blockwise Influence Functions Ahmed Alaa M. Schaar UQCV BDL 50 23 0 20 Jun 2020
Calibrated Reliable Regression using Maximum Mean Discrepancy Peng Cui Wenbo Hu Jun Zhu UQCV 45 47 0 18 Jun 2020
Getting a CLUE: A Method for Explaining Uncertainty Estimates Javier Antorán Umang Bhatt T. Adel Adrian Weller José Miguel Hernández-Lobato UQCV BDL 75 116 0 11 Jun 2020
DeBERTa: Decoding-enhanced BERT with Disentangled Attention Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen AAML 132 2,730 0 05 Jun 2020
Pretrained Transformers Improve Out-of-Distribution Robustness Dan Hendrycks Xiaoyuan Liu Eric Wallace Adam Dziedzic R. Krishnan D. Song OOD 175 434 0 13 Apr 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 348 1,482 0 18 Mar 2020
Calibration of Pre-trained Transformers Shrey Desai Greg Durrett UQLM 283 300 0 17 Mar 2020
Calibrating Deep Neural Networks using Focal Loss Jishnu Mukhoti Viveka Kulharia Amartya Sanyal Stuart Golodetz Philip Torr P. Dokania UQCV 81 459 0 21 Feb 2020
TANDA: Transfer and Adapt Pre-Trained Transformer Models for Answer Sentence Selection Siddhant Garg Thuy Vu Alessandro Moschitti 65 215 0 11 Nov 2019
Pre-trained Language Model for Biomedical Question Answering Wonjin Yoon Jinhyuk Lee Donghyeon Kim Minbyul Jeong Jaewoo Kang AI4MH 39 86 0 18 Sep 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 541 24,422 0 26 Jul 2019
XLNet: Generalized Autoregressive Pretraining for Language Understanding Zhilin Yang Zihang Dai Yiming Yang J. Carbonell Ruslan Salakhutdinov Quoc V. Le AI4CE 220 8,424 0 19 Jun 2019
When Does Label Smoothing Help? Rafael Müller Simon Kornblith Geoffrey E. Hinton UQCV 183 1,943 0 06 Jun 2019
Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift Yaniv Ovadia Emily Fertig Jie Jessie Ren Zachary Nado D. Sculley Sebastian Nowozin Joshua V. Dillon Balaji Lakshminarayanan Jasper Snoek UQCV 159 1,691 0 06 Jun 2019
HellaSwag: Can a Machine Really Finish Your Sentence? Rowan Zellers Ari Holtzman Yonatan Bisk Ali Farhadi Yejin Choi 156 2,464 0 19 May 2019
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems Alex Jinpeng Wang Yada Pruksachatkun Nikita Nangia Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 252 2,307 0 02 May 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.6K 94,729 0 11 Oct 2018
SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference Rowan Zellers Yonatan Bisk Roy Schwartz Yejin Choi 98 718 0 16 Aug 2018
Accurate Uncertainties for Deep Learning Using Calibrated Regression Volodymyr Kuleshov Nathan Fenner Stefano Ermon BDL UQCV 189 632 0 01 Jul 2018
Know What You Don't Know: Unanswerable Questions for SQuAD Pranav Rajpurkar Robin Jia Percy Liang RALM ELM 261 2,837 0 11 Jun 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.0K 7,152 0 20 Apr 2018
Analyzing Uncertainty in Neural Machine Translation Myle Ott Michael Auli David Grangier MarcÁurelio Ranzato UQLM 88 273 0 28 Feb 2018
On Calibration of Modern Neural Networks Chuan Guo Geoff Pleiss Yu Sun Kilian Q. Weinberger UQCV 287 5,825 0 14 Jun 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 656 131,414 0 12 Jun 2017
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour Priya Goyal Piotr Dollár Ross B. Girshick P. Noordhuis Lukasz Wesolowski Aapo Kyrola Andrew Tulloch Yangqing Jia Kaiming He 3DH 120 3,678 0 08 Jun 2017