v1v2v3v4 (latest)

Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Models

16 July 2023

Papers citing "Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Models"

37 / 37 papers shown

Title
Learning on LLM Output Signatures for gray-box Behavior Analysis Guy Bar-Shalom Fabrizio Frasca Derek Lim Yoav Gelberg Yftah Ziser Ran El-Yaniv Gal Chechik Haggai Maron 118 0 0 18 Mar 2025
Latent Space Chain-of-Embedding Enables Output-free LLM Self-Evaluation Yiming Wang Pei Zhang Baosong Yang Derek F. Wong Rui Wang LRM 95 15 0 17 Oct 2024
Functional-level Uncertainty Quantification for Calibrated Fine-tuning on LLMs Ruijia Niu D. Wu Rose Yu Yi-An Ma 80 2 0 09 Oct 2024
LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations Hadas Orgad Michael Toker Zorik Gekhman Roi Reichart Idan Szpektor Hadas Kotek Yonatan Belinkov HILM AIFin 101 45 0 03 Oct 2024
Context-aware LLM-based Safe Control Against Latent Risks Quang Khanh Luu Xiyu Deng Anh Van Ho Yorie Nakahira 108 4 0 18 Mar 2024
Uncertainty quantification in fine-tuned LLMs using LoRA ensembles Oleksandr Balabanov Hampus Linander UQCV 99 20 0 19 Feb 2024
Frontier AI Regulation: Managing Emerging Risks to Public Safety Markus Anderljung Joslyn Barnhart Anton Korinek Jade Leung Cullen O'Keefe ... Jonas Schuett Yonadav Shavit Divya Siddarth Robert F. Trager Kevin J. Wolf SILM 86 125 0 06 Jul 2023
A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation Xiaowei Huang Wenjie Ruan Wei Huang Gao Jin Yizhen Dong ... Sihao Wu Peipei Xu Dengyu Wu André Freitas Mustafa A. Mustafa ALM 111 93 0 19 May 2023
Capabilities of GPT-4 on Medical Challenge Problems Harsha Nori Nicholas King S. McKinney Dean Carignan Eric Horvitz LM&MA ELM AI4MH 134 809 0 20 Mar 2023
How Robust is GPT-3.5 to Predecessors? A Comprehensive Study on Language Understanding Tasks Xuanting Chen Junjie Ye Can Zu Nuo Xu Rui Zheng Minlong Peng Jie Zhou Tao Gui Qi Zhang Xuanjing Huang AI4MH ELM 61 83 0 01 Mar 2023
SantaCoder: don't reach for the stars! Loubna Ben Allal Raymond Li Denis Kocetkov Chenghao Mou Christopher Akiki ... Sean M. Hughes Daniel Fried Arjun Guha H. D. Vries Leandro von Werra 166 197 0 09 Jan 2023
An Exploratory Study of AI System Risk Assessment from the Lens of Data Distribution and Uncertainty Zhijie Wang Yuheng Huang Lei Ma Haruki Yokoyama Susumu Tokumoto Kazuki Munakata 64 4 0 13 Dec 2022
A Holistic Approach to Undesired Content Detection in the Real World Todor Markov Chong Zhang Sandhini Agarwal Tyna Eloundou Teddy Lee Steven Adler Angela Jiang L. Weng 107 235 0 05 Aug 2022
InCoder: A Generative Model for Code Infilling and Synthesis Daniel Fried Armen Aghajanyan Jessy Lin Sida I. Wang Eric Wallace Freda Shi Ruiqi Zhong Wen-tau Yih Luke Zettlemoyer M. Lewis SyDa 76 652 0 12 Apr 2022
Locally Typical Sampling Clara Meister Tiago Pimentel Gian Wiher Ryan Cotterell 187 89 0 01 Feb 2022
CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation Yue Wang Weishi Wang Shafiq Joty Guosheng Lin 292 1,560 0 02 Sep 2021
Program Synthesis with Large Language Models Jacob Austin Augustus Odena Maxwell Nye Maarten Bosma Henryk Michalewski ... Ellen Jiang Carrie J. Cai Michael Terry Quoc V. Le Charles Sutton ELM AIMat ReCod ALM 203 2,004 0 16 Aug 2021
Evaluating Large Language Models Trained on Code Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Pondé ... Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever Wojciech Zaremba ELM ALM 236 5,647 0 07 Jul 2021
A Survey of Uncertainty in Deep Neural Networks J. Gawlikowski Cedrique Rovile Njieutcheu Tassi Mohsin Ali Jongseo Lee Matthias Humt ... R. Roscher Muhammad Shahzad Wen Yang R. Bamler Xiaoxiang Zhu BDL UQCV OOD 231 1,155 0 07 Jul 2021
On Hallucination and Predictive Uncertainty in Conditional Language Generation Yijun Xiao Wenjie Wang HILM 156 192 0 28 Mar 2021
Persistent Anti-Muslim Bias in Large Language Models Abubakar Abid Maheen Farooqi James Zou AILaw 108 555 0 14 Jan 2021
Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty Umang Bhatt Javier Antorán Yunfeng Zhang Q. V. Liao P. Sattigeri ... L. Nachman R. Chunara Madhulika Srikumar Adrian Weller Alice Xiang 77 250 0 15 Nov 2020
CodeBLEU: a Method for Automatic Evaluation of Code Synthesis Shuo Ren Daya Guo Shuai Lu Long Zhou Shujie Liu Duyu Tang Neel Sundaresan M. Zhou Ambrosio Blanco Shuai Ma ELM 115 542 0 22 Sep 2020
GraphCodeBERT: Pre-training Code Representations with Data Flow Daya Guo Shuo Ren Shuai Lu Zhangyin Feng Duyu Tang ... Dawn Drain Neel Sundaresan Jian Yin Daxin Jiang M. Zhou 148 1,144 0 17 Sep 2020
Captum: A unified and generic model interpretability library for PyTorch Narine Kokhlikyan Vivek Miglani Miguel Martin Edward Wang B. Alsallakh ... Alexander Melnikov Natalia Kliushkina Carlos Araya Siqi Yan Orion Reblitz-Richardson FAtt 144 846 0 16 Sep 2020
Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data Yen-Chang Hsu Yilin Shen Hongxia Jin Z. Kira OODD 102 576 0 26 Feb 2020
Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning Arsenii Ashukha Alexander Lyzhov Dmitry Molchanov Dmitry Vetrov UQCV FedML 84 320 0 15 Feb 2020
Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods Eyke Hüllermeier Willem Waegeman PER UD 244 1,421 0 21 Oct 2019
Towards Making the Most of BERT in Neural Machine Translation Jiacheng Yang Mingxuan Wang Hao Zhou Chengqi Zhao Yong Yu Weinan Zhang Lei Li CLL 54 158 0 15 Aug 2019
Optimization under Uncertainty in the Era of Big Data and Deep Learning: When Machine Learning Meets Mathematical Programming C. Ning Fengqi You OOD 43 280 0 03 Apr 2019
Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift Stephan Rabanser Stephan Günnemann Zachary Chase Lipton 61 370 0 29 Oct 2018
Classification Uncertainty of Deep Neural Networks Based on Gradient Information Philipp Oberdiek Matthias Rottmann Hanno Gottschalk UQCV 64 64 0 22 May 2018
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? Alex Kendall Y. Gal BDL OOD UD UQCV PER 362 4,719 0 15 Mar 2017
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles Balaji Lakshminarayanan Alexander Pritzel Charles Blundell UQCV BDL 842 5,841 0 05 Dec 2016
A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks Dan Hendrycks Kevin Gimpel UQCV 166 3,472 0 07 Oct 2016
On the Safety of Machine Learning: Cyber-Physical Systems, Decision Sciences, and Data Products Kush R. Varshney H. Alemzadeh 126 225 0 05 Oct 2016
Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning Y. Gal Zoubin Ghahramani UQCV BDL 838 9,346 0 06 Jun 2015