Pretrained LLMs Learn Multiple Types of Uncertainty

27 May 2025

Papers citing "Pretrained LLMs Learn Multiple Types of Uncertainty"

50 / 58 papers shown

Title
I Don't Know: Explicit Modeling of Uncertainty with an [IDK] Token Roi Cohen Konstantin Dobler Eden Biran Gerard de Melo 163 9 0 09 Dec 2024
Large Language Models Must Be Taught to Know What They Don't Know Sanyam Kapoor Nate Gruver Manley Roberts Katherine Collins Arka Pal Umang Bhatt Adrian Weller Samuel Dooley Micah Goldblum Andrew Gordon Wilson 87 25 0 12 Jun 2024
Mechanistic Understanding and Mitigation of Language Model Non-Factual Hallucinations Lei Yu Meng Cao Jackie Chi Kit Cheung Yue Dong HILM 70 14 0 27 Mar 2024
On the Challenges and Opportunities in Generative AI Laura Manduchi Kushagra Pandey Robert Bamler Ryan Cotterell Sina Daubener ... F. Wenzel Frank Wood Stephan Mandt Vincent Fortuin Vincent Fortuin 263 21 0 28 Feb 2024
Narrowing the Knowledge Evaluation Gap: Open-Domain Question Answering with Multi-Granularity Answers G. Yona Roee Aharoni Mor Geva ELM 64 14 0 09 Jan 2024
R-Tuning: Instructing Large Language Models to Say `I Don't Know' Hanning Zhang Shizhe Diao Yong Lin Yi R. Fung Qing Lian Xingyao Wang Yangyi Chen Heng Ji Tong Zhang UQLM 90 46 0 16 Nov 2023
Characterizing Mechanisms for Factual Recall in Language Models Qinan Yu Jack Merullo Ellie Pavlick KELM 91 29 0 24 Oct 2023
Mistral 7B Albert Q. Jiang Alexandre Sablayrolles A. Mensch Chris Bamford Devendra Singh Chaplot ... Teven Le Scao Thibaut Lavril Thomas Wang Timothée Lacroix William El Sayed MoE LRM 79 2,229 0 10 Oct 2023
Qwen Technical Report Jinze Bai Shuai Bai Yunfei Chu Zeyu Cui Kai Dang ... Zhenru Zhang Chang Zhou Jingren Zhou Xiaohuan Zhou Tianhang Zhu OSLM 262 1,827 0 28 Sep 2023
Instruction Tuning for Large Language Models: A Survey Shengyu Zhang Linfeng Dong Xiaoya Li Sen Zhang Xiaofei Sun ... Jiwei Li Runyi Hu Tianwei Zhang Leilei Gan Guoyin Wang LM&MA 85 597 0 21 Aug 2023
Large Language Models and Knowledge Graphs: Opportunities and Challenges Jeff Z. Pan Simon Razniewski Jan-Christoph Kalo Sneha Singhania Jiaoyan Chen ... Gerard de Melo A. Bonifati Edlira Vakaj M. Dragoni D. Graux KELM 75 77 0 11 Aug 2023
Evaluating the Ripple Effects of Knowledge Editing in Language Models Roi Cohen Eden Biran Ori Yoran Amir Globerson Mor Geva KELM 78 177 0 24 Jul 2023
Do Large Language Models Know What They Don't Know? Zhangyue Yin Qiushi Sun Qipeng Guo Jiawen Wu Xipeng Qiu Xuanjing Huang ELM AI4MH 73 162 0 29 May 2023
Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigation Niels Mündler Jingxuan He Slobodan Jenko Martin Vechev HILM 62 118 0 25 May 2023
LM vs LM: Detecting Factual Errors via Cross Examination Roi Cohen May Hamri Mor Geva Amir Globerson HILM 101 138 0 22 May 2023
Automatic Evaluation of Attribution by Large Language Models Xiang Yue Boshi Wang Ziru Chen Kai Zhang Yu-Chuan Su Huan Sun ALM LRM HILM 70 56 0 10 May 2023
Finding Neurons in a Haystack: Case Studies with Sparse Probing Wes Gurnee Neel Nanda Matthew Pauly Katherine Harvey Dmitrii Troitskii Dimitris Bertsimas MILM 218 215 0 02 May 2023
Dissecting Recall of Factual Associations in Auto-Regressive Language Models Mor Geva Jasmijn Bastings Katja Filippova Amir Globerson KELM 252 318 0 28 Apr 2023
The Internal State of an LLM Knows When It's Lying A. Azaria Tom Michael Mitchell HILM 269 344 0 26 Apr 2023
Answering Questions by Meta-Reasoning over Multiple Chains of Thought Ori Yoran Tomer Wolfson Ben Bogin Uri Katz Daniel Deutch Jonathan Berant ReLM LRM KELM 68 99 0 25 Apr 2023
CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X Qinkai Zheng Xiao Xia Xu Zou Yuxiao Dong Shanshan Wang ... Andi Wang Yang Li Teng Su Zhilin Yang Jie Tang ELM ALM SyDa 118 338 0 30 Mar 2023
LLaMA: Open and Efficient Foundation Language Models Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux ... Faisal Azhar Aurelien Rodriguez Armand Joulin Edouard Grave Guillaume Lample ALM PILM 1.5K 13,247 0 27 Feb 2023
Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation Lorenz Kuhn Y. Gal Sebastian Farquhar UQLM 192 299 0 19 Feb 2023
Crawling the Internal Knowledge-Base of Language Models Roi Cohen Mor Geva Jonathan Berant Amir Globerson 212 84 0 30 Jan 2023
Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models Bernd Bohnet Vinh Q. Tran Pat Verga Roee Aharoni D. Andor ... Michael Collins Dipanjan Das Donald Metzler Slav Petrov Kellie Webster 98 64 0 15 Dec 2022
Evaluating the Factual Consistency of Large Language Models Through News Summarization Derek Tam Anisha Mascarenhas Shiyue Zhang Sarah Kwan Joey Tianyi Zhou Colin Raffel HILM 64 105 0 15 Nov 2022
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model BigScience Workshop : Teven Le Scao Angela Fan Christopher Akiki ... Zhongli Xie Zifan Ye M. Bras Younes Belkada Thomas Wolf VLM 392 2,388 0 09 Nov 2022
Prompting as Probing: Using Language Models for Knowledge Base Construction Dimitrios Alivanistos Selene Báez Santamaría Michael Cochez Jan-Christoph Kalo Emile van Krieken Thiviyan Thanapalasingam KELM 69 48 0 23 Aug 2022
Language Models (Mostly) Know What They Know Saurav Kadavath Tom Conerly Amanda Askell T. Henighan Dawn Drain ... Nicholas Joseph Benjamin Mann Sam McCandlish C. Olah Jared Kaplan ELM 119 826 0 11 Jul 2022
Teaching Models to Express Their Uncertainty in Words Stephanie C. Lin Jacob Hilton Owain Evans OOD 83 417 0 28 May 2022
QAMPARI: An Open-domain Question Answering Benchmark for Questions with Many Answers from Multiple Paragraphs S. Amouyal Tomer Wolfson Ohad Rubin Ori Yoran Jonathan Herzig Jonathan Berant RALM VLM 75 27 0 25 May 2022
Evaluating Factuality in Text Simplification Ashwin Devaraj William Sheffield Byron C. Wallace Junyi Jessy Li HILM 62 42 0 15 Apr 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 500 6,279 0 05 Apr 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 880 13,148 0 04 Mar 2022
Transformer Memory as a Differentiable Search Index Yi Tay Vinh Q. Tran Mostafa Dehghani Jianmo Ni Dara Bahri ... Zhe Zhao Jai Gupta Tal Schuster William W. Cohen Donald Metzler 89 284 0 14 Feb 2022
Locating and Editing Factual Associations in GPT Kevin Meng David Bau A. Andonian Yonatan Belinkov KELM 248 1,357 0 10 Feb 2022
Survey of Hallucination in Natural Language Generation Ziwei Ji Nayeon Lee Rita Frieske Tiezheng Yu D. Su ... Delong Chen Wenliang Dai Ho Shu Chan Andrea Madotto Pascale Fung HILM LRM 210 2,394 0 08 Feb 2022
Measuring Attribution in Natural Language Generation Models Hannah Rashkin Vitaly Nikolaev Matthew Lamm Lora Aroyo Michael Collins Dipanjan Das Slav Petrov Gaurav Singh Tomar Iulia Turc David Reitter 89 182 0 23 Dec 2021
Training Verifiers to Solve Math Word Problems K. Cobbe V. Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun ... Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse John Schulman ReLM OffRL LRM 308 4,533 0 27 Oct 2021
TruthfulQA: Measuring How Models Mimic Human Falsehoods Stephanie C. Lin Jacob Hilton Owain Evans HILM 142 1,929 0 08 Sep 2021
Program Synthesis with Large Language Models Jacob Austin Augustus Odena Maxwell Nye Maarten Bosma Henryk Michalewski ... Ellen Jiang Carrie J. Cai Michael Terry Quoc V. Le Charles Sutton ELM AIMat ReCod ALM 200 1,986 0 16 Aug 2021
A Diverse Corpus for Evaluating and Developing English Math Word Problem Solvers Shen-Yun Miao Chao-Chun Liang Keh-Yih Su 68 341 0 30 Jun 2021
Are NLP Models really able to Solve Simple Math Word Problems? Arkil Patel S. Bhattamishra Navin Goyal ReLM LRM 94 843 0 12 Mar 2021
Measuring and Improving Consistency in Pretrained Language Models Yanai Elazar Nora Kassner Shauli Ravfogel Abhilasha Ravichander Eduard H. Hovy Hinrich Schütze Yoav Goldberg HILM 325 367 0 01 Feb 2021
Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies Mor Geva Daniel Khashabi Elad Segal Tushar Khot Dan Roth Jonathan Berant RALM 339 731 0 06 Jan 2021
Transformer Feed-Forward Layers Are Key-Value Memories Mor Geva R. Schuster Jonathan Berant Omer Levy KELM 163 840 0 29 Dec 2020
How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering Zhengbao Jiang Jun Araki Haibo Ding Graham Neubig UQCV 60 434 0 02 Dec 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 820 42,055 0 28 May 2020
On Faithfulness and Factuality in Abstractive Summarization Joshua Maynez Shashi Narayan Bernd Bohnet Ryan T. McDonald HILM 81 1,035 0 02 May 2020
Calibration of Pre-trained Transformers Shrey Desai Greg Durrett UQLM 291 300 0 17 Mar 2020