Do Large Language Models Know What They Don't Know?

29 May 2023

Zhangyue Yin

Qiushi Sun

Qipeng Guo

Jiawen Wu

Xipeng Qiu

Xuanjing Huang

ELM

AI4MH

ArXiv PDF HTML

Papers citing "Do Large Language Models Know What They Don't Know?"

50 / 119 papers shown

Title
Perception of Knowledge Boundary for Large Language Models through Semi-open-ended Question Answering Zhihua Wen Zhiliang Tian Z. Jian Zhen Huang Pei Ke Yifu Gao Minlie Huang Dongsheng Li 44 9 0 23 May 2024
OpenFactCheck: A Unified Framework for Factuality Evaluation of LLMs Yuxia Wang Minghan Wang Hasan Iqbal Georgi Georgiev Jiahui Geng Preslav Nakov HILM 36 1 0 09 May 2024
Towards Incremental Learning in Large Language Models: A Critical Review M. Jovanovic Peter Voss ELM CLL KELM 37 5 0 28 Apr 2024
Fake Artificial Intelligence Generated Contents (FAIGC): A Survey of Theories, Detection Methods, and Opportunities Xiaomin Yu Yezhaohui Wang Yanfang Chen Zhen Tao Dinghao Xi Shichao Song Simin Niu Zhiyu Li 67 8 0 25 Apr 2024
Characterizing LLM Abstention Behavior in Science QA with Context Perturbations Bingbing Wen Bill Howe Lucy Lu Wang 28 8 0 18 Apr 2024
Stance Detection on Social Media with Fine-Tuned Large Language Models Ilker Gül R. Lebret Karl Aberer 27 7 0 18 Apr 2024
Aligning Language Models to Explicitly Handle Ambiguity Hyuhng Joon Kim Youna Kim Cheonbok Park Junyeob Kim Choonghyun Park Kang Min Yoo Sang-goo Lee Taeuk Kim 34 14 0 18 Apr 2024
Uncertainty-Based Abstention in LLMs Improves Safety and Reduces Hallucinations Christian Tomani Kamalika Chaudhuri Ivan Evtimov Daniel Cremers Mark Ibrahim 56 9 0 16 Apr 2024
LLM Evaluators Recognize and Favor Their Own Generations Arjun Panickssery Samuel R. Bowman Shi Feng 50 159 0 15 Apr 2024
Entropy Guided Extrapolative Decoding to Improve Factuality in Large Language Models Souvik Das Lifeng Jin Linfeng Song Haitao Mi Baolin Peng Dong Yu HILM 40 2 0 14 Apr 2024
Calibrating the Confidence of Large Language Models by Eliciting Fidelity Mozhi Zhang Mianqiu Huang Rundong Shi Linsen Guo Chong Peng Peng Yan Yaqian Zhou Xipeng Qiu 24 10 0 03 Apr 2024
Evaluating the Factuality of Large Language Models using Large-Scale Knowledge Graphs Xiaoze Liu Feijie Wu Tianyang Xu Zhuo Chen Yichi Zhang Xiaoqian Wang Jing Gao HILM 45 8 0 01 Apr 2024
LUQ: Long-text Uncertainty Quantification for LLMs Caiqi Zhang Fangyu Liu Marco Basaldella Nigel Collier HILM 58 24 0 29 Mar 2024
On Large Language Models' Hallucination with Regard to Known Facts Che Jiang Biqing Qi Xiangyu Hong Dayuan Fu Yang Cheng Fandong Meng Mo Yu Bowen Zhou Jie Zhou HILM LRM 39 16 0 29 Mar 2024
CAUSE: Counterfactual Assessment of User Satisfaction Estimation in Task-Oriented Dialogue Systems Amin Abolghasemi Z. Ren Arian Askari Mohammad Aliannejadi Maarten de Rijke Suzan Verberne 45 3 0 27 Mar 2024
AutoGuide: Automated Generation and Selection of State-Aware Guidelines for Large Language Model Agents Yao Fu Dong-Ki Kim Jaekyeom Kim Sungryull Sohn Lajanugen Logeswaran Kyunghoon Bae Honglak Lee LLMAG 59 7 0 13 Mar 2024
In-context Exploration-Exploitation for Reinforcement Learning Zhenwen Dai Federico Tomasi Sina Ghiassian OffRL OnRL 40 3 0 11 Mar 2024
Benchmarking Hallucination in Large Language Models based on Unanswerable Math Word Problem Yuhong Sun Zhangyue Yin Qipeng Guo Jiawen Wu Xipeng Qiu Hui Zhao 33 14 0 06 Mar 2024
A Comprehensive Evaluation of Quantization Strategies for Large Language Models Renren Jin Jiangcun Du Wuwei Huang Wei Liu Jian Luan Bin Wang Deyi Xiong MQ 32 31 0 26 Feb 2024
RetrievalQA: Assessing Adaptive Retrieval-Augmented Generation for Short-form Open-Domain Question Answering Zihan Zhang Meng Fang Ling-Hao Chen RALM 56 13 0 26 Feb 2024
HypoTermQA: Hypothetical Terms Dataset for Benchmarking Hallucination Tendency of LLMs Cem Uluoglakci T. Taşkaya-Temizel HILM 35 2 0 25 Feb 2024
How Large Language Models Encode Context Knowledge? A Layer-Wise Probing Study Tianjie Ju Weiwei Sun Wei Du Xinwei Yuan Zhaochun Ren Gongshen Liu KELM 39 23 0 25 Feb 2024
Enabling Weak LLMs to Judge Response Reliability via Meta Ranking Zijun Liu Boqun Kou Peng Li Ming Yan Ji Zhang Fei Huang Yang Liu 32 2 0 19 Feb 2024
When Do LLMs Need Retrieval Augmentation? Mitigating LLMs' Overconfidence Helps Retrieval Augmentation Shiyu Ni Keping Bi J. Guo Xueqi Cheng RALM 32 32 0 18 Feb 2024
Into the Unknown: Self-Learning Large Language Models Teddy Ferdinan Jan Kocoñ P. Kazienko 33 2 0 14 Feb 2024
Introspective Planning: Aligning Robots' Uncertainty with Inherent Task Ambiguity Kaiqu Liang Zixu Zhang J. F. Fisac LLMAG 49 6 0 09 Feb 2024
INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection Chao Chen Kai-Chun Liu Ze Chen Yi Gu Yue-bo Wu Mingyuan Tao Zhihang Fu Jieping Ye HILM 85 85 0 06 Feb 2024
Integration of cognitive tasks into artificial general intelligence test for large models Youzhi Qu Chen Wei Penghui Du Wenxin Che Chi Zhang ... Bin Hu Kai Du Haiyan Wu Jia Liu Quanying Liu ELM 34 7 0 04 Feb 2024
Factuality of Large Language Models in the Year 2024 Yuxia Wang Minghan Wang Muhammad Arslan Manzoor Fei Liu Georgi Georgiev Rocktim Jyoti Das Preslav Nakov LRM HILM 38 22 0 04 Feb 2024
Can AI Assistants Know What They Don't Know? Qinyuan Cheng Tianxiang Sun Xiangyang Liu Wenwei Zhang Zhangyue Yin Shimin Li Linyang Li Zhengfu He Kai Chen Xipeng Qiu 39 23 0 24 Jan 2024
SLANG: New Concept Comprehension of Large Language Models Lingrui Mei Shenghua Liu Yiwei Wang Baolong Bi Xueqi Chen KELM 40 5 0 23 Jan 2024
Aligning Translation-Specific Understanding to General Understanding in Large Language Models Yi-Chong Huang Xiaocheng Feng Baohang Li Chengpeng Fu Wenshuai Huo Ting Liu Bing Qin 30 0 0 10 Jan 2024
LLMs for Robotic Object Disambiguation Connie Jiang Yiqing Xu David Hsu LM&Ro 30 1 0 07 Jan 2024
Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models Matthew Dahl Varun Magesh Mirac Suzgun Daniel E. Ho HILM AILaw 25 73 0 02 Jan 2024
Alignment for Honesty Yuqing Yang Ethan Chern Xipeng Qiu Graham Neubig Pengfei Liu 44 30 0 12 Dec 2023
Learning to Break: Knowledge-Enhanced Reasoning in Multi-Agent Debate System Haotian Wang Xiyuan Du Weijiang Yu Qianglong Chen Kun Zhu Zheng Chu Lian Yan Yi Guan 32 10 0 08 Dec 2023
Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communication Zhangyue Yin Qiushi Sun Cheng Chang Qipeng Guo Junqi Dai Xuanjing Huang Xipeng Qiu LRM 56 49 0 04 Dec 2023
Towards Accurate Differential Diagnosis with Large Language Models Daniel J. McDuff M. Schaekermann Tao Tu Anil Palepu Amy Wang ... G. Corrado Yossi Matias Jacob Sunshine Alan Karthikesalingam Vivek Natarajan ELM LM&MA 40 74 0 30 Nov 2023
FFT: Towards Harmlessness Evaluation and Analysis for LLMs with Factuality, Fairness, Toxicity Shiyao Cui Zhenyu Zhang Yilong Chen Wenyuan Zhang Tianyun Liu Siqi Wang Tingwen Liu 41 13 0 30 Nov 2023
UHGEval: Benchmarking the Hallucination of Chinese Large Language Models via Unconstrained Generation Xun Liang Shichao Song Simin Niu Zhiyu Li Zhiyu Li ... Zhaohui Wy Dawei He Peng Cheng Zhonghao Wang Haiying Deng HILM 34 19 0 26 Nov 2023
Testing Language Model Agents Safely in the Wild Silen Naihin David Atkinson Marc Green Merwane Hamadi Craig Swift Douglas Schonholtz Adam Tauman Kalai David Bau LLMAG 29 19 0 17 Nov 2023
Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkers Yuxia Wang Revanth Gangi Reddy Zain Muhammad Mujahid Arnav Arora Aleksandr Rubashevskii ... Nadav Borenstein Aditya Pillai Isabelle Augenstein Iryna Gurevych Preslav Nakov HILM 41 33 0 15 Nov 2023
A Survey of Confidence Estimation and Calibration in Large Language Models Jiahui Geng Fengyu Cai Yuxia Wang Heinz Koeppl Preslav Nakov Iryna Gurevych UQCV 41 56 0 14 Nov 2023
Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applications Zhangyin Feng Weitao Ma Weijiang Yu Lei Huang Haotian Wang Qianglong Chen Weihua Peng Xiaocheng Feng Bing Qin Ting Liu KELM 23 37 0 10 Nov 2023
TCM-GPT: Efficient Pre-training of Large Language Models for Domain Adaptation in Traditional Chinese Medicine Guoxing Yang Jianyu Shi Zan Wang Xiaohong Liu Guangyu Wang 21 16 0 03 Nov 2023
Learn to Refuse: Making Large Language Models More Controllable and Reliable through Knowledge Scope Limitation and Refusal Mechanism Lang Cao 23 13 0 02 Nov 2023
Do Language Models Learn about Legal Entity Types during Pretraining? Claire Barale Michael Rovatsos Nehal Bhuta ELM 33 2 0 19 Oct 2023
FactCHD: Benchmarking Fact-Conflicting Hallucination Detection Xiang Chen Duanzheng Song Honghao Gui Chengxi Wang Ningyu Zhang Jiang Yong Yan Zhang Chengfei Lv Dan Zhang Huajun Chen HILM 35 14 0 18 Oct 2023
UNK-VQA: A Dataset and a Probe into the Abstention Ability of Multi-modal Large Models Yanyang Guo Fangkai Jiao Zhiqi Shen Liqiang Nie Mohan S. Kankanhalli MLLM 30 5 0 17 Oct 2023
Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity Cunxiang Wang Xiaoze Liu Yuanhao Yue Xiangru Tang Tianhang Zhang ... Linyi Yang Jindong Wang Xing Xie Zheng-Wei Zhang Yue Zhang HILM KELM 51 184 0 11 Oct 2023