Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of
Language Models

Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models

26 September 2023

Mert Yuksekgonul

Varun Chandrasekaran

Suriya Gunasekar

Papers citing "Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models"

19 / 19 papers shown

Title
Language Models Can Predict Their Own Behavior Dhananjay Ashok Jonathan May ReLM AI4TS LRM 63 0 0 18 Feb 2025
Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens Zhangqi Jiang Junkai Chen Beier Zhu Tingjin Luo Yankun Shen Xu Yang 103 4 0 23 Nov 2024
Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models Javier Ferrando Oscar Obeso Senthooran Rajamanoharan Neel Nanda 82 10 0 21 Nov 2024
LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations Hadas Orgad Michael Toker Zorik Gekhman Roi Reichart Idan Szpektor Hadas Kotek Yonatan Belinkov HILM AIFin 61 25 0 03 Oct 2024
Enhancing elusive clues in knowledge learning by contrasting attention of language models Jian Gao Xiao Zhang Ji Wu Miao Li 40 0 0 26 Sep 2024
Zero-resource Hallucination Detection for Text Generation via Graph-based Contextual Knowledge Triples Modeling Xinyue Fang Zhen Huang Zhiliang Tian Minghui Fang Ziyi Pan Quntian Fang Zhihua Wen Hengyue Pan Dongsheng Li HILM 93 2 0 17 Sep 2024
Understanding Information Storage and Transfer in Multi-modal Large Language Models Samyadeep Basu Martin Grayson C. Morrison Besmira Nushi S. Feizi Daniela Massiceti 25 10 0 06 Jun 2024
Knowledge Circuits in Pretrained Transformers Yunzhi Yao Ningyu Zhang Zekun Xi Meng Wang Ziwen Xu Shumin Deng Huajun Chen KELM 64 20 0 28 May 2024
Small Language Models Need Strong Verifiers to Self-Correct Reasoning Yunxiang Zhang Muhammad Khalifa Lajanugen Logeswaran Jaekyeom Kim Moontae Lee Honglak Lee Lu Wang LRM KELM ReLM 31 31 0 26 Apr 2024
The Garden of Forking Paths: Observing Dynamic Parameters Distribution in Large Language Models Carlo Nicolini Jacopo Staiano Bruno Lepri Raffaele Marino MoE 31 1 0 13 Mar 2024
"I Want It That Way": Enabling Interactive Decision Support Using Large Language Models and Constraint Programming Connor Lawless Jakob Schoeffer Lindy Le Kael Rowan Shilad Sen Cristina St. Hill Jina Suh Bahar Sarrafzadeh 41 8 0 12 Dec 2023
How Language Model Hallucinations Can Snowball Muru Zhang Ofir Press William Merrill Alisa Liu Noah A. Smith HILM LRM 85 253 0 22 May 2023
Finding Neurons in a Haystack: Case Studies with Sparse Probing Wes Gurnee Neel Nanda Matthew Pauly Katherine Harvey Dmitrii Troitskii Dimitris Bertsimas MILM 160 188 0 02 May 2023
Dissecting Recall of Factual Associations in Auto-Regressive Language Models Mor Geva Jasmijn Bastings Katja Filippova Amir Globerson KELM 191 261 0 28 Apr 2023
SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models Potsawee Manakul Adian Liusie Mark J. F. Gales HILM LRM 152 391 0 15 Mar 2023
In-context Learning and Induction Heads Catherine Olsson Nelson Elhage Neel Nanda Nicholas Joseph Nova Dassarma ... Tom B. Brown Jack Clark Jared Kaplan Sam McCandlish C. Olah 250 460 0 24 Sep 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 313 11,953 0 04 Mar 2022
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 290 1,815 0 14 Dec 2020
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 415 2,588 0 03 Sep 2019