Negation is a fundamental linguistic phenomenon that poses persistent challenges for Large Language Models (LLMs), particularly in tasks requiring deep semantic understanding. Existing benchmarks often treat negation as a side case within broader tasks like natural language inference, resulting in a lack of benchmarks that exclusively target negation understanding. In this work, we introduce \textbf{Thunder-NUBench}, a novel benchmark explicitly designed to assess sentence-level negation understanding in LLMs. Thunder-NUBench goes beyond surface-level cue detection by contrasting standard negation with structurally diverse alternatives such as local negation, contradiction, and paraphrase. The benchmark consists of manually curated sentence-negation pairs and a multiple-choice dataset that enables in-depth evaluation of models' negation understanding.
View on arXiv@article{so2025_2506.14397,
title={ Thunder-NUBench: A Benchmark for LLMs' Sentence-Level Negation Understanding },
author={ Yeonkyoung So and Gyuseong Lee and Sungmok Jung and Joonhak Lee and JiA Kang and Sangho Kim and Jaejin Lee },
journal={arXiv preprint arXiv:2506.14397},
year={ 2025 }
}