What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models

31 July 2019

Papers citing "What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models"

50 / 113 papers shown

Title
Do Large Language Models know who did what to whom? Joseph M. Denning Xiaohan Bryor Snefjella Idan A. Blank 62 1 0 23 Apr 2025
From No to Know: Taxonomy, Challenges, and Opportunities for Negation Understanding in Multimodal Foundation Models Mayank Vatsa Aparna Bharati S. Mittal Richa Singh 58 0 0 10 Feb 2025
Calling a Spade a Heart: Gaslighting Multimodal Large Language Models via Negation Bin Zhu Hui yan Qi Yinxuan Gui Jingjing Chen Chong-Wah Ngo Ee-Peng Lim 120 1 0 31 Jan 2025
Probing structural constraints of negation in Pretrained Language Models David Kletz Marie Candito Pascal Amsili 31 0 0 06 Aug 2024
Does ChatGPT Have a Mind? Simon Goldstein B. Levinstein AI4MH LRM 39 5 0 27 Jun 2024
Paraphrasing in Affirmative Terms Improves Negation Understanding MohammadHossein Rezaei Eduardo Blanco 44 1 0 11 Jun 2024
Filtered Corpus Training (FiCT) Shows that Language Models can Generalize from Indirect Evidence Abhinav Patil Jaap Jumelet Yu Ying Chiu Andy Lapastora Peter Shen Lexie Wang Clevis Willrich Shane Steinert-Threlkeld 35 13 0 24 May 2024
Natural Language Processing RELIES on Linguistics Juri Opitz Shira Wein Nathan Schneider AI4CE 55 7 0 09 May 2024
Influence of Solution Efficiency and Valence of Instruction on Additive and Subtractive Solution Strategies in Humans and GPT-4 Lydia Uhler Verena Jordan Jürgen Buder Markus Huff F. Papenmeier 43 0 0 25 Apr 2024
Language in Vivo vs. in Silico: Size Matters but Larger Language Models Still Do Not Comprehend Language on a Par with Humans Vittoria Dentella Fritz Guenther Evelina Leivada ELM 49 1 0 23 Apr 2024
Probing Large Language Models for Scalar Adjective Lexical Semantics and Scalar Diversity Pragmatics Fangru Lin Daniel Altshuler J. Pierrehumbert 38 1 0 04 Apr 2024
Should We Fear Large Language Models? A Structural Analysis of the Human Reasoning System for Elucidating LLM Capabilities and Risks Through the Lens of Heidegger's Philosophy Jianqiiu Zhang ELM 40 1 0 05 Mar 2024
Do Large Language Models Mirror Cognitive Language Processing? Yuqi Ren Renren Jin Tongxuan Zhang Deyi Xiong 50 4 0 28 Feb 2024
Uncovering Intermediate Variables in Transformers using Circuit Probing Michael A. Lepori Thomas Serre Ellie Pavlick 75 7 0 07 Nov 2023
Chain-of-Thought Tuning: Masked Language Models can also Think Step By Step in Natural Language Understanding Caoyun Fan Jidong Tian Yitian Li Wenqing Chen Hao He Yaohui Jin LRM 32 3 0 18 Oct 2023
Disentangling the Linguistic Competence of Privacy-Preserving BERT Stefan Arnold Nils Kemmerzell Annika Schreiner 30 0 0 17 Oct 2023
Trustworthy Formal Natural Language Specifications Colin S. Gordon Sergey Matskevich HILM 29 3 0 05 Oct 2023
Agentività e telicità in GilBERTo: implicazioni cognitive A. Lombardi Alessandro Lenci 23 1 0 06 Jul 2023
Morphosyntactic probing of multilingual BERT models Judit Ács Endre Hamerlik Roy Schwartz Noah A. Smith András Kornai 32 9 0 09 Jun 2023
Prompt to be Consistent is Better than Self-Consistent? Few-Shot and Zero-Shot Fact Verification with Pre-trained Language Models Fengzhu Zeng Wei Gao 17 5 0 05 Jun 2023
AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap Q. V. Liao J. Vaughan 38 158 0 02 Jun 2023
Syntax and Semantics Meet in the "Middle": Probing the Syntax-Semantics Interface of LMs Through Agentivity Lindia Tjuatja Emmy Liu Lori S. Levin Graham Neubig 35 2 0 29 May 2023
Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models Yuhui Zhang Michihiro Yasunaga Zhengping Zhou Jeff Z. HaoChen James Zou Percy Liang Serena Yeung 47 7 0 27 May 2023
Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarization Dongqi Pu Yifa Wang Vera Demberg 34 21 0 26 May 2023
Not wacky vs. definitely wacky: A study of scalar adverbs in pretrained language models Isabelle Lorge J. Pierrehumbert 36 0 0 25 May 2023
NevIR: Negation in Neural Information Retrieval Orion Weller Dawn J Lawrie Benjamin Van Durme MU 61 18 0 12 May 2023
Discourse over Discourse: The Need for an Expanded Pragmatic Focus in Conversational AI S. M. Seals V. Shalin 29 4 0 27 Apr 2023
Rethinking Visual Prompt Learning as Masked Visual Token Modeling Ning Liao Bowen Shi Xiaopeng Zhang Min Cao Junchi Yan Qi Tian VLM 34 7 0 09 Mar 2023
Testing AI on language comprehension tasks reveals insensitivity to underlying meaning Vittoria Dentella Fritz Guenther Elliot Murphy G. Marcus Evelina Leivada ELM 40 26 0 23 Feb 2023
A Discerning Several Thousand Judgments: GPT-3 Rates the Article + Adjective + Numeral + Noun Construction Kyle Mahowald 22 24 0 29 Jan 2023
Dissociating language and thought in large language models Kyle Mahowald Anna A. Ivanova I. Blank Nancy Kanwisher J. Tenenbaum Evelina Fedorenko ELM ReLM 29 209 0 16 Jan 2023
Measure More, Question More: Experimental Studies on Transformer-based Language Models and Complement Coercion Yuling Gu 22 1 0 20 Dec 2022
On the Blind Spots of Model-Based Evaluation Metrics for Text Generation Tianxing He Jingyu Zhang Tianle Wang Sachin Kumar Kyunghyun Cho James R. Glass Yulia Tsvetkov 40 44 0 20 Dec 2022
Implicit causality in GPT-2: a case study H. Huynh T. Lentz Emiel van Miltenburg LRM 27 3 0 08 Dec 2022
Event knowledge in large language models: the gap between the impossible and the unlikely Carina Kauf Anna A. Ivanova Giulia Rambelli Emmanuele Chersoni Jingyuan Selena She Zawad Chowdhury Evelina Fedorenko Alessandro Lenci 37 67 0 02 Dec 2022
Scientific and Creative Analogies in Pretrained Language Models Tamara Czinczoll H. Yannakoudakis Pushkar Mishra Ekaterina Shutova ELM 23 8 0 28 Nov 2022
On the Effect of Pre-training for Transformer in Different Modality on Offline Reinforcement Learning S. Takagi OffRL 18 7 0 17 Nov 2022
Collateral facilitation in humans and language models J. Michaelov Benjamin Bergen 19 11 0 09 Nov 2022
CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negation Abhilasha Ravichander Matt Gardner Ana Marasović 33 34 0 01 Nov 2022
Modeling structure-building in the brain with CCG parsing and large language models Miloš Stanojević Jonathan Brennan Donald Dunagan Mark Steedman John T. Hale 19 12 0 28 Oct 2022
TASA: Deceiving Question Answering Models by Twin Answer Sentences Attack Yu Cao Dianqi Li Meng Fang Dinesh Manocha Jun Gao Yibing Zhan Dacheng Tao AAML 26 15 0 27 Oct 2022
Leveraging Affirmative Interpretations from Negation Improves Natural Language Understanding Md Mosharaf Hossain Eduardo Blanco 32 4 0 26 Oct 2022
Visualizing the Obvious: A Concreteness-based Ensemble Model for Noun Property Prediction Yue Yang Artemis Panagopoulou Marianna Apidianaki Mark Yatskar Chris Callison-Burch 26 2 0 24 Oct 2022
DALL-E 2 Fails to Reliably Capture Common Syntactic Processes Evelina Leivada Elliot Murphy G. Marcus 138 37 0 23 Oct 2022
Probing with Noise: Unpicking the Warp and Weft of Embeddings Filip Klubicka John D. Kelleher 30 4 0 21 Oct 2022
Can Language Representation Models Think in Bets? Zhi–Bin Tang Mayank Kejriwal 15 6 0 14 Oct 2022
Transparency Helps Reveal When Language Models Learn Meaning Zhaofeng Wu William Merrill Hao Peng Iz Beltagy Noah A. Smith 19 9 0 14 Oct 2022
Understanding Prior Bias and Choice Paralysis in Transformer-based Language Representation Models through Four Experimental Probes Ke Shen Mayank Kejriwal 27 4 0 03 Oct 2022
Representing Affect Information in Word Embeddings Yuhan Zhang Wenqi Chen Ruihan Zhang Xiajie Zhang CVBM 57 3 0 21 Sep 2022
Subject Verb Agreement Error Patterns in Meaningless Sentences: Humans vs. BERT Karim Lasri Olga Seminck Alessandro Lenci Thierry Poibeau 24 4 0 21 Sep 2022