Language model acceptability judgements are not always robust to context

18 December 2022

Papers citing "Language model acceptability judgements are not always robust to context"

23 / 23 papers shown

Title
Who Relies More on World Knowledge and Bias for Syntactic Ambiguity Resolution: Humans or LLMs? So Young Lee Russell Scheinberg Amber Shore Ameeta Agrawal 46 1 0 13 Mar 2025
What makes a good metric? Evaluating automatic metrics for text-to-image consistency Candace Ross Melissa Hall Adriana Romero Soriano Adina Williams 90 3 0 18 Dec 2024
Non-Determinism of "Deterministic" LLM Settings Berk Atil Alexa Chittams Liseng Fu Ferhan Ture Lixinyu Xu ... Tomasz Tudrej Ferhan Ture Zhe Wu Lixinyu Xu Breck Baldwin 26 0 0 06 Aug 2024
Changing Answer Order Can Decrease MMLU Accuracy Vipul Gupta David Pantoja Candace Ross Adina Williams Megan Ung 56 22 0 27 Jun 2024
Do Language Models Exhibit Human-like Structural Priming Effects? Jaap Jumelet Willem H. Zuidema Arabella J. Sinclair 38 5 0 07 Jun 2024
To Drop or Not to Drop? Predicting Argument Ellipsis Judgments: A Case Study in Japanese Yukiko Ishizuki Tatsuki Kuribayashi Yuichiroh Matsubayashi Ryohei Sasano Kentaro Inui 28 2 0 17 Apr 2024
Robust Pronoun Fidelity with English LLMs: Are they Reasoning, Repeating, or Just Biased? Vagrant Gautam Eileen Bingert D. Zhu Anne Lauscher Dietrich Klakow 43 8 0 04 Apr 2024
Experimental Contexts Can Facilitate Robust Semantic Property Inference in Language Models, but Inconsistently Kanishka Misra Allyson Ettinger Kyle Mahowald 21 4 0 12 Jan 2024
Evaluating Large Language Models on Controlled Generation Tasks Jiao Sun Yufei Tian Wangchunshu Zhou Nan Xu Qian Hu Rahul Gupta John Wieting Nanyun Peng Xuezhe Ma LRM ELM 35 60 0 23 Oct 2023
Revisiting Acceptability Judgements Hai Hu Ziyin Zhang Wei Huang J. Lai Aini Li Yi Ma Jiahui Huang Peng Zhang Chien-Jer Charles Lin Rui Wang 37 2 0 23 May 2023
ReCOGS: How Incidental Details of a Logical Form Overshadow an Evaluation of Semantic Interpretation Zhengxuan Wu Christopher D. Manning Christopher Potts 32 22 0 24 Mar 2023
Language Model Behavior: A Comprehensive Survey Tyler A. Chang Benjamin Bergen VLM LRM LM&MA 27 103 0 20 Mar 2023
Testing AI on language comprehension tasks reveals insensitivity to underlying meaning Vittoria Dentella Fritz Guenther Elliot Murphy G. Marcus Evelina Leivada ELM 34 26 0 23 Feb 2023
The Curious Case of Absolute Position Embeddings Koustuv Sinha Amirhossein Kazemnejad Siva Reddy J. Pineau Dieuwke Hupkes Adina Williams 83 15 0 23 Oct 2022
State-of-the-art generalisation research in NLP: A taxonomy and review Dieuwke Hupkes Mario Giulianelli Verna Dankers Mikel Artetxe Yanai Elazar ... Leila Khalatbari Maria Ryskina Rita Frieske Ryan Cotterell Zhijing Jin 114 93 0 06 Oct 2022
COMPS: Conceptual Minimal Pair Sentences for testing Robust Property Knowledge and its Inheritance in Pre-trained Language Models Kanishka Misra Julia Taylor Rayz Allyson Ettinger 25 10 0 05 Oct 2022
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 310 4,077 0 24 May 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 355 8,457 0 28 Jan 2022
Structural Persistence in Language Models: Priming as a Window into Abstract Language Representations Arabella J. Sinclair Jaap Jumelet Willem H. Zuidema Raquel Fernández 58 38 0 30 Sep 2021
Sequence Length is a Domain: Length-based Overfitting in Transformer Models Dusan Varis Ondrej Bojar 49 57 0 15 Sep 2021
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity Yao Lu Max Bartolo Alastair Moore Sebastian Riedel Pontus Stenetorp AILaw LRM 279 1,121 0 18 Apr 2021
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 241 1,918 0 31 Dec 2020
Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference Timo Schick Hinrich Schütze 258 1,588 0 21 Jan 2020