Refining Targeted Syntactic Evaluation of Language Models

19 April 2021

Papers citing "Refining Targeted Syntactic Evaluation of Language Models"

33 / 33 papers shown

Title
Investigating Syntactic Biases in Multilingual Transformers with RC Attachment Ambiguities in Italian and English Michael Kamerath Aniello De Santo 24 0 0 14 Apr 2025
Linguistic Interpretability of Transformer-based Language Models: a systematic review Miguel López-Otal Jorge Gracia Jordi Bernad Carlos Bobed Lucía Pitarch-Ballesteros Emma Anglés-Herrero VLM 36 0 0 09 Apr 2025
MultiBLiMP 1.0: A Massively Multilingual Benchmark of Linguistic Minimal Pairs Jaap Jumelet Leonie Weissweiler Arianna Bisazza 40 2 0 03 Apr 2025
Language Models at the Syntax-Semantics Interface: A Case Study of the Long-Distance Binding of Chinese Reflexive ziji Xiulin Yang 40 0 0 02 Apr 2025
Are formal and functional linguistic mechanisms dissociated in language models? Michael Hanna Sandro Pezzelle Yonatan Belinkov 47 0 0 14 Mar 2025
Training an NLP Scholar at a Small Liberal Arts College: A Backwards Designed Course Proposal Grusha Prasad Forrest Davis 26 0 0 11 Aug 2024
On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMs Nitay Calderon Roi Reichart 40 10 0 27 Jul 2024
LLM Circuit Analyses Are Consistent Across Training and Scale Curt Tigges Michael Hanna Qinan Yu Stella Biderman 36 10 0 15 Jul 2024
Probing the Category of Verbal Aspect in Transformer Language Models Anisia Katinskaia R. Yangarber 58 2 0 04 Jun 2024
Filtered Corpus Training (FiCT) Shows that Language Models can Generalize from Indirect Evidence Abhinav Patil Jaap Jumelet Yu Ying Chiu Andy Lapastora Peter Shen Lexie Wang Clevis Willrich Shane Steinert-Threlkeld 32 13 0 24 May 2024
Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanisms Michael Hanna Sandro Pezzelle Yonatan Belinkov 51 34 0 26 Mar 2024
Towards Visual Syntactical Understanding Sayeed Shafayet Chowdhury Soumyadeep Chandra Kaushik Roy NAI 32 0 0 30 Jan 2024
How Well Do Large Language Models Understand Syntax? An Evaluation by Asking Natural Language Questions Houquan Zhou Yang Hou Zhenghua Li Xuebin Wang Zhefeng Wang Xinyu Duan Min Zhang ELM 15 5 0 14 Nov 2023
How Well Do Text Embedding Models Understand Syntax? Yan Zhang Zhaopeng Feng Zhiyang Teng Zuozhu Liu Haizhou Li 40 3 0 14 Nov 2023
Multilingual Nonce Dependency Treebanks: Understanding how Language Models represent and process syntactic structure David Arps Laura Kallmeyer Younes Samih Hassan Sajjad 24 1 0 13 Nov 2023
How Abstract Is Linguistic Generalization in Large Language Models? Experiments with Argument Structure Michael Wilson Jackson Petty Robert Frank 26 15 0 08 Nov 2023
Mixture-of-Linguistic-Experts Adapters for Improving and Interpreting Pre-trained Language Models Raymond Li Gabriel Murray Giuseppe Carenini MoE 41 2 0 24 Oct 2023
How to Plant Trees in Language Models: Data and Architectural Effects on the Emergence of Syntactic Inductive Biases Aaron Mueller Tal Linzen AI4CE 29 20 0 31 May 2023
Language Model Behavior: A Comprehensive Survey Tyler A. Chang Benjamin Bergen VLM LRM LM&MA 27 103 0 20 Mar 2023
Assessing the Capacity of Transformer to Abstract Syntactic Representations: A Contrastive Analysis Based on Long-distance Agreement Bingzhi Li Guillaume Wisniewski Benoît Crabbé 64 12 0 08 Dec 2022
Emergent Linguistic Structures in Neural Networks are Fragile Emanuele La Malfa Matthew Wicker Marta Kiatkowska 22 1 0 31 Oct 2022
Probing for targeted syntactic knowledge through grammatical error detection Christopher Davis Christopher Bryant Andrew Caines Marek Rei P. Buttery 22 3 0 28 Oct 2022
A computational psycholinguistic evaluation of the syntactic abilities of Galician BERT models at the interface of dependency resolution and training time Iria de-Dios-Flores Marcos Garcia 25 2 0 06 Jun 2022
When a sentence does not introduce a discourse entity, Transformer-based models still sometimes refer to it Sebastian Schuster Tal Linzen 13 25 0 06 May 2022
Probing for the Usage of Grammatical Number Karim Lasri Tiago Pimentel Alessandro Lenci Thierry Poibeau Ryan Cotterell 30 55 0 19 Apr 2022
Does BERT really agree ? Fine-grained Analysis of Lexical Dependence on a Syntactic Task Karim Lasri Alessandro Lenci Thierry Poibeau 26 13 0 14 Apr 2022
Probing for Constituency Structure in Neural Language Models David Arps Younes Samih Laura Kallmeyer Hassan Sajjad 24 12 0 13 Apr 2022
minicons: Enabling Flexible Behavioral and Representational Analyses of Transformer Language Models Kanishka Misra 19 58 0 24 Mar 2022
Schrödinger's Tree -- On Syntax and Neural Language Models Artur Kulmizev Joakim Nivre 35 6 0 17 Oct 2021
Structural Persistence in Language Models: Priming as a Window into Abstract Language Representations Arabella J. Sinclair Jaap Jumelet Willem H. Zuidema Raquel Fernández 61 38 0 30 Sep 2021
On the Limits of Minimal Pairs in Contrastive Evaluation Jannis Vamvas Rico Sennrich 49 16 0 15 Sep 2021
Frequency Effects on Syntactic Rule Learning in Transformers Jason W. Wei Dan Garrette Tal Linzen Ellie Pavlick 88 62 0 14 Sep 2021
Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models Matthew Finlayson Aaron Mueller Sebastian Gehrmann Stuart M. Shieber Tal Linzen Yonatan Belinkov 40 101 0 10 Jun 2021