Not All Models Localize Linguistic Knowledge in the Same Place: A
Layer-wise Probing on BERToids' Representations

Not All Models Localize Linguistic Knowledge in the Same Place: A Layer-wise Probing on BERToids' Representations

13 September 2021

Ehsan Aghazadeh

Mohammad Taher Pilehvar

Papers citing "Not All Models Localize Linguistic Knowledge in the Same Place: A Layer-wise Probing on BERToids' Representations"

14 / 14 papers shown

Title
Linguistic Interpretability of Transformer-based Language Models: a systematic review Miguel López-Otal Jorge Gracia Jordi Bernad Carlos Bobed Lucía Pitarch-Ballesteros Emma Anglés-Herrero VLM 84 0 0 09 Apr 2025
Inspecting the concept knowledge graph encoded by modern language models Carlos Aspillaga Marcelo Mendoza Alvaro Soto 56 13 0 27 May 2021
What Happens To BERT Embeddings During Fine-tuning? Amil Merchant Elahe Rahimtoroghi Ellie Pavlick Ian Tenney 59 186 0 29 Apr 2020
Analyzing analytical methods: The case of phonology in neural models of spoken language Grzegorz Chrupała Bertrand Higy Afra Alishahi 37 20 0 15 Apr 2020
What's so special about BERT's layers? A closer look at the NLP pipeline in monolingual and multilingual models Wietse de Vries Andreas van Cranenburgh Malvina Nissim MILM SSeg MoE 95 64 0 14 Apr 2020
oLMpics -- On what Language Model Pre-training Captures Alon Talmor Yanai Elazar Yoav Goldberg Jonathan Berant LRM 79 303 0 31 Dec 2019
The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives Elena Voita Rico Sennrich Ivan Titov 252 186 0 03 Sep 2019
What do you learn from context? Probing for sentence structure in contextualized word representations Ian Tenney Patrick Xia Berlin Chen Alex Jinpeng Wang Adam Poliak ... Najoung Kim Benjamin Van Durme Samuel R. Bowman Dipanjan Das Ellie Pavlick 163 853 0 15 May 2019
BERT Rediscovers the Classical NLP Pipeline Ian Tenney Dipanjan Das Ellie Pavlick MILM SSeg 109 1,458 0 15 May 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 163 3,714 0 09 Jan 2019
Neural Network Acceptability Judgments Alex Warstadt Amanpreet Singh Samuel R. Bowman 194 1,390 0 31 May 2018
What you can cram into a single vector: Probing sentence embeddings for linguistic properties Alexis Conneau Germán Kruszewski Guillaume Lample Loïc Barrault Marco Baroni 295 888 0 03 May 2018
Deep contextualized word representations Matthew E. Peters Mark Neumann Mohit Iyyer Matt Gardner Christopher Clark Kenton Lee Luke Zettlemoyer NAI 143 11,520 0 15 Feb 2018
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference Adina Williams Nikita Nangia Samuel R. Bowman 456 4,444 0 18 Apr 2017