On the Interplay Between Fine-tuning and Sentence-level Probing for
Linguistic Knowledge in Pre-trained Transformers

On the Interplay Between Fine-tuning and Sentence-level Probing for Linguistic Knowledge in Pre-trained Transformers

6 October 2020

Michael A. Hedderich

Dietrich Klakow

Papers citing "On the Interplay Between Fine-tuning and Sentence-level Probing for Linguistic Knowledge in Pre-trained Transformers"

13 / 13 papers shown

Title
Statistical Deficiency for Task Inclusion Estimation Loïc Fosse Frédéric Béchet Benoit Favre Géraldine Damnati Gwénolé Lecorvé Maxime Darrin Philippe Formont Pablo Piantanida 142 0 0 07 Mar 2025
Can We Use Probing to Better Understand Fine-tuning and Knowledge Distillation of the BERT NLU? Jakub Ho'scilowicz Marcin Sowanski Piotr Czubowski Artur Janicki 25 2 0 27 Jan 2023
Interpretability in Activation Space Analysis of Transformers: A Focused Survey Soniya Vijayakumar AI4CE 35 3 0 22 Jan 2023
Training language models to summarize narratives improves brain alignment Khai Loong Aw Mariya Toneva 38 24 0 21 Dec 2022
On the Transformation of Latent Space in Fine-Tuned NLP Models Nadir Durrani Hassan Sajjad Fahim Dalvi Firoj Alam 32 18 0 23 Oct 2022
Hidden State Variability of Pretrained Language Models Can Guide Computation Reduction for Transfer Learning Shuo Xie Jiahao Qiu Ankita Pasad Li Du Qing Qu Hongyuan Mei 35 16 0 18 Oct 2022
Life after BERT: What do Other Muppets Understand about Language? Vladislav Lialin Kevin Zhao Namrata Shivagunde Anna Rumshisky 44 6 0 21 May 2022
Probing Pre-Trained Language Models for Cross-Cultural Differences in Values Arnav Arora Lucie-Aimée Kaffee Isabelle Augenstein VLM 31 123 0 25 Mar 2022
Not All Models Localize Linguistic Knowledge in the Same Place: A Layer-wise Probing on BERToids' Representations Mohsen Fayyaz Ehsan Aghazadeh Ali Modarressi Hosein Mohebbi Mohammad Taher Pilehvar 18 21 0 13 Sep 2021
A Closer Look at How Fine-tuning Changes BERT Yichu Zhou Vivek Srikumar 26 63 0 27 Jun 2021
How transfer learning impacts linguistic knowledge in deep NLP models? Nadir Durrani Hassan Sajjad Fahim Dalvi 13 48 0 31 May 2021
What you can cram into a single vector: Probing sentence embeddings for linguistic properties Alexis Conneau Germán Kruszewski Guillaume Lample Loïc Barrault Marco Baroni 201 882 0 03 May 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,959 0 20 Apr 2018