Subspace Chronicles: How Linguistic Information Emerges, Shifts and Interacts during Language Model Training

25 October 2023

Papers citing "Subspace Chronicles: How Linguistic Information Emerges, Shifts and Interacts during Language Model Training"

6 / 6 papers shown

Title
Tracing Multilingual Factual Knowledge Acquisition in Pretraining Yihong Liu Mingyang Wang Amir Hossein Kargaran Felicia Körner Ercong Nie Yun Xue François Yvon Hinrich Schutze HILM KELM 14 0 0 20 May 2025
PolyPythias: Stability and Outliers across Fifty Language Model Pre-Training Runs Oskar van der Wal Pietro Lesci Max Muller-Eberstein Naomi Saphra Hailey Schoelkopf Willem H. Zuidema Stella Biderman LRM 63 1 0 12 Mar 2025
Generalisation First, Memorisation Second? Memorisation Localisation for Natural Language Classification Tasks Verna Dankers Ivan Titov 45 5 0 09 Aug 2024
Interpretability of Language Models via Task Spaces Lucas Weber Jaap Jumelet Elia Bruni Dieuwke Hupkes 37 4 0 10 Jun 2024
OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient Large-scale Multilingual Continued Pretraining Yihong Liu Peiqin Lin Mingyang Wang Hinrich Schütze 40 23 0 15 Nov 2023
What you can cram into a single vector: Probing sentence embeddings for linguistic properties Alexis Conneau Germán Kruszewski Guillaume Lample Loïc Barrault Marco Baroni 201 883 0 03 May 2018