Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of
Multilingual Language Models

Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language Models

24 May 2022

Luke Zettlemoyer

Papers citing "Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language Models"

7 / 7 papers shown

Title
Modular Sentence Encoders: Separating Language Specialization from Cross-Lingual Alignment Yongxin Huang Kexin Wang Goran Glavavs Iryna Gurevych 46 0 0 20 Jul 2024
Babysit A Language Model From Scratch: Interactive Language Learning by Trials and Demonstrations Ziqiao Ma Zekun Wang Joyce Chai 58 2 0 22 May 2024
When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languages Tyler A. Chang Catherine Arnett Zhuowen Tu Benjamin Bergen LRM 41 7 0 15 Nov 2023
How do languages influence each other? Studying cross-lingual data sharing during LM fine-tuning Rochelle Choenni Dan Garrette Ekaterina Shutova 40 16 0 22 May 2023
Data-Efficient Cross-Lingual Transfer with Language-Specific Subnetworks Rochelle Choenni Dan Garrette Ekaterina Shutova 24 2 0 31 Oct 2022
Language Modelling with Pixels Phillip Rust Jonas F. Lotz Emanuele Bugliarello Elizabeth Salesky Miryam de Lhoneux Desmond Elliott VLM 38 46 0 14 Jul 2022
Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models Kushal Tirumala Aram H. Markosyan Luke Zettlemoyer Armen Aghajanyan TDI 29 185 0 22 May 2022