Papers
Communities
Events
Blog
Pricing
Search
Open menu
Home
Papers
2205.11758
Cited By
v1
v2 (latest)
Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language Models
24 May 2022
Terra Blevins
Hila Gonen
Luke Zettlemoyer
LRM
Re-assign community
ArXiv (abs)
PDF
HTML
Papers citing
"Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language Models"
22 / 22 papers shown
Title
Domain Pre-training Impact on Representations
César González-Gutiérrez
A. Quattoni
40
0
0
30 May 2025
Semantic Pivots Enable Cross-Lingual Transfer in Large Language Models
Kaiyu He
Tong Zhou
Yubo Chen
Delai Qiu
Shengping Liu
Kang Liu
Jun Zhao
LRM
73
0
0
22 May 2025
Tracing Multilingual Factual Knowledge Acquisition in Pretraining
Yihong Liu
Mingyang Wang
Amir Hossein Kargaran
Felicia Körner
Ercong Nie
Yun Xue
François Yvon
Hinrich Schutze
HILM
KELM
130
0
0
20 May 2025
On the Acquisition of Shared Grammatical Representations in Bilingual Language Models
Catherine Arnett
Tyler A. Chang
J. Michaelov
Benjamin Bergen
90
0
0
05 Mar 2025
Deploying Multi-task Online Server with Large Language Model
Yincen Qu
Chao Ma
Xiangying Dai
Hui Zhou
Yiting Wu
Hengyue Liu
58
0
0
06 Nov 2024
Modular Sentence Encoders: Separating Language Specialization from Cross-Lingual Alignment
Yongxin Huang
Kexin Wang
Goran Glavaš
Iryna Gurevych
100
1
0
20 Jul 2024
Probing the Emergence of Cross-lingual Alignment during LLM Training
Hetong Wang
Pasquale Minervini
Edoardo Ponti
139
15
0
19 Jun 2024
Babysit A Language Model From Scratch: Interactive Language Learning by Trials and Demonstrations
Ziqiao Ma
Zekun Wang
Joyce Chai
146
4
0
22 May 2024
Multi-Task Contrastive Learning for 8192-Token Bilingual Text Embeddings
Isabelle Mohr
Markus Krimmel
Saba Sturua
Mohammad Kalim Akram
Andreas Koukounas
...
Susana Guzman
Bo Wang
Maximilian Werk
Nan Wang
Han Xiao
84
16
0
26 Feb 2024
Breaking the Curse of Multilinguality with Cross-lingual Expert Language Models
Terra Blevins
Tomasz Limisiewicz
Suchin Gururangan
Margaret Li
Hila Gonen
Noah A. Smith
Luke Zettlemoyer
100
27
0
19 Jan 2024
When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languages
Tyler A. Chang
Catherine Arnett
Zhuowen Tu
Benjamin Bergen
LRM
142
8
0
15 Nov 2023
OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient Large-scale Multilingual Continued Pretraining
Yihong Liu
Peiqin Lin
Mingyang Wang
Hinrich Schütze
71
29
0
15 Nov 2023
Counterfactually Probing Language Identity in Multilingual Models
Anirudh Srinivasan
Venkata S Govindarajan
Kyle Mahowald
70
1
0
29 Oct 2023
Unveiling Multilinguality in Transformer Models: Exploring Language Specificity in Feed-Forward Networks
Sunit Bhattacharya
Ondrej Bojar
49
12
0
24 Oct 2023
Characterizing Learning Curves During Language Model Pre-Training: Learning, Forgetting, and Stability
Tyler A. Chang
Zhuowen Tu
Benjamin Bergen
59
13
0
29 Aug 2023
Second Language Acquisition of Neural Language Models
Miyu Oba
Tatsuki Kuribayashi
Hiroki Ouchi
Taro Watanabe
63
6
0
05 Jun 2023
How do languages influence each other? Studying cross-lingual data sharing during LM fine-tuning
Rochelle Choenni
Dan Garrette
Ekaterina Shutova
100
16
0
22 May 2023
Training Trajectories of Language Models Across Scales
Mengzhou Xia
Mikel Artetxe
Chunting Zhou
Xi Lin
Ramakanth Pasunuru
Danqi Chen
Luke Zettlemoyer
Ves Stoyanov
AIFin
LRM
98
64
0
19 Dec 2022
Data-Efficient Cross-Lingual Transfer with Language-Specific Subnetworks
Rochelle Choenni
Dan Garrette
Ekaterina Shutova
94
2
0
31 Oct 2022
WeLM: A Well-Read Pre-trained Language Model for Chinese
Hui Su
Xiao Zhou
Houjin Yu
Xiaoyu Shen
Yuwen Chen
Zilin Zhu
Yang Yu
Jie Zhou
87
23
0
21 Sep 2022
Language Modelling with Pixels
Phillip Rust
Jonas F. Lotz
Emanuele Bugliarello
Elizabeth Salesky
Miryam de Lhoneux
Desmond Elliott
VLM
107
46
0
14 Jul 2022
Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models
Kushal Tirumala
Aram H. Markosyan
Luke Zettlemoyer
Armen Aghajanyan
TDI
127
197
0
22 May 2022
1