ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2205.11758
  4. Cited By
Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of
  Multilingual Language Models
v1v2 (latest)

Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language Models

24 May 2022
Terra Blevins
Hila Gonen
Luke Zettlemoyer
    LRM
ArXiv (abs)PDFHTML

Papers citing "Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language Models"

22 / 22 papers shown
Title
Domain Pre-training Impact on Representations
Domain Pre-training Impact on Representations
César González-Gutiérrez
A. Quattoni
40
0
0
30 May 2025
Semantic Pivots Enable Cross-Lingual Transfer in Large Language Models
Semantic Pivots Enable Cross-Lingual Transfer in Large Language Models
Kaiyu He
Tong Zhou
Yubo Chen
Delai Qiu
Shengping Liu
Kang Liu
Jun Zhao
LRM
73
0
0
22 May 2025
Tracing Multilingual Factual Knowledge Acquisition in Pretraining
Tracing Multilingual Factual Knowledge Acquisition in Pretraining
Yihong Liu
Mingyang Wang
Amir Hossein Kargaran
Felicia Körner
Ercong Nie
Yun Xue
François Yvon
Hinrich Schutze
HILMKELM
130
0
0
20 May 2025
On the Acquisition of Shared Grammatical Representations in Bilingual Language Models
On the Acquisition of Shared Grammatical Representations in Bilingual Language Models
Catherine Arnett
Tyler A. Chang
J. Michaelov
Benjamin Bergen
90
0
0
05 Mar 2025
Deploying Multi-task Online Server with Large Language Model
Deploying Multi-task Online Server with Large Language Model
Yincen Qu
Chao Ma
Xiangying Dai
Hui Zhou
Yiting Wu
Hengyue Liu
58
0
0
06 Nov 2024
Modular Sentence Encoders: Separating Language Specialization from Cross-Lingual Alignment
Modular Sentence Encoders: Separating Language Specialization from Cross-Lingual Alignment
Yongxin Huang
Kexin Wang
Goran Glavaš
Iryna Gurevych
100
1
0
20 Jul 2024
Probing the Emergence of Cross-lingual Alignment during LLM Training
Probing the Emergence of Cross-lingual Alignment during LLM Training
Hetong Wang
Pasquale Minervini
Edoardo Ponti
139
15
0
19 Jun 2024
Babysit A Language Model From Scratch: Interactive Language Learning by Trials and Demonstrations
Babysit A Language Model From Scratch: Interactive Language Learning by Trials and Demonstrations
Ziqiao Ma
Zekun Wang
Joyce Chai
146
4
0
22 May 2024
Multi-Task Contrastive Learning for 8192-Token Bilingual Text Embeddings
Multi-Task Contrastive Learning for 8192-Token Bilingual Text Embeddings
Isabelle Mohr
Markus Krimmel
Saba Sturua
Mohammad Kalim Akram
Andreas Koukounas
...
Susana Guzman
Bo Wang
Maximilian Werk
Nan Wang
Han Xiao
84
16
0
26 Feb 2024
Breaking the Curse of Multilinguality with Cross-lingual Expert Language
  Models
Breaking the Curse of Multilinguality with Cross-lingual Expert Language Models
Terra Blevins
Tomasz Limisiewicz
Suchin Gururangan
Margaret Li
Hila Gonen
Noah A. Smith
Luke Zettlemoyer
100
27
0
19 Jan 2024
When Is Multilinguality a Curse? Language Modeling for 250 High- and
  Low-Resource Languages
When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languages
Tyler A. Chang
Catherine Arnett
Zhuowen Tu
Benjamin Bergen
LRM
142
8
0
15 Nov 2023
OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient
  Large-scale Multilingual Continued Pretraining
OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient Large-scale Multilingual Continued Pretraining
Yihong Liu
Peiqin Lin
Mingyang Wang
Hinrich Schütze
71
29
0
15 Nov 2023
Counterfactually Probing Language Identity in Multilingual Models
Counterfactually Probing Language Identity in Multilingual Models
Anirudh Srinivasan
Venkata S Govindarajan
Kyle Mahowald
70
1
0
29 Oct 2023
Unveiling Multilinguality in Transformer Models: Exploring Language
  Specificity in Feed-Forward Networks
Unveiling Multilinguality in Transformer Models: Exploring Language Specificity in Feed-Forward Networks
Sunit Bhattacharya
Ondrej Bojar
49
12
0
24 Oct 2023
Characterizing Learning Curves During Language Model Pre-Training:
  Learning, Forgetting, and Stability
Characterizing Learning Curves During Language Model Pre-Training: Learning, Forgetting, and Stability
Tyler A. Chang
Zhuowen Tu
Benjamin Bergen
59
13
0
29 Aug 2023
Second Language Acquisition of Neural Language Models
Second Language Acquisition of Neural Language Models
Miyu Oba
Tatsuki Kuribayashi
Hiroki Ouchi
Taro Watanabe
63
6
0
05 Jun 2023
How do languages influence each other? Studying cross-lingual data
  sharing during LM fine-tuning
How do languages influence each other? Studying cross-lingual data sharing during LM fine-tuning
Rochelle Choenni
Dan Garrette
Ekaterina Shutova
100
16
0
22 May 2023
Training Trajectories of Language Models Across Scales
Training Trajectories of Language Models Across Scales
Mengzhou Xia
Mikel Artetxe
Chunting Zhou
Xi Lin
Ramakanth Pasunuru
Danqi Chen
Luke Zettlemoyer
Ves Stoyanov
AIFinLRM
98
64
0
19 Dec 2022
Data-Efficient Cross-Lingual Transfer with Language-Specific Subnetworks
Data-Efficient Cross-Lingual Transfer with Language-Specific Subnetworks
Rochelle Choenni
Dan Garrette
Ekaterina Shutova
94
2
0
31 Oct 2022
WeLM: A Well-Read Pre-trained Language Model for Chinese
WeLM: A Well-Read Pre-trained Language Model for Chinese
Hui Su
Xiao Zhou
Houjin Yu
Xiaoyu Shen
Yuwen Chen
Zilin Zhu
Yang Yu
Jie Zhou
87
23
0
21 Sep 2022
Language Modelling with Pixels
Language Modelling with Pixels
Phillip Rust
Jonas F. Lotz
Emanuele Bugliarello
Elizabeth Salesky
Miryam de Lhoneux
Desmond Elliott
VLM
107
46
0
14 Jul 2022
Memorization Without Overfitting: Analyzing the Training Dynamics of
  Large Language Models
Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models
Kushal Tirumala
Aram H. Markosyan
Luke Zettlemoyer
Armen Aghajanyan
TDI
127
197
0
22 May 2022
1