nmT5 -- Is parallel data still relevant for pre-training massively multilingual language models?

3 June 2021

Papers citing "nmT5 -- Is parallel data still relevant for pre-training massively multilingual language models?"

8 / 8 papers shown

Title
LEIA: Facilitating Cross-lingual Knowledge Transfer in Language Models with Entity-based Data Augmentation Ikuya Yamada Ryokan Ri KELM 25 0 0 18 Feb 2024
The Interpreter Understands Your Meaning: End-to-end Spoken Language Understanding Aided by Speech Translation Mutian He Philip N. Garner 44 4 0 16 May 2023
On the Role of Parallel Data in Cross-lingual Transfer Learning Machel Reid Mikel Artetxe 21 10 0 20 Dec 2022
Advancing Multilingual Pre-training: TRIP Triangular Document-level Pre-training for Multilingual Language Models Hongyuan Lu Haoyang Huang Shuming Ma Dongdong Zhang W. Lam Furu Wei 27 4 0 15 Dec 2022
Enhancing Cross-lingual Transfer by Manifold Mixup Huiyun Yang Huadong Chen Hao Zhou Lei Li AAML 31 44 0 09 May 2022
mSLAM: Massively multilingual joint pre-training for speech and text Ankur Bapna Colin Cherry Yu Zhang Ye Jia Melvin Johnson Yong Cheng Simran Khanuja Jason Riesa Alexis Conneau VLM 30 111 0 03 Feb 2022
PARADISE: Exploiting Parallel Data for Multilingual Sequence-to-Sequence Pretraining Machel Reid Mikel Artetxe VLM 50 26 0 04 Aug 2021
ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora Ouyang Xuan Shuohuan Wang Chao Pang Yu Sun Hao Tian Hua Wu Haifeng Wang 62 100 0 31 Dec 2020