Cross-Lingual Language Model Meta-Pretraining

23 September 2021

Papers citing "Cross-Lingual Language Model Meta-Pretraining"

22 / 22 papers shown

Title
Cross-lingual Transferring of Pre-trained Contextualized Language Models Zuchao Li Kevin Parnow Hai Zhao Zhuosheng Zhang Rui Wang Masao Utiyama Eiichiro Sumita 31 8 0 27 Jul 2021
Consistency Regularization for Cross-Lingual Fine-Tuning Bo Zheng Li Dong Shaohan Huang Wenhui Wang Zewen Chi Saksham Singhal Wanxiang Che Ting Liu Xia Song Furu Wei 42 58 0 15 Jun 2021
Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment Zewen Chi Li Dong Bo Zheng Shaohan Huang Xian-Ling Mao Heyan Huang Furu Wei 61 68 0 11 Jun 2021
nmT5 -- Is parallel data still relevant for pre-training massively multilingual language models? Mihir Kale Aditya Siddhant Noah Constant Melvin Johnson Rami Al-Rfou Linting Xue LRM 59 25 0 03 Jun 2021
MT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs Zewen Chi Li Dong Shuming Ma Shaohan Huang Xian-Ling Mao Heyan Huang Furu Wei LRM 72 73 0 18 Apr 2021
VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generation Fuli Luo Wei Wang Jiahao Liu Yijia Liu Bin Bi Songfang Huang Fei Huang Luo Si 66 51 0 30 Oct 2020
Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information Zehui Lin Xiao Pan Mingxuan Wang Xipeng Qiu Jiangtao Feng Hao Zhou Lei Li 55 127 0 07 Oct 2020
Language-agnostic BERT Sentence Embedding Fangxiaoyu Feng Yinfei Yang Daniel Cer N. Arivazhagan Wei Wang 159 904 0 03 Jul 2020
Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation Nils Reimers Iryna Gurevych 97 1,023 0 21 Apr 2020
SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings Masoud Jalili Sabet Philipp Dufter François Yvon Hinrich Schütze 62 235 0 18 Apr 2020
XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization Junjie Hu Sebastian Ruder Aditya Siddhant Graham Neubig Orhan Firat Melvin Johnson ELM 177 973 0 24 Mar 2020
TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages J. Clark Eunsol Choi Michael Collins Dan Garrette Tom Kwiatkowski Vitaly Nikolaev J. Palomaki 141 608 0 10 Mar 2020
Multilingual Alignment of Contextual Word Representations Steven Cao Nikita Kitaev Dan Klein 81 193 0 10 Feb 2020
Cross-Lingual Ability of Multilingual BERT: An Empirical Study Karthikeyan K Zihan Wang Stephen D. Mayhew Dan Roth LRM 71 339 0 17 Dec 2019
Unsupervised Cross-lingual Representation Learning at Scale Alexis Conneau Kartikay Khandelwal Naman Goyal Vishrav Chaudhary Guillaume Wenzek Francisco Guzmán Edouard Grave Myle Ott Luke Zettlemoyer Veselin Stoyanov 209 6,546 0 05 Nov 2019
On the Cross-lingual Transferability of Monolingual Representations Mikel Artetxe Sebastian Ruder Dani Yogatama 193 797 0 25 Oct 2019
Cross-Lingual Natural Language Generation via Pre-Training Zewen Chi Li Dong Furu Wei Wenhui Wang Xian-Ling Mao Heyan Huang 76 137 0 23 Sep 2019
How multilingual is Multilingual BERT? Telmo Pires Eva Schlinger Dan Garrette LRM VLM 143 1,409 0 04 Jun 2019
Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT Shijie Wu Mark Dredze VLM SSeg 95 678 0 19 Apr 2019
SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing Taku Kudo John Richardson 193 3,518 0 19 Aug 2018
Word Translation Without Parallel Data Alexis Conneau Guillaume Lample MarcÁurelio Ranzato Ludovic Denoyer Hervé Jégou 287 1,657 0 11 Oct 2017
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 274 8,127 0 16 Jun 2016