Allocating Large Vocabulary Capacity for Cross-lingual Language Model Pre-training

15 September 2021

Xia Song

Papers citing "Allocating Large Vocabulary Capacity for Cross-lingual Language Model Pre-training"

22 / 22 papers shown

Title
Scaling Embedding Layers in Language Models Da Yu Edith Cohen Badih Ghazi Yangsibo Huang Pritish Kamath Ravi Kumar Daogao Liu Chiyuan Zhang 82 0 0 03 Feb 2025
Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Models Lucas Bandarkar Benjamin Muller Pritish Yuvraj Rui Hou Nayan Singhal Hongjiang Lv Bing-Quan Liu KELM LRM MoMe 52 3 0 02 Oct 2024
Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies Chaofan Tao Qian Liu Longxu Dou Niklas Muennighoff Zhongwei Wan Ping Luo Min-Bin Lin Ngai Wong PILM 60 46 0 18 Jul 2024
UniBridge: A Unified Approach to Cross-Lingual Transfer Learning for Low-Resource Languages Trinh Pham Khoi M. Le Luu Anh Tuan 42 1 0 14 Jun 2024
Pointer-Generator Networks for Low-Resource Machine Translation: Don't Copy That! Niyati Bafna Philipp Koehn David Yarowsky 37 1 0 16 Mar 2024
MYTE: Morphology-Driven Byte Encoding for Better and Fairer Multilingual Language Modeling Tomasz Limisiewicz Terra Blevins Hila Gonen Orevaoghene Ahia Luke Zettlemoyer 30 13 0 15 Mar 2024
Multi-Task Contrastive Learning for 8192-Token Bilingual Text Embeddings Isabelle Mohr Markus Krimmel Saba Sturua Mohammad Kalim Akram Andreas Koukounas ... Susana Guzman Bo Wang Maximilian Werk Nan Wang Han Xiao 35 15 0 26 Feb 2024
Structure-informed Positional Encoding for Music Generation Manvi Agarwal Changhong Wang Gaël Richard 36 2 0 20 Feb 2024
Tessel: Boosting Distributed Execution of Large DNN Models via Flexible Schedule Search Zhiqi Lin Youshan Miao Guanbin Xu Cheng Li Olli Saarikivi Saeed Maleki Fan Yang 17 6 0 26 Nov 2023
An Efficient Multilingual Language Model Compression through Vocabulary Trimming Asahi Ushio Yi Zhou Jose Camacho-Collados 41 7 0 24 May 2023
XLM-V: Overcoming the Vocabulary Bottleneck in Multilingual Masked Language Models Davis Liang Hila Gonen Yuning Mao Rui Hou Naman Goyal Marjan Ghazvininejad Luke Zettlemoyer Madian Khabsa 17 72 0 25 Jan 2023
SuperScaler: Supporting Flexible DNN Parallelization via a Unified Abstraction Zhiqi Lin Youshan Miao Guodong Liu Xiaoxiang Shi Quanlu Zhang ... Xu Cao Cheng-Wu Li Mao Yang Lintao Zhang Lidong Zhou 18 6 0 21 Jan 2023
HIT-SCIR at MMNLU-22: Consistency Regularization for Multilingual Spoken Language Understanding Bo Zheng Zhouyang Li Fuxuan Wei Qiguang Chen Libo Qin Wanxiang Che 20 3 0 05 Jan 2023
Lessons learned from the evaluation of Spanish Language Models Rodrigo Agerri Eneko Agirre ELM 28 15 0 16 Dec 2022
Beyond English-Centric Bitexts for Better Multilingual Language Representation Learning Barun Patra Saksham Singhal Shaohan Huang Zewen Chi Li Dong Furu Wei Vishrav Chaudhary Xia Song 56 23 0 26 Oct 2022
Language Modelling with Pixels Phillip Rust Jonas F. Lotz Emanuele Bugliarello Elizabeth Salesky Miryam de Lhoneux Desmond Elliott VLM 38 46 0 14 Jul 2022
XLM-E: Cross-lingual Language Model Pre-training via ELECTRA Zewen Chi Shaohan Huang Li Dong Shuming Ma Bo Zheng ... Payal Bajaj Xia Song Xian-Ling Mao Heyan Huang Furu Wei 50 118 0 30 Jun 2021
MT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs Zewen Chi Li Dong Shuming Ma Shaohan Huang Xian-Ling Mao Heyan Huang Furu Wei LRM 50 71 0 18 Apr 2021
Rethinking embedding coupling in pre-trained language models Hyung Won Chung Thibault Févry Henry Tsai Melvin Johnson Sebastian Ruder 95 142 0 24 Oct 2020
Improving Multilingual Models with Language-Clustered Vocabularies Hyung Won Chung Dan Garrette Kiat Chuan Tan Jason Riesa VLM 77 65 0 24 Oct 2020
MLQA: Evaluating Cross-lingual Extractive Question Answering Patrick Lewis Barlas Oğuz Ruty Rinott Sebastian Riedel Holger Schwenk ELM 246 492 0 16 Oct 2019
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Z. Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 716 6,746 0 26 Sep 2016