Fixing MoE Over-Fitting on Low-Resource Languages in Multilingual
Machine Translation

Fixing MoE Over-Fitting on Low-Resource Languages in Multilingual Machine Translation

15 December 2022

Papers citing "Fixing MoE Over-Fitting on Low-Resource Languages in Multilingual Machine Translation"

9 / 9 papers shown

Title
A Unified Framework for Iris Anti-Spoofing: Introducing Iris Anti-Spoofing Cross-Domain-Testing Protocol and Masked-MoE Method Hang Zou Chenxi Du A. Liu Yuan Zhang Jing Liu Mingchuan Yang Jun Wan Hui Zhang Zhenan Sun 37 0 0 19 Aug 2024
Investigating Neural Machine Translation for Low-Resource Languages: Using Bavarian as a Case Study Wan-Hua Her Udo Kruschwitz 37 4 0 12 Apr 2024
SiRA: Sparse Mixture of Low Rank Adaptation Yun Zhu Nevan Wichers Chu-Cheng Lin Xinyi Wang Tianlong Chen ... Han Lu Canoee Liu Liangchen Luo Jindong Chen Lei Meng MoE 25 27 0 15 Nov 2023
Task-Based MoE for Multitask Multilingual Machine Translation Hai Pham Young Jin Kim Subhabrata Mukherjee David P. Woodruff Barnabás Póczós Hany Awadalla MoE 30 4 0 30 Aug 2023
Tutel: Adaptive Mixture-of-Experts at Scale Changho Hwang Wei Cui Yifan Xiong Ziyue Yang Ze Liu ... Joe Chau Peng Cheng Fan Yang Mao Yang Y. Xiong MoE 97 110 0 07 Jun 2022
Multilingual Machine Translation with Hyper-Adapters Christos Baziotis Mikel Artetxe James Cross Shruti Bhosale 72 21 0 22 May 2022
Scalable and Efficient MoE Training for Multitask Multilingual Models Young Jin Kim A. A. Awan Alexandre Muzio Andres Felipe Cruz Salinas Liyang Lu Amr Hendy Samyam Rajbhandari Yuxiong He Hany Awadalla MoE 98 84 0 22 Sep 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 249 4,489 0 23 Jan 2020
Dynamic Sentence Sampling for Efficient Training of Neural Machine Translation Rui Wang Masao Utiyama Eiichiro Sumita 27 27 0 01 May 2018