Learning Light-Weight Translation Models from Deep Transformer

Learning Light-Weight Translation Models from Deep Transformer

27 December 2020

Chunliang Zhang

Jingbo Zhu

Papers citing "Learning Light-Weight Translation Models from Deep Transformer"

16 / 16 papers shown

Title
Multi-Path Transformer is Better: A Case Study on Neural Machine Translation Ye Lin Shuhan Zhou Yanyang Li Anxiang Ma Tong Xiao Jingbo Zhu 22 0 0 10 May 2023
Improved Knowledge Distillation for Pre-trained Language Models via Knowledge Selection Chenglong Wang Yi Lu Yongyu Mu Yimin Hu Tong Xiao Jingbo Zhu 32 8 0 01 Feb 2023
Optimizing Deep Transformers for Chinese-Thai Low-Resource Translation Wenjie Hao Hongfei Xu Lingling Mu Hongying Zan MoE 16 4 0 24 Dec 2022
EIT: Enhanced Interactive Transformer Tong Zheng Bei Li Huiwen Bao Tong Xiao Jingbo Zhu 26 2 0 20 Dec 2022
SMaLL-100: Introducing Shallow Multilingual Machine Translation Model for Low-Resource Languages Alireza Mohammadshahi Vassilina Nikoulina Alexandre Berard Caroline Brun James Henderson Laurent Besacier VLM MoE LRM 21 20 0 20 Oct 2022
Metadata Archaeology: Unearthing Data Subsets by Leveraging Training Dynamics Shoaib Ahmed Siddiqui Nitarshan Rajkumar Tegan Maharaj David M. Krueger Sara Hooker 37 27 0 20 Sep 2022
What Do Compressed Multilingual Machine Translation Models Forget? Alireza Mohammadshahi Vassilina Nikoulina Alexandre Berard Caroline Brun James Henderson Laurent Besacier AI4CE 40 9 0 22 May 2022
The NiuTrans Machine Translation Systems for WMT21 Yuhao Zhang Tao Zhou Bin Wei Runzhe Cao Yongyu Mu ... Weiqiao Shan Yinqiao Li Bei Li Tong Xiao Jingbo Zhu 30 17 0 22 Sep 2021
The NiuTrans System for the WMT21 Efficiency Task Chenglong Wang Chi Hu Yongyu Mu Zhongxiang Yan Siming Wu ... Hang Cao Bei Li Ye Lin Tong Xiao Jingbo Zhu 22 2 0 16 Sep 2021
An Effective Non-Autoregressive Model for Spoken Language Understanding Lizhi Cheng Weijia Jia Wenmian Yang OffRL 24 15 0 16 Aug 2021
The NiuTrans End-to-End Speech Translation System for IWSLT 2021 Offline Task Chen Xu Xiaoqian Liu Xiaowen Liu Laohu Wang Canan Huang Tong Xiao Jingbo Zhu 29 5 0 06 Jul 2021
Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoders Chen Xu Bojie Hu Yanyang Li Yuhao Zhang Shen Huang Qi Ju Tong Xiao Jingbo Zhu 17 75 0 12 May 2021
Rewiring the Transformer with Depth-Wise LSTMs Hongfei Xu Yang Song Qiuhui Liu Josef van Genabith Deyi Xiong 29 6 0 13 Jul 2020
Knowledge Distillation: A Survey Jianping Gou B. Yu Stephen J. Maybank Dacheng Tao VLM 19 2,835 0 09 Jun 2020
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Z. Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 716 6,743 0 26 Sep 2016
Improving neural networks by preventing co-adaptation of feature detectors Geoffrey E. Hinton Nitish Srivastava A. Krizhevsky Ilya Sutskever Ruslan Salakhutdinov VLM 266 7,634 0 03 Jul 2012