Dynamic Multi-Branch Layers for On-Device Neural Machine Translation

14 May 2021

Zhixing Tan

Zeyuan Yang

Meng Zhang

Qun Liu

Maosong Sun

Yang Liu

AI4CE

ArXiv PDF HTML

Papers citing "Dynamic Multi-Branch Layers for On-Device Neural Machine Translation"

23 / 23 papers shown

Title
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity W. Fedus Barret Zoph Noam M. Shazeer MoE 70 2,136 0 11 Jan 2021
Extremely Low Bit Transformer Quantization for On-Device Neural Machine Translation Insoo Chung Byeongwook Kim Yoonjung Choi S. Kwon Yongkweon Jeon Baeseong Park Sangha Kim Dongsoo Lee MQ 41 27 0 16 Sep 2020
HAT: Hardware-Aware Transformers for Efficient Natural Language Processing Hanrui Wang Zhanghao Wu Zhijian Liu Han Cai Ligeng Zhu Chuang Gan Song Han 79 259 0 28 May 2020
Lite Transformer with Long-Short Range Attention Zhanghao Wu Zhijian Liu Ji Lin Chengyue Wu Song Han 53 321 0 24 Apr 2020
Controlling Computation versus Quality for Neural Sequence Models Ankur Bapna N. Arivazhagan Orhan Firat 42 30 0 17 Feb 2020
Fully Quantized Transformer for Machine Translation Gabriele Prato Ella Charlaix Mehdi Rezagholizadeh MQ 34 70 0 17 Oct 2019
The Evolved Transformer David R. So Chen Liang Quoc V. Le ViT 89 462 0 30 Jan 2019
A Call for Clarity in Reporting BLEU Scores Matt Post 116 2,941 0 23 Apr 2018
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks Jonathan Frankle Michael Carbin 187 3,433 0 09 Mar 2018
To prune, or not to prune: exploring the efficacy of pruning for model compression Michael Zhu Suyog Gupta 159 1,262 0 05 Oct 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 526 129,831 0 12 Jun 2017
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer Noam M. Shazeer Azalia Mirhoseini Krzysztof Maziarz Andy Davis Quoc V. Le Geoffrey E. Hinton J. Dean MoE 192 2,614 0 23 Jan 2017
Sequence-Level Knowledge Distillation Yoon Kim Alexander M. Rush 95 1,109 0 25 Jun 2016
Conditional Computation in Neural Networks for faster models Emmanuel Bengio Pierre-Luc Bacon Joelle Pineau Doina Precup AI4CE 118 320 0 19 Nov 2015
Neural Machine Translation of Rare Words with Subword Units Rico Sennrich Barry Haddow Alexandra Birch 174 7,683 0 31 Aug 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 1.2K 149,474 0 22 Dec 2014
Sequence to Sequence Learning with Neural Networks Ilya Sutskever Oriol Vinyals Quoc V. Le AIMat 326 20,491 0 10 Sep 2014
Neural Machine Translation by Jointly Learning to Align and Translate Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio AIMat 427 27,205 0 01 Sep 2014
Exponentially Increasing the Capacity-to-Computation Ratio for Conditional Computation in Deep Learning Kyunghyun Cho Yoshua Bengio 71 38 0 28 Jun 2014
Low-Rank Approximations for Conditional Feedforward Computation in Deep Neural Networks Andrew S. Davis I. Arel 74 78 0 16 Dec 2013
Learning Factored Representations in a Deep Mixture of Experts David Eigen MarcÁurelio Ranzato Ilya Sutskever MoE 78 373 0 16 Dec 2013
Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation Yoshua Bengio Nicholas Léonard Aaron Courville 338 3,099 0 15 Aug 2013
Deep Learning of Representations: Looking Forward Yoshua Bengio 167 679 0 02 May 2013