Learning Deep Transformer Models for Machine Translation

5 June 2019

Jingbo Zhu

Papers citing "Learning Deep Transformer Models for Machine Translation"

50 / 152 papers shown

Title
AutoFormer: Searching Transformers for Visual Recognition Minghao Chen Houwen Peng Jianlong Fu Haibin Ling ViT 49 259 0 01 Jul 2021
Early Convolutions Help Transformers See Better Tete Xiao Mannat Singh Eric Mintun Trevor Darrell Piotr Dollár Ross B. Girshick 20 753 0 28 Jun 2021
Time-Series Representation Learning via Temporal and Contextual Contrasting Emadeldeen Eldele Mohamed Ragab Zhenghua Chen Min-man Wu C. Kwoh Xiaoli Li Cuntai Guan AI4TS 42 492 0 26 Jun 2021
Revisiting Deep Learning Models for Tabular Data Yu. V. Gorishniy Ivan Rubachev Valentin Khrulkov Artem Babenko LMTD 48 703 0 22 Jun 2021
On Adversarial Robustness of Synthetic Code Generation Mrinal Anand Pratik Kayal M. Singh 34 4 0 22 Jun 2021
Multi-head or Single-head? An Empirical Comparison for Transformer Training Liyuan Liu Jialu Liu Jiawei Han 23 32 0 17 Jun 2021
A Survey of Transformers Tianyang Lin Yuxin Wang Xiangyang Liu Xipeng Qiu ViT 53 1,088 0 08 Jun 2021
Contrastive Learning for Many-to-many Multilingual Neural Machine Translation Xiao Pan Mingxuan Wang Liwei Wu Lei Li 21 200 0 20 May 2021
SafeDrug: Dual Molecular Graph Encoders for Recommending Effective and Safe Drug Combinations Chaoqi Yang Cao Xiao Fenglong Ma Lucas Glass Jimeng Sun 30 78 0 05 May 2021
Inpainting Transformer for Anomaly Detection Jonathan Pirnay K. Chai ViT 107 165 0 28 Apr 2021
Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey Danielle Saunders AI4CE 27 86 0 14 Apr 2021
Grounding Dialogue Systems via Knowledge Graph Aware Decoding with Pre-trained Transformers Debanjan Chaudhuri Md. Rony Jens Lehmann 27 12 0 30 Mar 2021
API2Com: On the Improvement of Automatically Generated Code Comments Using API Documentations Ramin Shahbazi Rishab Sharma Fatemeh H. Fard 27 25 0 19 Mar 2021
3D Human Pose Estimation with Spatial and Temporal Transformers Ce Zheng Sijie Zhu Matías Mendieta Taojiannan Yang Chong Chen Zhengming Ding ViT 47 438 0 18 Mar 2021
Automated Query Reformulation for Efficient Search based on Query Logs From Stack Overflow Kaibo Cao Chunyang Chen Sebastian Baltes Christoph Treude Xiang Chen 50 60 0 01 Feb 2021
Investigating the Vision Transformer Model for Image Retrieval Tasks S. Gkelios Y. Boutalis S. Chatzichristofis VLM ViT 26 30 0 11 Jan 2021
An Efficient Transformer Decoder with Compressed Sub-layers Yanyang Li Ye Lin Tong Xiao Jingbo Zhu 33 29 0 03 Jan 2021
Optimizing Deeper Transformers on Small Datasets Peng Xu Dhruv Kumar Wei Yang Wenjie Zi Keyi Tang Chenyang Huang Jackie C.K. Cheung S. Prince Yanshuai Cao AI4CE 24 69 0 30 Dec 2020
Learning Light-Weight Translation Models from Deep Transformer Bei Li Ziyang Wang Hui Liu Quan Du Tong Xiao Chunliang Zhang Jingbo Zhu VLM 120 40 0 27 Dec 2020
Optimizing Transformer for Low-Resource Neural Machine Translation Ali Araabi Christof Monz VLM 41 78 0 04 Nov 2020
Dual-decoder Transformer for Joint Automatic Speech Recognition and Multilingual Speech Translation Hang Le J. Pino Changhan Wang Jiatao Gu D. Schwab Laurent Besacier 39 82 0 02 Nov 2020
Memory Attentive Fusion: External Language Model Integration for Transformer-based Sequence-to-Sequence Model Mana Ihori Ryo Masumura Naoki Makishima Tomohiro Tanaka Akihiko Takashima Shota Orihashi KELM 16 1 0 29 Oct 2020
Recent Developments on ESPnet Toolkit Boosted by Conformer Pengcheng Guo Florian Boyer Xuankai Chang Tomoki Hayashi Yosuke Higuchi ... Jing Shi Shinji Watanabe Kun Wei Wangyou Zhang Yuekai Zhang 45 262 0 26 Oct 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 41 39,551 0 22 Oct 2020
Dual Averaging is Surprisingly Effective for Deep Learning Optimization Samy Jelassi Aaron Defazio 36 4 0 20 Oct 2020
Query-Key Normalization for Transformers Alex Henry Prudhvi Raj Dachapally S. Pawar Yuxuan Chen 17 77 0 08 Oct 2020
PRover: Proof Generation for Interpretable Reasoning over Rules Swarnadeep Saha Sayan Ghosh Shashank Srivastava Joey Tianyi Zhou ReLM LRM 34 77 0 06 Oct 2020
WeChat Neural Machine Translation Systems for WMT20 Fandong Meng Jianhao Yan Yijin Liu Yuan Gao Xia Zeng ... Peng Li Ming Chen Jie Zhou Sifan Liu Hao Zhou 30 21 0 01 Oct 2020
Towards Fully 8-bit Integer Inference for the Transformer Model Ye Lin Yanyang Li Tengbo Liu Tong Xiao Tongran Liu Jingbo Zhu MQ 8 62 0 17 Sep 2020
AutoTrans: Automating Transformer Design via Reinforced Architecture Search Wei-wei Zhu Xiaoling Wang Xipeng Qiu Yuan Ni Guotong Xie 30 18 0 04 Sep 2020
Very Deep Transformers for Neural Machine Translation Xiaodong Liu Kevin Duh Liyuan Liu Jianfeng Gao 19 102 0 18 Aug 2020
Neural Machine Translation model for University Email Application Sandhya Aneja Siti Nur Afikah Bte Abdul Mazid Nagender Aneja 11 3 0 20 Jul 2020
Rewiring the Transformer with Depth-Wise LSTMs Hongfei Xu Yang Song Qiuhui Liu Josef van Genabith Deyi Xiong 47 6 0 13 Jul 2020
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding Dmitry Lepikhin HyoukJoong Lee Yuanzhong Xu Dehao Chen Orhan Firat Yanping Huang M. Krikun Noam M. Shazeer Zhehuai Chen MoE 43 1,116 0 30 Jun 2020
Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translation Jungo Kasai Nikolaos Pappas Hao Peng James Cross Noah A. Smith 41 134 0 18 Jun 2020
The Lipschitz Constant of Self-Attention Hyunjik Kim George Papamakarios A. Mnih 14 135 0 08 Jun 2020
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning Z. Yao A. Gholami Sheng Shen Mustafa Mustafa Kurt Keutzer Michael W. Mahoney ODL 39 275 0 01 Jun 2020
Many-to-Many Voice Transformer Network Hirokazu Kameoka Wen-Chin Huang Kou Tanaka Takuhiro Kaneko Nobukatsu Hojo T. Toda ViT 30 30 0 18 May 2020
Conformer: Convolution-augmented Transformer for Speech Recognition Anmol Gulati James Qin Chung-Cheng Chiu Niki Parmar Yu Zhang ... Wei Han Shibo Wang Zhengdong Zhang Yonghui Wu Ruoming Pang 104 3,044 0 16 May 2020
On the Inference Calibration of Neural Machine Translation Shuo Wang Zhaopeng Tu Shuming Shi Yang Liu 19 80 0 03 May 2020
A Transformer-based Approach for Source Code Summarization Wasi Uddin Ahmad Saikat Chakraborty Baishakhi Ray Kai-Wei Chang ViT 24 375 0 01 May 2020
Multiscale Collaborative Deep Models for Neural Machine Translation Xiangpeng Wei Heng Yu Yue Hu Yue Zhang Rongxiang Weng Weihua Luo 27 28 0 29 Apr 2020
A Survey of Deep Learning Techniques for Neural Machine Translation Shu Yang Yuxin Wang Xiaowen Chu VLM AI4TS AI4CE 24 138 0 18 Feb 2020
On Layer Normalization in the Transformer Architecture Ruibin Xiong Yunchang Yang Di He Kai Zheng Shuxin Zheng Chen Xing Huishuai Zhang Yanyan Lan Liwei Wang Tie-Yan Liu AI4CE 35 949 0 12 Feb 2020
Multi-Graph Transformer for Free-Hand Sketch Recognition Peng Xu Chaitanya K. Joshi Xavier Bresson ViT 25 85 0 24 Dec 2019
FlauBERT: Unsupervised Language Model Pre-training for French Hang Le Loïc Vial Jibril Frej Vincent Segonne Maximin Coavoux Benjamin Lecouteux A. Allauzen Benoît Crabbé Laurent Besacier D. Schwab AI4CE 49 395 0 11 Dec 2019
Lipschitz Constrained Parameter Initialization for Deep Transformers Hongfei Xu Qiuhui Liu Josef van Genabith Deyi Xiong Jingyi Zhang ODL 12 26 0 08 Nov 2019
Transformers without Tears: Improving the Normalization of Self-Attention Toan Q. Nguyen Julian Salazar 41 224 0 14 Oct 2019
Reducing Transformer Depth on Demand with Structured Dropout Angela Fan Edouard Grave Armand Joulin 43 584 0 25 Sep 2019
OpenNMT: Open-Source Toolkit for Neural Machine Translation Guillaume Klein Yoon Kim Yuntian Deng Jean Senellart Alexander M. Rush 273 1,896 0 10 Jan 2017