Reducing Transformer Depth on Demand with Structured Dropout

Reducing Transformer Depth on Demand with Structured Dropout

25 September 2019

Angela Fan

ArXiv (abs)PDF HTML

Papers citing "Reducing Transformer Depth on Demand with Structured Dropout"

6 / 406 papers shown

Title
End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures Gabriel Synnaeve Qiantong Xu Jacob Kahn Tatiana Likhomanenko Edouard Grave Vineel Pratap Anuroop Sriram Vitaliy Liptchinsky R. Collobert SSL AI4TS 134 248 0 19 Nov 2019
CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEB Holger Schwenk Guillaume Wenzek Sergey Edunov Edouard Grave Armand Joulin 98 263 0 10 Nov 2019
Structured Pruning of a BERT-based Question Answering Model J. Scott McCarley Rishav Chakravarti Avirup Sil 98 53 0 14 Oct 2019
Structured Pruning of Large Language Models Ziheng Wang Jeremy Wohlwend Tao Lei 96 293 0 10 Oct 2019
TinyBERT: Distilling BERT for Natural Language Understanding Xiaoqi Jiao Yichun Yin Lifeng Shang Xin Jiang Xiao Chen Linlin Li F. Wang Qun Liu VLM 130 1,881 0 23 Sep 2019
DDPNAS: Efficient Neural Architecture Search via Dynamic Distribution Pruning Xiawu Zheng Chenyi Yang Shaokun Zhang Yan Wang Baochang Zhang Yongjian Wu Yunsheng Wu Ling Shao Rongrong Ji 88 22 0 28 May 2019