Improving Multi-Task Deep Neural Networks via Knowledge Distillation for
Natural Language Understanding

Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding

20 April 2019

Xiaodong Liu

Papers citing "Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding"

16 / 16 papers shown

Title
Measuring Social Biases in Masked Language Models by Proxy of Prediction Quality Rahul Zalkikar Kanchan Chandra 58 1 0 21 Feb 2024
Multilingual Neural Machine Translation with Knowledge Distillation Xu Tan Yi Ren Di He Tao Qin Zhou Zhao Tie-Yan Liu 54 248 0 27 Feb 2019
Multi-Task Deep Neural Networks for Natural Language Understanding Xiaodong Liu Pengcheng He Weizhu Chen Jianfeng Gao AI4CE 77 1,269 0 31 Jan 2019
Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks Jason Phang Thibault Févry Samuel R. Bowman 44 467 0 02 Nov 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 655 93,936 0 11 Oct 2018
Neural Approaches to Conversational AI Jianfeng Gao Michel Galley Lihong Li 70 672 0 21 Sep 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 457 7,080 0 20 Apr 2018
Stochastic Answer Networks for Machine Reading Comprehension Xiaodong Liu Yelong Shen Kevin Duh Jianfeng Gao RALM 28 198 0 10 Dec 2017
FusionNet: Fusing via Fully-Aware Attention with Application to Machine Comprehension Hsin-Yuan Huang Chenguang Zhu Yelong Shen Weizhu Chen FedML 55 183 0 16 Nov 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 264 129,831 0 12 Jun 2017
Layer Normalization Jimmy Lei Ba J. Kiros Geoffrey E. Hinton 175 10,412 0 21 Jul 2016
Net2Net: Accelerating Learning via Knowledge Transfer Tianqi Chen Ian Goodfellow Jonathon Shlens 56 663 0 18 Nov 2015
Bayesian Dark Knowledge Masashi Sugiyama Vivek Rathod R. Garnett Max Welling BDL UQCV 43 258 0 14 Jun 2015
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 91 19,448 0 09 Mar 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 316 149,474 0 22 Dec 2014
Natural Language Processing (almost) from Scratch R. Collobert Jason Weston Léon Bottou Michael Karlen Koray Kavukcuoglu Pavel P. Kuksa 78 7,711 0 02 Mar 2011

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.