Reinforced Multi-Teacher Selection for Knowledge Distillation

v1v2 (latest)

Reinforced Multi-Teacher Selection for Knowledge Distillation

11 December 2020

ArXiv (abs)PDF HTML

Papers citing "Reinforced Multi-Teacher Selection for Knowledge Distillation"

19 / 19 papers shown

Title
Heuristic-Free Multi-Teacher Learning Huy Thong Nguyen En-Hung Chu Lenord Melvix Jazon Jiao Chunglin Wen Benjamin Louie 105 0 0 19 Nov 2024
MIDAS: Multi-level Intent, Domain, And Slot Knowledge Distillation for Multi-turn NLU Yan Li So-Eon Kim Seong-Bae Park S. Han 80 1 0 15 Aug 2024
Model Compression with Two-stage Multi-teacher Knowledge Distillation for Web Question Answering System Ze Yang Linjun Shou Ming Gong Wutao Lin Daxin Jiang 60 94 0 18 Oct 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 371 6,463 0 26 Sep 2019
TinyBERT: Distilling BERT for Natural Language Understanding Xiaoqi Jiao Yichun Yin Lifeng Shang Xin Jiang Xiao Chen Linlin Li F. Wang Qun Liu VLM 109 1,860 0 23 Sep 2019
Patient Knowledge Distillation for BERT Model Compression S. Sun Yu Cheng Zhe Gan Jingjing Liu 134 843 0 25 Aug 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 665 24,528 0 26 Jul 2019
XLNet: Generalized Autoregressive Pretraining for Language Understanding Zhilin Yang Zihang Dai Yiming Yang J. Carbonell Ruslan Salakhutdinov Quoc V. Le AI4CE 232 8,433 0 19 Jun 2019
Distilling Task-Specific Knowledge from BERT into Simple Neural Networks Raphael Tang Yao Lu Linqing Liu Lili Mou Olga Vechtomova Jimmy J. Lin 69 421 0 28 Mar 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,114 0 11 Oct 2018
Reinforcement Learning for Relation Classification from Noisy Data Jun Feng Minlie Huang Li Zhao Yang Yang Xiaoyan Zhu NoLa 59 343 0 24 Aug 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,182 0 20 Apr 2018
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference Adina Williams Nikita Nangia Samuel R. Bowman 524 4,492 0 18 Apr 2017
An Actor-Critic Algorithm for Sequence Prediction Dzmitry Bahdanau Philemon Brakel Kelvin Xu Anirudh Goyal Ryan J. Lowe Joelle Pineau Aaron Courville Yoshua Bengio 133 639 0 24 Jul 2016
Sequence-Level Knowledge Distillation Yoon Kim Alexander M. Rush 122 1,120 0 25 Jun 2016
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 289 8,160 0 16 Jun 2016
Learning both Weights and Connections for Efficient Neural Networks Song Han Jeff Pool J. Tran W. Dally CVBM 313 6,681 0 08 Jun 2015
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 362 19,660 0 09 Mar 2015
Compressing Deep Convolutional Networks using Vector Quantization Yunchao Gong Liu Liu Ming Yang Lubomir D. Bourdev MQ 165 1,170 0 18 Dec 2014

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.