Teacher Guided Training: An Efficient Framework for Knowledge Transfer

14 August 2022

Papers citing "Teacher Guided Training: An Efficient Framework for Knowledge Transfer"

22 / 22 papers shown

Title
Preventing Catastrophic Forgetting and Distribution Mismatch in Knowledge Distillation via Synthetic Data Kuluhan Binici N. Pham T. Mitra K. Leman 38 40 0 11 Aug 2021
Domain-matched Pre-training Tasks for Dense Retrieval Barlas Oğuz Kushal Lakhotia Anchit Gupta Patrick Lewis Vladimir Karpukhin ... Xilun Chen Sebastian Riedel Wen-tau Yih Sonal Gupta Yashar Mehdad RALM 33 66 0 28 Jul 2021
RSG: A Simple but Effective Module for Learning Imbalanced Datasets Jianfeng Wang Thomas Lukasiewicz Xiaolin Hu Jianfei Cai Zhenghua Xu SyDa 50 84 0 18 Jun 2021
MATE-KD: Masked Adversarial TExt, a Companion to Knowledge Distillation Ahmad Rashid Vasileios Lioutas Mehdi Rezagholizadeh AAML 47 37 0 12 May 2021
Perceiver: General Perception with Iterative Attention Andrew Jaegle Felix Gimeno Andrew Brock Andrew Zisserman Oriol Vinyals João Carreira VLM ViT MDE 115 999 0 04 Mar 2021
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 284 4,873 0 24 Feb 2021
PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them Patrick Lewis Yuxiang Wu Linqing Liu Pasquale Minervini Heinrich Küttler Aleksandra Piktus Pontus Stenetorp Sebastian Riedel RALM 58 233 0 13 Feb 2021
Long-tail learning via logit adjustment A. Menon Sadeep Jayasumana A. S. Rawat Himanshu Jain Andreas Veit Sanjiv Kumar 76 696 0 14 Jul 2020
MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices Zhiqing Sun Hongkun Yu Xiaodan Song Renjie Liu Yiming Yang Denny Zhou MQ 73 807 0 06 Apr 2020
Meta Pseudo Labels Hieu H. Pham Zihang Dai Qizhe Xie Minh-Thang Luong Quoc V. Le VLM 317 662 0 23 Mar 2020
Self-training with Noisy Student improves ImageNet classification Qizhe Xie Minh-Thang Luong Eduard H. Hovy Quoc V. Le NoLa 152 2,375 0 11 Nov 2019
Knowledge Distillation from Internal Representations Gustavo Aguilar Yuan Ling Yu Zhang Benjamin Yao Xing Fan Edward Guo 43 179 0 08 Oct 2019
On the Efficacy of Knowledge Distillation Ligang He Rui Mao 66 603 0 03 Oct 2019
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter Victor Sanh Lysandre Debut Julien Chaumond Thomas Wolf 54 7,386 0 02 Oct 2019
Zero-Shot Knowledge Distillation in Deep Networks Gaurav Kumar Nayak Konda Reddy Mopuri Vaisakh Shaj R. Venkatesh Babu Anirban Chakraborty 54 245 0 20 May 2019
Searching for MobileNetV3 Andrew G. Howard Mark Sandler Grace Chu Liang-Chieh Chen Bo Chen ... Yukun Zhu Ruoming Pang Vijay Vasudevan Quoc V. Le Hartwig Adam 219 6,685 0 06 May 2019
Unsupervised Data Augmentation for Consistency Training Qizhe Xie Zihang Dai Eduard H. Hovy Minh-Thang Luong Quoc V. Le 78 2,306 0 29 Apr 2019
Large-Scale Long-Tailed Recognition in an Open World Ziwei Liu Zhongqi Miao Xiaohang Zhan Jiayun Wang Boqing Gong Stella X. Yu 87 1,148 0 10 Apr 2019
Label Propagation for Deep Semi-supervised Learning Ahmet Iscen Giorgos Tolias Yannis Avrithis Ondřej Chum SSL 52 622 0 09 Apr 2019
Detecting Overfitting of Deep Generative Networks via Latent Recovery Ryan Webster Julien Rabin Loïc Simon F. Jurie GAN 19 99 0 09 Jan 2019
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference Adina Williams Nikita Nangia Samuel R. Bowman 298 4,444 0 18 Apr 2017
Temporal Ensembling for Semi-Supervised Learning S. Laine Timo Aila UQCV 142 2,543 0 07 Oct 2016