Improved Knowledge Distillation via Teacher Assistant

9 February 2019

Papers citing "Improved Knowledge Distillation via Teacher Assistant"

50 / 166 papers shown

Title
Utilizing Excess Resources in Training Neural Networks Amit Henig Raja Giryes 50 0 0 12 Jul 2022
PrUE: Distilling Knowledge from Sparse Teacher Networks Shaopu Wang Xiaojun Chen Mengzhen Kou Jinqiao Shi 19 2 0 03 Jul 2022
Knowledge Distillation of Transformer-based Language Models Revisited Chengqiang Lu Jianwei Zhang Yunfei Chu Zhengyu Chen Jingren Zhou Fei Wu Haiqing Chen Hongxia Yang VLM 27 10 0 29 Jun 2022
Variational Distillation for Multi-View Learning Xudong Tian Zhizhong Zhang Cong Wang Wensheng Zhang Yanyun Qu Lizhuang Ma Zongze Wu Yuan Xie Dacheng Tao 26 5 0 20 Jun 2022
Multi scale Feature Extraction and Fusion for Online Knowledge Distillation Panpan Zou Yinglei Teng Tao Niu 32 3 0 16 Jun 2022
Confidence-aware Self-Semantic Distillation on Knowledge Graph Embedding Yichen Liu C. Wang Defang Chen Zhehui Zhou Yan Feng Chun-Yen Chen 19 0 0 07 Jun 2022
Parameter-Efficient and Student-Friendly Knowledge Distillation Jun Rao Xv Meng Liang Ding Shuhan Qi Dacheng Tao 37 46 0 28 May 2022
A Closer Look at Self-Supervised Lightweight Vision Transformers Shaoru Wang Jin Gao Zeming Li Jian Sun Weiming Hu ViT 67 41 0 28 May 2022
Learning ReLU networks to high uniform accuracy is intractable Julius Berner Philipp Grohs F. Voigtlaender 32 4 0 26 May 2022
Knowledge Distillation from A Stronger Teacher Tao Huang Shan You Fei Wang Chao Qian Chang Xu 22 237 0 21 May 2022
Generalized Knowledge Distillation via Relationship Matching Han-Jia Ye Su Lu De-Chuan Zhan FedML 22 20 0 04 May 2022
2D Human Pose Estimation: A Survey Haoming Chen Runyang Feng Sifan Wu Hao Xu F. Zhou Zhenguang Liu 3DH 25 55 0 15 Apr 2022
Localization Distillation for Object Detection Zhaohui Zheng Rongguang Ye Ping Wang Dongwei Ren Jun Wang W. Zuo Ming-Ming Cheng 27 64 0 12 Apr 2022
Enabling All In-Edge Deep Learning: A Literature Review Praveen Joshi Mohammed Hasanuzzaman Chandra Thapa Haithem Afli T. Scully 34 22 0 07 Apr 2022
PCA-Based Knowledge Distillation Towards Lightweight and Content-Style Balanced Photorealistic Style Transfer Models Tai-Yin Chiu Danna Gurari 23 19 0 25 Mar 2022
Channel Self-Supervision for Online Knowledge Distillation Shixi Fan Xuan Cheng Xiaomin Wang Chun Yang Pan Deng Minghui Liu Jiali Deng Meilin Liu 16 1 0 22 Mar 2022
Stacked Hybrid-Attention and Group Collaborative Learning for Unbiased Scene Graph Generation Xingning Dong Tian Gan Xuemeng Song Jianlong Wu Yuan Cheng Liqiang Nie 24 92 0 18 Mar 2022
Ensemble Knowledge Guided Sub-network Search and Fine-tuning for Filter Pruning Seunghyun Lee B. Song 19 8 0 05 Mar 2022
Extracting Effective Subnetworks with Gumbel-Softmax Robin Dupont M. Alaoui H. Sahbi A. Lebois 16 6 0 25 Feb 2022
Learn From the Past: Experience Ensemble Knowledge Distillation Chaofei Wang Shaowei Zhang S. Song Gao Huang 30 4 0 25 Feb 2022
Meta Knowledge Distillation Jihao Liu Boxiao Liu Hongsheng Li Yu Liu 18 25 0 16 Feb 2022
It's All in the Head: Representation Knowledge Distillation through Classifier Sharing Emanuel Ben-Baruch M. Karklinsky Yossi Biton Avi Ben-Cohen Hussam Lawen Nadav Zamir 24 11 0 18 Jan 2022
An Incremental Learning Approach to Automatically Recognize Pulmonary Diseases from the Multi-vendor Chest Radiographs Mehreen Sirshar Taimur Hassan M. Akram S. Khan 27 33 0 07 Jan 2022
Incremental Cross-view Mutual Distillation for Self-supervised Medical CT Synthesis Chaowei Fang Liang Wang Dingwen Zhang Jun Xu Yixuan Yuan Junwei Han OOD 32 13 0 20 Dec 2021
Controlling the Quality of Distillation in Response-Based Network Compression Vibhas Kumar Vats David J. Crandall 15 1 0 19 Dec 2021
Safe Distillation Box Jingwen Ye Yining Mao Mingli Song Xinchao Wang Cheng Jin Xiuming Zhang AAML 24 13 0 05 Dec 2021
Improved Knowledge Distillation via Adversarial Collaboration Zhiqiang Liu Chengkai Huang Yanxia Liu 29 2 0 29 Nov 2021
Meta-Teacher For Face Anti-Spoofing Yunxiao Qin Zitong Yu Longbin Yan Zezheng Wang Chenxu Zhao Zhen Lei CVBM 25 61 0 12 Nov 2021
Oracle Teacher: Leveraging Target Information for Better Knowledge Distillation of CTC Models J. Yoon H. Kim Hyeon Seung Lee Sunghwan Ahn N. Kim 36 1 0 05 Nov 2021
A methodology for training homomorphicencryption friendly neural networks Moran Baruch Nir Drucker L. Greenberg Guy Moshkowich 23 14 0 05 Nov 2021
Rethinking the Knowledge Distillation From the Perspective of Model Calibration Lehan Yang Jincen Song 14 2 0 31 Oct 2021
Adaptive Distillation: Aggregating Knowledge from Multiple Paths for Efficient Distillation Sumanth Chennupati Mohammad Mahdi Kamani Zhongwei Cheng Lin Chen 26 4 0 19 Oct 2021
Mitigating Memorization of Noisy Labels via Regularization between Representations Hao Cheng Zhaowei Zhu Xing Sun Yang Liu NoLa 38 28 0 18 Oct 2021
Prune Your Model Before Distill It Jinhyuk Park Albert No VLM 46 27 0 30 Sep 2021
Partial to Whole Knowledge Distillation: Progressive Distilling Decomposed Knowledge Boosts Student Better Xuanyang Zhang Xinming Zhang Jian Sun 25 1 0 26 Sep 2021
Dynamic Knowledge Distillation for Pre-trained Language Models Lei Li Yankai Lin Shuhuai Ren Peng Li Jie Zhou Xu Sun 25 49 0 23 Sep 2021
How to Select One Among All? An Extensive Empirical Study Towards the Robustness of Knowledge Distillation in Natural Language Understanding Tianda Li Ahmad Rashid A. Jafari Pranav Sharma A. Ghodsi Mehdi Rezagholizadeh AAML 30 5 0 13 Sep 2021
Learning to Teach with Student Feedback Yitao Liu Tianxiang Sun Xipeng Qiu Xuanjing Huang VLM 20 6 0 10 Sep 2021
Knowledge Distillation Using Hierarchical Self-Supervision Augmented Distribution Chuanguang Yang Zhulin An Linhang Cai Yongjun Xu 22 15 0 07 Sep 2021
SIGN: Spatial-information Incorporated Generative Network for Generalized Zero-shot Semantic Segmentation Jiaxin Cheng Soumyaroop Nandi Premkumar Natarajan Wael AbdAlmageed VLM 32 55 0 27 Aug 2021
PocketNet: Extreme Lightweight Face Recognition Network using Neural Architecture Search and Multi-Step Knowledge Distillation Fadi Boutros Patrick Siebke Marcel Klemt Naser Damer Florian Kirchbuchner Arjan Kuijper CVBM 33 63 0 24 Aug 2021
Multi-Target Domain Adaptation with Collaborative Consistency Learning Takashi Isobe Xu Jia Shuaijun Chen Jianzhong He Yongjie Shi Jian-zhuo Liu Huchuan Lu Shengjin Wang 30 84 0 07 Jun 2021
Privileged Graph Distillation for Cold Start Recommendation Shuai Wang Kun Zhang Le Wu Haiping Ma Richang Hong Meng Wang 12 28 0 31 May 2021
AutoReCon: Neural Architecture Search-based Reconstruction for Data-free Compression Baozhou Zhu P. Hofstee J. Peltenburg Jinho Lee Zaid Al-Ars 24 22 0 25 May 2021
BasisNet: Two-stage Model Synthesis for Efficient Inference Ruotong Wang Chun-Te Chu A. Zhmoginov Andrew G. Howard Brendan Jou Yukun Zhu Li Zhang R. Hwa Adriana Kovashka 3DH 31 7 0 07 May 2021
Distilling and Transferring Knowledge via cGAN-generated Samples for Image Classification and Regression Xin Ding Z. J. Wang Zuheng Xu Z. Jane Wang William J. Welch 36 22 0 07 Apr 2021
Distilling Object Detectors via Decoupled Features Jianyuan Guo Kai Han Yunhe Wang Han Wu Xinghao Chen Chunjing Xu Chang Xu 41 199 0 26 Mar 2021
Compacting Deep Neural Networks for Internet of Things: Methods and Applications Ke Zhang Hanbo Ying Hongning Dai Lin Li Yuangyuang Peng Keyi Guo Hongfang Yu 16 38 0 20 Mar 2021
Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges Yoshitomo Matsubara Marco Levorato Francesco Restuccia 33 199 0 08 Mar 2021
Localization Distillation for Dense Object Detection Zhaohui Zheng Rongguang Ye Ping Wang Dongwei Ren W. Zuo Qibin Hou Ming-Ming Cheng ObjD 101 115 0 24 Feb 2021