QKD: Quantization-aware Knowledge Distillation

QKD: Quantization-aware Knowledge Distillation

28 November 2019

Chirag I. Patel

Papers citing "QKD: Quantization-aware Knowledge Distillation"

18 / 18 papers shown

Title
On the Impact of White-box Deployment Strategies for Edge AI on Latency and Model Performance Jaskirat Singh Bram Adams Ahmed E. Hassan VLM 43 0 0 01 Nov 2024
Quantized Feature Distillation for Network Quantization Kevin Zhu Yin He Jianxin Wu MQ 29 9 0 20 Jul 2023
Self-Distilled Quantization: Achieving High Compression Rates in Transformer-Based Language Models James OÑeill Sourav Dutta VLM MQ 40 1 0 12 Jul 2023
MobileNMT: Enabling Translation in 15MB and 30ms Ye Lin Xiaohui Wang Zhexi Zhang Mingxuan Wang Tong Xiao Jingbo Zhu MQ 30 1 0 07 Jun 2023
Vertical Layering of Quantized Neural Networks for Heterogeneous Inference Hai Wu Ruifei He Hao Hao Tan Xiaojuan Qi Kaibin Huang MQ 24 2 0 10 Dec 2022
Bimodal Distributed Binarized Neural Networks T. Rozen Moshe Kimhi Brian Chmiel A. Mendelson Chaim Baskin MQ 47 4 0 05 Apr 2022
Arch-Net: Model Distillation for Architecture Agnostic Model Deployment Weixin Xu Zipeng Feng Shuangkang Fang Song Yuan Yi Yang Shuchang Zhou MQ 27 1 0 01 Nov 2021
PQK: Model Compression via Pruning, Quantization, and Knowledge Distillation Jang-Hyun Kim Simyung Chang Nojun Kwak 24 44 0 25 Jun 2021
ERNIE-Tiny : A Progressive Distillation Framework for Pretrained Transformer Compression Weiyue Su Xuyi Chen Shi Feng Jiaxiang Liu Weixin Liu Yu Sun Hao Tian Hua Wu Haifeng Wang 34 13 0 04 Jun 2021
Training Multi-bit Quantized and Binarized Networks with A Learnable Symmetric Quantizer Phuoc Pham J. Abraham Jaeyong Chung MQ 37 11 0 01 Apr 2021
Once Quantization-Aware Training: High Performance Extremely Low-bit Architecture Search Mingzhu Shen Feng Liang Ruihao Gong Yuhang Li Chuming Li Chen Lin F. Yu Junjie Yan Wanli Ouyang MQ 28 36 0 09 Oct 2020
Stochastic Precision Ensemble: Self-Knowledge Distillation for Quantized Deep Neural Networks Yoonho Boo Sungho Shin Jungwook Choi Wonyong Sung MQ 30 29 0 30 Sep 2020
TernaryBERT: Distillation-aware Ultra-low Bit BERT Wei Zhang Lu Hou Yichun Yin Lifeng Shang Xiao Chen Xin Jiang Qun Liu MQ 33 208 0 27 Sep 2020
Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming Itay Hubara Yury Nahshan Y. Hanani Ron Banner Daniel Soudry MQ 35 122 0 14 Jun 2020
Knowledge Distillation: A Survey Jianping Gou B. Yu Stephen J. Maybank Dacheng Tao VLM 19 2,843 0 09 Jun 2020
Feature Fusion for Online Mutual Knowledge Distillation Jangho Kim Minsung Hyun Inseop Chung Nojun Kwak FedML 26 91 0 19 Apr 2019
Knowledge Distillation by On-the-Fly Native Ensemble Xu Lan Xiatian Zhu S. Gong 200 473 0 12 Jun 2018
OpenNMT: Open-Source Toolkit for Neural Machine Translation Guillaume Klein Yoon Kim Yuntian Deng Jean Senellart Alexander M. Rush 273 1,896 0 10 Jan 2017