Mosaicking to Distill: Knowledge Distillation from Out-of-Domain Data

Mosaicking to Distill: Knowledge Distillation from Out-of-Domain Data

27 October 2021

Papers citing "Mosaicking to Distill: Knowledge Distillation from Out-of-Domain Data"

14 / 14 papers shown

Title
Multi-Level Optimal Transport for Universal Cross-Tokenizer Knowledge Distillation on Language Models Xiao Cui Mo Zhu Yulei Qin Liang Xie Wengang Zhou Hao Li 83 4 0 19 Dec 2024
MUST: A Multilingual Student-Teacher Learning approach for low-resource speech recognition Muhammad Umar Farooq Rehan Ahmad Thomas Hain 23 0 0 29 Oct 2023
Weight Averaging Improves Knowledge Distillation under Domain Shift Valeriy Berezovskiy Nikita Morozov MoMe 31 1 0 20 Sep 2023
f-Divergence Minimization for Sequence-Level Knowledge Distillation Yuqiao Wen Zichao Li Wenyu Du Lili Mou 30 53 0 27 Jul 2023
Towards domain generalisation in ASR with elitist sampling and ensemble knowledge distillation Rehan Ahmad Md. Asif Jalal Muhammad Umar Farooq A. Ollerenshaw Thomas Hain 18 2 0 01 Mar 2023
Momentum Adversarial Distillation: Handling Large Distribution Shifts in Data-Free Knowledge Distillation Kien Do Hung Le D. Nguyen Dang Nguyen Haripriya Harikumar T. Tran Santu Rana Svetha Venkatesh 18 32 0 21 Sep 2022
On-Device Domain Generalization Kaiyang Zhou Yuanhan Zhang Yuhang Zang Jingkang Yang Chen Change Loy Ziwei Liu OOD 30 6 0 15 Sep 2022
Dynamic Data-Free Knowledge Distillation by Easy-to-Hard Learning Strategy Jingru Li Sheng Zhou Liangcheng Li Haishuai Wang Zhi Yu Jiajun Bu 34 14 0 29 Aug 2022
Dense Depth Distillation with Out-of-Distribution Simulated Images Junjie Hu Chenyou Fan Mete Ozay Hualie Jiang Tin Lun Lam 16 4 0 26 Aug 2022
IDEAL: Query-Efficient Data-Free Learning from Black-box Models Jie M. Zhang Chen Chen Lingjuan Lyu 55 14 0 23 May 2022
Spot-adaptive Knowledge Distillation Jie Song Ying Chen Jingwen Ye Mingli Song 20 72 0 05 May 2022
Distilling Knowledge from Graph Convolutional Networks Yiding Yang Jiayan Qiu Xiuming Zhang Dacheng Tao Xinchao Wang 160 226 0 23 Mar 2020
A Style-Based Generator Architecture for Generative Adversarial Networks Tero Karras S. Laine Timo Aila 282 10,354 0 12 Dec 2018
Image-to-Image Translation with Conditional Adversarial Networks Phillip Isola Jun-Yan Zhu Tinghui Zhou Alexei A. Efros SSeg 212 19,450 0 21 Nov 2016