Information Theoretic Representation Distillation

1 December 2021

Papers citing "Information Theoretic Representation Distillation"

17 / 17 papers shown

Title
HDC: Hierarchical Distillation for Multi-level Noisy Consistency in Semi-Supervised Fetal Ultrasound Segmentation Tran Quoc Khanh Le Nguyen Lan Vi Vu Ha-Hieu Pham Xuan-Loc Huynh T. Nguyen Minh Huu Nhat Le Quan Nguyen Hien Nguyen 46 0 0 14 Apr 2025
Quantifying Knowledge Distillation Using Partial Information Decomposition Pasan Dissanayake Faisal Hamman Barproda Halder Ilia Sucholutsky Qiuyi Zhang Sanghamitra Dutta 36 0 0 12 Nov 2024
On the Surprising Efficacy of Distillation as an Alternative to Pre-Training Small Models Sean Farhat Deming Chen 47 0 0 04 Apr 2024
Learning to Project for Cross-Task Knowledge Distillation Dylan Auty Roy Miles Benedikt Kolbeinsson K. Mikolajczyk 45 0 0 21 Mar 2024
$V_kD:$ Improving Knowledge Distillation using Orthogonal Projections Roy Miles Ismail Elezi Jiankang Deng 52 10 0 10 Mar 2024
Knowledge Distillation Based on Transformed Teacher Matching Kaixiang Zheng En-Hui Yang 32 19 0 17 Feb 2024
Multi-Label Knowledge Distillation Penghui Yang Ming-Kun Xie Chen-Chen Zong Lei Feng Gang Niu Masashi Sugiyama Sheng-Jun Huang 36 10 0 12 Aug 2023
Teacher-Student Architecture for Knowledge Distillation: A Survey Chengming Hu Xuan Li Danyang Liu Haolun Wu Xi Chen Ju Wang Xue Liu 21 16 0 08 Aug 2023
MobileVOS: Real-Time Video Object Segmentation Contrastive Learning meets Knowledge Distillation Roy Miles M. K. Yucel Bruno Manganelli Albert Saà-Garriga VOS 38 24 0 14 Mar 2023
Robust and Fast Measure of Information via Low-rank Representation Yuxin Dong Tieliang Gong Shujian Yu Hong Chen Chen Li 28 1 0 30 Nov 2022
Teacher-Student Architecture for Knowledge Learning: A Survey Chengming Hu Xuan Li Dan Liu Xi Chen Ju Wang Xue Liu 27 35 0 28 Oct 2022
Optimal Randomized Approximations for Matrix based Renyi's Entropy Yuxin Dong Tieliang Gong Shujian Yu Chen Li 32 7 0 16 May 2022
Computationally Efficient Approximations for Matrix-based Renyi's Entropy Tieliang Gong Yuxin Dong Shujian Yu B. Dong 67 2 0 27 Dec 2021
Reconstructing Pruned Filters using Cheap Spatial Transformations Roy Miles K. Mikolajczyk 29 0 0 25 Oct 2021
Distilling Knowledge via Knowledge Review Pengguang Chen Shu Liu Hengshuang Zhao Jiaya Jia 155 423 0 19 Apr 2021
Forward and Backward Information Retention for Accurate Binary Neural Networks Haotong Qin Ruihao Gong Xianglong Liu Mingzhu Shen Ziran Wei F. Yu Jingkuan Song MQ 133 324 0 24 Sep 2019
Understanding Autoencoders with Information Theoretic Concepts Shujian Yu José C. Príncipe AI4CE 49 132 0 30 Mar 2018