Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillation

19 May 2021

Papers citing "Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillation"

35 / 35 papers shown

Title
Rethinking Circuit Completeness in Language Models: AND, OR, and ADDER Gates Hang Chen Jiaying Zhu Xinyu Yang Wenya Wang LRM 14 0 0 15 May 2025
Federated Learning with LoRA Optimized DeiT and Multiscale Patch Embedding for Secure Eye Disease Recognition Md. Naimur Asif Borno Md Sakib Hossain Shovon MD Hanif Sikder Iffat Firozy Rimi Tahani Jaser Alahmadi Mohammad Ali Moni MedIm 31 0 0 11 May 2025
Learning Critically: Selective Self Distillation in Federated Learning on Non-IID Data Yuting He Yiqiang Chen Xiaodong Yang H. Yu Yi-Hua Huang Yang Gu FedML 63 21 0 20 Apr 2025
Efficient Split Learning LSTM Models for FPGA-based Edge IoT Devices Romina Soledad Molina Vukan Ninkovic D. Vukobratović Maria Liz Crespo Marco Zennaro 45 0 0 12 Feb 2025
Learning an Adaptive and View-Invariant Vision Transformer for Real-Time UAV Tracking You Wu Yongxin Li Mengyuan Liu Xucheng Wang Xiangyang Yang Hengzhou Ye Dan Zeng Qijun Zhao Shuiwang Li 179 0 0 28 Dec 2024
Spatially Constrained Transformer with Efficient Global Relation Modelling for Spatio-Temporal Prediction Ashutosh Sao Simon Gottschalk AI4TS 39 0 0 11 Nov 2024
Dataset Distillation via Knowledge Distillation: Towards Efficient Self-Supervised Pre-Training of Deep Networks S. Joshi Jiayi Ni Baharan Mirzasoleiman DD 72 2 0 03 Oct 2024
Overcoming Uncertain Incompleteness for Robust Multimodal Sequential Diagnosis Prediction via Curriculum Data Erasing Guided Knowledge Distillation Heejoon Koo 48 0 0 28 Jul 2024
Temporal Feature Matters: A Framework for Diffusion Model Quantization Yushi Huang Ruihao Gong Xianglong Liu Jing Liu Yuhang Li Jiwen Lu Dacheng Tao DiffM MQ 49 0 0 28 Jul 2024
Enhancing Multilingual Voice Toxicity Detection with Speech-Text Alignment Joseph Liu Mahesh Kumar Nandwana Janne Pylkkönen Hannes Heikinheimo Morgan McGuire 37 1 0 14 Jun 2024
UniBridge: A Unified Approach to Cross-Lingual Transfer Learning for Low-Resource Languages Trinh Pham Khoi M. Le Luu Anh Tuan 42 1 0 14 Jun 2024
DistilDoc: Knowledge Distillation for Visually-Rich Document Applications Jordy Van Landeghem Subhajit Maity Ayan Banerjee Matthew Blaschko Marie-Francine Moens Josep Lladós Sanket Biswas 50 2 0 12 Jun 2024
FedDr+: Stabilizing Dot-regression with Global Feature Distillation for Federated Learning Seongyoon Kim Minchan Jeong Sungnyun Kim Sungwoo Cho Sumyeong Ahn Se-Young Yun FedML 50 0 0 04 Jun 2024
GIFT: Unlocking Full Potential of Labels in Distilled Dataset at Near-zero Cost Xinyi Shang Peng Sun Tao Lin 53 3 0 23 May 2024
Object criticality for safer navigation Andrea Ceccarelli Leonardo Montecchi 41 0 0 25 Apr 2024
Reinforcement Learning as a Parsimonious Alternative to Prediction Cascades: A Case Study on Image Segmentation Bharat Srikishan Anika Tabassum S. Allu Ramakrishnan Kannan Nikhil Muralidhar 53 1 0 19 Feb 2024
An Empirical Investigation into the Effect of Parameter Choices in Knowledge Distillation Md Arafat Sultan Aashka Trivedi Parul Awasthy Avirup Sil 38 0 0 12 Jan 2024
Extracting Low-/High- Frequency Knowledge from Graph Neural Networks and Injecting it into MLPs: An Effective GNN-to-MLP Distillation Framework Lirong Wu Haitao Lin Yufei Huang Tianyu Fan Stan Z. Li 24 29 0 18 May 2023
Knowledge Distillation Under Ideal Joint Classifier Assumption Huayu Li Xiwen Chen G. Ditzler Janet Roveda Ao Li 18 1 0 19 Apr 2023
Smooth and Stepwise Self-Distillation for Object Detection Jieren Deng Xiaoxia Zhou Hao Tian Zhihong Pan Derek Aguiar ObjD 31 0 0 09 Mar 2023
3D generation on ImageNet Ivan Skorokhodov Aliaksandr Siarohin Yinghao Xu Jian Ren Hsin-Ying Lee Peter Wonka Sergey Tulyakov 69 55 0 02 Mar 2023
DSD $^2$ : Can We Dodge Sparse Double Descent and Compress the Neural Network Worry-Free? Victor Quétu Enzo Tartaglione 32 7 0 02 Mar 2023
A Comprehensive Survey on Graph Summarization with Graph Neural Networks Nasrin Shabani Jia Wu Amin Beheshti Quan.Z Sheng Jin Foo Venus Haghighi Ambreen Hanif Maryam Shahabikargar GNN AI4TS 40 12 0 13 Feb 2023
Bridging Images and Videos: A Simple Learning Framework for Large Vocabulary Video Object Detection Sanghyun Woo Kwanyong Park Seoung Wug Oh In So Kweon Joon-Young Lee VLM VOS 28 6 0 20 Dec 2022
Compressing Transformer-based self-supervised models for speech processing Tzu-Quan Lin Tsung-Huan Yang Chun-Yao Chang Kuang-Ming Chen Tzu-hsun Feng Hung-yi Lee Hao Tang 40 6 0 17 Nov 2022
Multimodal Dyadic Impression Recognition via Listener Adaptive Cross-Domain Fusion Yuanchao Li P. Bell Catherine Lai 28 3 0 09 Nov 2022
GCT: Gated Contextual Transformer for Sequential Audio Tagging Yuanbo Hou Yun Wang Wenwu Wang Dick Botteldooren 33 0 0 22 Oct 2022
PROD: Progressive Distillation for Dense Retrieval Zhenghao Lin Yeyun Gong Xiao Liu Hang Zhang Chen Lin ... Jian Jiao Jing Lu Daxin Jiang Rangan Majumder Nan Duan 51 27 0 27 Sep 2022
Colonoscopy Landmark Detection using Vision Transformers Aniruddha Tamhane Tseéla Mida Erez Posner Moshe Bouhnik ViT MedIm 59 5 0 22 Sep 2022
PANDA: Prompt Transfer Meets Knowledge Distillation for Efficient Model Adaptation Qihuang Zhong Liang Ding Juhua Liu Bo Du Dacheng Tao VLM CLL 34 41 0 22 Aug 2022
Few-Shot Class-Incremental Learning via Entropy-Regularized Data-Free Replay Huan Liu Li Gu Zhixiang Chi Yang Wang Yuanhao Yu Jun Chen Jingshan Tang 33 82 0 22 Jul 2022
Relation-guided acoustic scene classification aided with event embeddings Yuanbo Hou Bo Kang Wout Van Hauwermeiren Dick Botteldooren 24 16 0 01 May 2022
A Cross-Domain Approach for Continuous Impression Recognition from Dyadic Audio-Visual-Physio Signals Yuanchao Li Catherine Lai 19 1 0 25 Mar 2022
CT-SAT: Contextual Transformer for Sequential Audio Tagging Yuanbo Hou Zhaoyi Liu Bo Kang Yun Wang Dick Botteldooren ViT 32 5 0 22 Mar 2022
FINE Samples for Learning with Noisy Labels Taehyeon Kim Jongwoo Ko Sangwook Cho J. Choi Se-Young Yun NoLa 38 103 0 23 Feb 2021