Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer

12 December 2016

Papers citing "Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer"

50 / 1,156 papers shown

Title
Maximizing Discrimination Capability of Knowledge Distillation with Energy Function Seonghak Kim Gyeongdo Ham Suin Lee Donggon Jang Daeshik Kim 34 4 0 23 May 2025
Intra-class Patch Swap for Self-Distillation Hongjun Choi Eun Som Jeon Ankita Shukla Pavan Turaga 7 0 0 20 May 2025
FiGKD: Fine-Grained Knowledge Distillation via High-Frequency Detail Transfer Seonghak Kim 14 0 0 17 May 2025
JointDistill: Adaptive Multi-Task Distillation for Joint Depth Estimation and Scene Segmentation Tiancong Cheng Ying Zhang Yuxuan Liang Roger Zimmermann Zhiwen Yu Bin Guo VLM 24 0 0 15 May 2025
DCSNet: A Lightweight Knowledge Distillation-Based Model with Explainable AI for Lung Cancer Diagnosis from Histopathological Images Sadman Sakib Alif Nasim Anzum Promise Fiaz Al Abid Aniqua Nusrat Zereen 26 0 0 14 May 2025
ABKD: Pursuing a Proper Allocation of the Probability Mass in Knowledge Distillation via $α$ - $β$ -Divergence Guanghui Wang Zhiyong Yang Zihan Wang Shi Wang Qianqian Xu Qingming Huang 42 0 0 07 May 2025
Robust Understanding of Human-Robot Social Interactions through Multimodal Distillation Tongfei Bian Mathieu Chollet T. Guha 31 0 0 06 May 2025
Swapped Logit Distillation via Bi-level Teacher Alignment Stephen Ekaputra Limantoro Jhe-Hao Lin Chih-Yu Wang Yi-Lung Tsai Hong-Han Shuai Ching-Chun Huang Wen-Huang Cheng 54 0 0 27 Apr 2025
DNAD: Differentiable Neural Architecture Distillation Xuan Rao Bo Zhao Derong Liu 39 1 0 25 Apr 2025
Hybrid Knowledge Transfer through Attention and Logit Distillation for On-Device Vision Systems in Agricultural IoT Stanley Mugisha Rashid Kisitu Florence Tushabe 22 0 0 21 Apr 2025
Artifact detection and localization in single-channel mobile EEG for sleep research using deep learning and attention mechanisms Khrystyna Semkiv Jia Zhang Maria Laura Ferster W. Karlen 38 0 0 11 Apr 2025
S^4M: Boosting Semi-Supervised Instance Segmentation with SAM Heeji Yoon Heeseong Shin Eunbeen Hong Hyunwook Choi Hansang Cho Daun Jeong Seungryong Kim 33 0 0 07 Apr 2025
Random Conditioning with Distillation for Data-Efficient Diffusion Model Compression Dohyun Kim S. Park Geonhee Han Seung Wook Kim Paul Hongsuck Seo DiffM 58 0 0 02 Apr 2025
Sample-level Adaptive Knowledge Distillation for Action Recognition Ping Li Chenhao Ping Wenxiao Wang Mingli Song 49 0 0 01 Apr 2025
Penrose Tiled Low-Rank Compression and Section-Wise Q&A Fine-Tuning: A General Framework for Domain-Specific Large Language Model Adaptation Chuan-Wei Kuo Siyu Chen Chenqi Yan Yu Liu 63 0 0 28 Mar 2025
FedSKD: Aggregation-free Model-heterogeneous Federated Learning using Multi-dimensional Similarity Knowledge Distillation Ziqiao Weng Weidong (Tom) Cai Bo Zhou 51 0 0 23 Mar 2025
Semantic-Supervised Spatial-Temporal Fusion for LiDAR-based 3D Object Detection Chaoqun Wang Xiaobin Hong Wenzhong Li Ruimao Zhang 3DPC 204 0 0 13 Mar 2025
Adaptive Temperature Based on Logits Correlation in Knowledge Distillation Kazuhiro Matsuyama Usman Anjum Satoko Matsuyama Tetsuo Shoda J. Zhan 60 0 0 12 Mar 2025
BDPFL: Backdoor Defense for Personalized Federated Learning via Explainable Distillation Chengcheng Zhu J. Zhang Di Wu Guodong Long FedML AAML 48 0 0 09 Mar 2025
MIDAS: Modeling Ground-Truth Distributions with Dark Knowledge for Domain Generalized Stereo Matching Peng Xu Zhiyu Xiang Jingyun Fu Tianyu Pu Hanzhi Zhong Eryun Liu OOD 72 0 0 06 Mar 2025
VRM: Knowledge Distillation via Virtual Relation Matching W. Zhang Fei Xie Weidong Cai Chao Ma 76 0 0 28 Feb 2025
AfroXLMR-Comet: Multilingual Knowledge Distillation with Attention Matching for Low-Resource languages Joshua Sakthivel Raju Shri Kiran Srinivasan Jaskaran Singh Walia Srinivas Raghav Vukosi Marivate 69 0 0 25 Feb 2025
I2CKD : Intra- and Inter-Class Knowledge Distillation for Semantic Segmentation Ayoub Karine Thibault Napoléon M. Jridi VLM 109 0 0 24 Feb 2025
Multi-Level Decoupled Relational Distillation for Heterogeneous Architectures Yaoxin Yang Peng Ye Weihao Lin Kangcong Li Yan Wen Jia Hao Tao Chen 38 0 0 10 Feb 2025
Contrastive Representation Distillation via Multi-Scale Feature Decoupling Cuipeng Wang Tieyuan Chen Haipeng Wang 54 0 0 09 Feb 2025
Variational Bayesian Adaptive Learning of Deep Latent Variables for Acoustic Knowledge Transfer Hu Hu Sabato Marco Siniscalchi Chao-Han Huck Yang Chin-Hui Lee 85 0 0 28 Jan 2025
Rethinking Knowledge in Distillation: An In-context Sample Retrieval Perspective Jinjing Zhu Songze Li Lin Wang 54 0 0 13 Jan 2025
Normalizing Batch Normalization for Long-Tailed Recognition Yuxiang Bao Guoliang Kang Linlin Yang Xiaoyue Duan Bo Zhao Baochang Zhang MQ 57 0 0 06 Jan 2025
Knowledge Distillation with Adapted Weight Sirong Wu Xi Luo Junjie Liu Yuhui Deng 48 0 0 06 Jan 2025
Cross-View Consistency Regularisation for Knowledge Distillation W. Zhang Dongnan Liu Weidong Cai Chao Ma 75 1 0 21 Dec 2024
LiRCDepth: Lightweight Radar-Camera Depth Estimation via Knowledge Distillation and Uncertainty Guidance Huawei Sun Nastassia Vysotskaya Tobias Sukianto Hao Feng Julius Ott Xiangyuan Peng Lorenzo Servadei Robert Wille 79 0 0 20 Dec 2024
Knowledge Distillation in RNN-Attention Models for Early Prediction of Student Performance Sukrit Leelaluk Cheng Tang Valdemar Švábenský Atsushi Shimada 83 1 0 19 Dec 2024
Neural Collapse Inspired Knowledge Distillation Shuoxi Zhang Zijian Song Kun He 71 1 0 16 Dec 2024
Knowledge Migration Framework for Smart Contract Vulnerability Detection Luqi Wang Wenbao Jiang 89 0 0 15 Dec 2024
On Distilling the Displacement Knowledge for Few-Shot Class-Incremental Learning Pengfei Fang Yongchun Qin H. Xue CLL 84 0 0 15 Dec 2024
Multi-Surrogate-Teacher Assistance for Representation Alignment in Fingerprint-based Indoor Localization Son Minh Nguyen Linh Duy Tran Duc Viet Le Paul J. M Havinga 69 0 0 13 Dec 2024
Wasserstein Distance Rivals Kullback-Leibler Divergence for Knowledge Distillation Jiaming Lv Haoyuan Yang P. Li 81 1 0 11 Dec 2024
Distillation of Diffusion Features for Semantic Correspondence Frank Fundel Johannes Schusterbauer Vincent Tao Hu Bjorn Ommer DiffM 91 3 0 04 Dec 2024
Mutli-View 3D Reconstruction using Knowledge Distillation Aditya Dutt Ishikaa Lunawat Manpreet Kaur 64 0 0 02 Dec 2024
CLFace: A Scalable and Resource-Efficient Continual Learning Framework for Lifelong Face Recognition Md Golam Moula Mehedi Hasan S. Sami Nasser M. Nasrabadi CLL 73 0 0 21 Nov 2024
Exploring Feature-based Knowledge Distillation for Recommender System: A Frequency Perspective Zhangchi Zhu Wei Zhang 43 0 0 16 Nov 2024
On the Surprising Effectiveness of Attention Transfer for Vision Transformers Alexander C. Li Yuandong Tian Bin Chen Deepak Pathak Xinlei Chen 43 0 0 14 Nov 2024
Toward Robust Incomplete Multimodal Sentiment Analysis via Hierarchical Representation Learning Mingxing Li Dingkang Yang Y. Liu Shunli Wang Jiawei Chen ... Xiaolu Hou Mingyang Sun Ziyun Qian Dongliang Kou Li Zhang 37 1 0 05 Nov 2024
Decoupling Dark Knowledge via Block-wise Logit Distillation for Feature-level Alignment Chengting Yu Fengzhao Zhang Ruizhe Chen Zuozhu Liu Shurun Tan Er-ping Li Aili Wang 44 2 0 03 Nov 2024
Emotion Recognition with Facial Attention and Objective Activation Functions Andrzej Miskow Abdulrahman Altahhan CVBM 33 0 0 23 Oct 2024
Pre-training Distillation for Large Language Models: A Design Space Exploration Hao Peng Xin Lv Yushi Bai Zijun Yao J. Zhang Lei Hou Juanzi Li 36 4 0 21 Oct 2024
Preview-based Category Contrastive Learning for Knowledge Distillation Muhe Ding Jianlong Wu Xue Dong Xiaojie Li Pengda Qin Tian Gan Liqiang Nie VLM 39 0 0 18 Oct 2024
Swiss Army Knife: Synergizing Biases in Knowledge from Vision Foundation Models for Multi-Task Learning Yuxiang Lu Shengcao Cao Yu-xiong Wang 55 1 0 18 Oct 2024
D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement Yansong Peng Hebei Li Peixi Wu Yueyi Zhang Xingchen Sun Feng Wu 44 13 0 17 Oct 2024
HASN: Hybrid Attention Separable Network for Efficient Image Super-resolution Weifeng Cao Xiaoyan Lei Jun Shi Wanyong Liang Jie Liu Zongfei Bai SupR 34 0 0 13 Oct 2024