Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer

12 December 2016

Papers citing "Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer"

50 / 1,157 papers shown

Title
Group channel pruning and spatial attention distilling for object detection Yun Chu Pu Li Yong Bai Zhuhua Hu Yongqing Chen Jiafeng Lu VLM 24 13 0 02 Jun 2023
Symmetric Uncertainty-Aware Feature Transmission for Depth Super-Resolution Wuxuan Shi Mang Ye Bo Du 25 17 0 01 Jun 2023
Are Large Kernels Better Teachers than Transformers for ConvNets? Tianjin Huang Lu Yin Zhenyu Zhang Lijuan Shen Meng Fang Mykola Pechenizkiy Zhangyang Wang Shiwei Liu 38 13 0 30 May 2023
Budget-Aware Graph Convolutional Network Design using Probabilistic Magnitude Pruning H. Sahbi 23 0 0 30 May 2023
Improving Knowledge Distillation via Regularizing Feature Norm and Direction Yuzhu Wang Lechao Cheng Manni Duan Yongheng Wang Zunlei Feng Shu Kong 47 20 0 26 May 2023
Triplet Knowledge Distillation Xijun Wang Dongyang Liu Meina Kan Chunrui Han Zhongqin Wu Shiguang Shan 42 3 0 25 May 2023
VanillaKD: Revisit the Power of Vanilla Knowledge Distillation from Small Scale to Large Scale Zhiwei Hao Jianyuan Guo Kai Han Han Hu Chang Xu Yunhe Wang 38 16 0 25 May 2023
On the Impact of Knowledge Distillation for Model Interpretability Hyeongrok Han Siwon Kim Hyun-Soo Choi Sungroh Yoon 29 4 0 25 May 2023
HARD: Hard Augmentations for Robust Distillation Arne F. Nix Max F. Burg Fabian H. Sinz AAML 44 1 0 24 May 2023
AdvFunMatch: When Consistent Teaching Meets Adversarial Robustness Ziuhi Wu Haichang Gao Bingqian Zhou Ping Wang AAML 24 0 0 24 May 2023
Decoupled Kullback-Leibler Divergence Loss Jiequan Cui Zhuotao Tian Zhisheng Zhong Xiaojuan Qi Bei Yu Hanwang Zhang 39 38 0 23 May 2023
NORM: Knowledge Distillation via N-to-One Representation Matching Xiaolong Liu Lujun Li Chao Li Anbang Yao 55 68 0 23 May 2023
Revisiting Data Augmentation in Model Compression: An Empirical and Comprehensive Study Muzhou Yu Linfeng Zhang Kaisheng Ma 28 2 0 22 May 2023
Distilling ChatGPT for Explainable Automated Student Answer Assessment Jiazheng Li Lin Gui Yuxiang Zhou David West Cesare Aloisi Yulan He 28 28 0 22 May 2023
Is Synthetic Data From Diffusion Models Ready for Knowledge Distillation? Zheng Li Yuxuan Li Penghai Zhao Renjie Song Xiang Li Jian Yang 34 19 0 22 May 2023
Cross-modality Data Augmentation for End-to-End Sign Language Translation Jinhui Ye Wenxiang Jiao Xing Wang Zhaopeng Tu Hui Xiong SLR 21 21 0 18 May 2023
Student-friendly Knowledge Distillation Mengyang Yuan Bo Lang Fengnan Quan 20 17 0 18 May 2023
Cross-Modality Time-Variant Relation Learning for Generating Dynamic Scene Graphs Jingyi Wang Jinfa Huang Can Zhang Zhidong Deng 49 7 0 15 May 2023
On enhancing the robustness of Vision Transformers: Defensive Diffusion Raza Imam Muhammad Huzaifa Mohammed El-Amine Azz MedIm DiffM 46 5 0 14 May 2023
DynamicKD: An Effective Knowledge Distillation via Dynamic Entropy Correction-Based Distillation for Gap Optimizing Songling Zhu Ronghua Shang Bo Yuan Weitong Zhang Yangyang Li Licheng Jiao 35 7 0 09 May 2023
Leveraging Synthetic Targets for Machine Translation Sarthak Mittal Oleksii Hrinchuk Oleksii Kuchaiev 35 2 0 07 May 2023
Structural and Statistical Texture Knowledge Distillation for Semantic Segmentation Deyi Ji Haoran Wang Mingyuan Tao Jianqiang Huang Xiansheng Hua Hongtao Lu 35 61 0 06 May 2023
Avatar Knowledge Distillation: Self-ensemble Teacher Paradigm with Uncertainty Yuan Zhang Weihua Chen Yichen Lu Tao Huang Xiuyu Sun Jian Cao 52 8 0 04 May 2023
Learngene: Inheriting Condensed Knowledge from the Ancestry Model to Descendant Models Qiufeng Wang Xu Yang Shuxia Lin Jing Wang Xin Geng 43 10 0 03 May 2023
CORSD: Class-Oriented Relational Self Distillation Muzhou Yu S. Tan Kailu Wu Runpei Dong Linfeng Zhang Kaisheng Ma 24 0 0 28 Apr 2023
Class Attention Transfer Based Knowledge Distillation Ziyao Guo Haonan Yan Hui Li Xiao-La Lin 18 64 0 25 Apr 2023
Bayesian Optimization Meets Self-Distillation HyunJae Lee Heon Song Hyeonsoo Lee Gi-hyeon Lee Suyeong Park Donggeun Yoo UQCV BDL 41 1 0 25 Apr 2023
Improving Knowledge Distillation via Transferring Learning Ability Long Liu Tong Li Hui Cheng 13 1 0 24 Apr 2023
Function-Consistent Feature Distillation Dongyang Liu Meina Kan Shiguang Shan Xilin Chen 52 18 0 24 Apr 2023
Knowledge Distillation Under Ideal Joint Classifier Assumption Huayu Li Xiwen Chen G. Ditzler Janet Roveda Ao Li 18 1 0 19 Apr 2023
Deep Collective Knowledge Distillation Jihyeon Seo Kyusam Oh Chanho Min Yongkeun Yun Sungwoo Cho 19 0 0 18 Apr 2023
MMANet: Margin-aware Distillation and Modality-aware Regularization for Incomplete Multimodal Learning Shicai Wei Yang Luo Chunbo Luo 26 29 0 17 Apr 2023
LaSNN: Layer-wise ANN-to-SNN Distillation for Effective and Efficient Training in Deep Spiking Neural Networks Di Hong Jiangrong Shen Yu Qi Yueming Wang 25 5 0 17 Apr 2023
SFT-KD-Recon: Learning a Student-friendly Teacher for Knowledge Distillation in Magnetic Resonance Image Reconstruction NagaGayathri Matcha Sriprabha Ramanarayanan Mohammad Al Fahim S. RahulG Keerthi Ram M. Sivaprakasam 21 2 0 11 Apr 2023
Towards Efficient Task-Driven Model Reprogramming with Foundation Models Shoukai Xu Jiangchao Yao Ran Luo Shuhai Zhang Zihao Lian Mingkui Tan Bo Han Yaowei Wang 29 6 0 05 Apr 2023
Cross-Class Feature Augmentation for Class Incremental Learning Taehoon Kim Jaeyoo Park Bohyung Han CLL 32 5 0 04 Apr 2023
Label-guided Attention Distillation for Lane Segmentation Zhikang Liu Lanyun Zhu 24 15 0 04 Apr 2023
Long-Tailed Visual Recognition via Self-Heterogeneous Integration with Knowledge Excavation Yang Jin Mengke Li Yang Lu Y. Cheung Hanzi Wang 43 21 0 03 Apr 2023
DIME-FM: DIstilling Multimodal and Efficient Foundation Models Ximeng Sun Pengchuan Zhang Peizhao Zhang Hardik Shah Kate Saenko Xide Xia VLM 30 20 0 31 Mar 2023
CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society Ge Li Hasan Hammoud Hani Itani Dmitrii Khizbullin Guohao Li SyDa ALM 52 413 0 31 Mar 2023
Decomposed Cross-modal Distillation for RGB-based Temporal Action Detection Pilhyeon Lee Taeoh Kim Minho Shim Dongyoon Wee H. Byun 41 11 0 30 Mar 2023
Information-Theoretic GAN Compression with Variational Energy-based Model Minsoo Kang Hyewon Yoo Eunhee Kang Sehwan Ki Hyong-Euk Lee Bohyung Han GAN 31 3 0 28 Mar 2023
DisWOT: Student Architecture Search for Distillation WithOut Training Peijie Dong Lujun Li Zimian Wei 46 57 0 28 Mar 2023
Hard Sample Matters a Lot in Zero-Shot Quantization Huantong Li Xiangmiao Wu Fanbing Lv Daihai Liao Thomas H. Li Yonggang Zhang Bo Han Mingkui Tan MQ 24 20 0 24 Mar 2023
Exploiting Unlabelled Photos for Stronger Fine-Grained SBIR Aneeshan Sain A. Bhunia Subhadeep Koley Pinaki Nath Chowdhury Soumitri Chattopadhyay Tao Xiang Yi-Zhe Song 30 18 0 24 Mar 2023
A Simple and Generic Framework for Feature Distillation via Channel-wise Transformation Ziwei Liu Yongtao Wang Xiaojie Chu 32 5 0 23 Mar 2023
From Knowledge Distillation to Self-Knowledge Distillation: A Unified Approach with Normalized Loss and Customized Soft Labels Zhendong Yang Ailing Zeng Zhe Li Tianke Zhang Chun Yuan Yu Li 31 74 0 23 Mar 2023
Understanding the Role of the Projector in Knowledge Distillation Roy Miles K. Mikolajczyk 35 21 0 20 Mar 2023
Channel-Aware Distillation Transformer for Depth Estimation on Nano Drones Ning Zhang F. Nex G. Vosselman N. Kerle 34 1 0 18 Mar 2023
Neural Architecture Search for Effective Teacher-Student Knowledge Transfer in Language Models Aashka Trivedi Takuma Udagawa Michele Merler Yikang Shen Yousef El-Kurdi Bishwaranjan Bhattacharjee 40 7 0 16 Mar 2023