Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer

12 December 2016

Papers citing "Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer"

50 / 1,157 papers shown

Title
Fast End-to-End Speech Recognition via Non-Autoregressive Models and Cross-Modal Knowledge Transferring from BERT Ye Bai Jiangyan Yi J. Tao Zhengkun Tian Zhengqi Wen Shuai Zhang RALM 33 51 0 15 Feb 2021
Self Regulated Learning Mechanism for Data Efficient Knowledge Distillation Sourav Mishra Suresh Sundaram 20 1 0 14 Feb 2021
Semantically-Conditioned Negative Samples for Efficient Contrastive Learning J. Ó. Neill Danushka Bollegala 39 6 0 12 Feb 2021
Learning Student-Friendly Teacher Networks for Knowledge Distillation D. Park Moonsu Cha C. Jeong Daesin Kim Bohyung Han 121 101 0 12 Feb 2021
Show, Attend and Distill:Knowledge Distillation via Attention-based Feature Matching Mingi Ji Byeongho Heo Sungrae Park 65 145 0 05 Feb 2021
Truly Sparse Neural Networks at Scale Selima Curci Decebal Constantin Mocanu Mykola Pechenizkiy 45 19 0 02 Feb 2021
Rethinking Soft Labels for Knowledge Distillation: A Bias-Variance Tradeoff Perspective Helong Zhou Liangchen Song Jiajie Chen Ye Zhou Guoli Wang Junsong Yuan Qian Zhang 30 170 0 01 Feb 2021
Subjective and Objective Quality Assessment of Mobile Gaming Video Shaoguo Wen Suiyi Ling Junle Wang Ximing Chen Lizhi Fang Yanqing Jing P. Le Callet 6 16 0 27 Jan 2021
TCLR: Temporal Contrastive Learning for Video Representation I. Dave Rohit Gupta Mamshad Nayeem Rizve Mubarak Shah SSL AI4TS 36 175 0 20 Jan 2021
Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks Yige Li Lingjuan Lyu Nodens Koren X. Lyu Bo Li Xingjun Ma AAML FedML 30 429 0 15 Jan 2021
Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labels Sangdoo Yun Seong Joon Oh Byeongho Heo Dongyoon Han Junsuk Choe Sanghyuk Chun 414 143 0 13 Jan 2021
SEED: Self-supervised Distillation For Visual Representation Zhiyuan Fang Jianfeng Wang Lijuan Wang Lei Zhang Yezhou Yang Zicheng Liu SSL 247 190 0 12 Jan 2021
Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed Eric Luhman Troy Luhman DiffM 195 262 0 07 Jan 2021
MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers Wenhui Wang Hangbo Bao Shaohan Huang Li Dong Furu Wei MQ 30 257 0 31 Dec 2020
Binary Graph Neural Networks Mehdi Bahri Gaétan Bahl S. Zafeiriou GNN AI4CE 11 49 0 31 Dec 2020
FREA-Unet: Frequency-aware U-net for Modality Transfer H. Emami Qiong Liu Ming Dong MedIm 18 6 0 31 Dec 2020
Towards a Universal Continuous Knowledge Base Gang Chen Maosong Sun Yang Liu 28 3 0 25 Dec 2020
AttentionLite: Towards Efficient Self-Attention Models for Vision Souvik Kundu Sairam Sundaresan 24 22 0 21 Dec 2020
Knowledge Transfer Based Fine-grained Visual Classification Siqing Zhang Ruoyi Du Dongliang Chang Zhanyu Ma Jun Guo 28 3 0 21 Dec 2020
Computation-Efficient Knowledge Distillation via Uncertainty-Aware Mixup Guodong Xu Ziwei Liu Chen Change Loy UQCV 23 39 0 17 Dec 2020
ISD: Self-Supervised Learning by Iterative Similarity Distillation Ajinkya Tejankar Soroush Abbasi Koohpayegani Vipin Pillai Paolo Favaro Hamed Pirsiavash SSL 27 44 0 16 Dec 2020
Wasserstein Contrastive Representation Distillation Liqun Chen Dong Wang Zhe Gan Jingjing Liu Ricardo Henao Lawrence Carin 23 93 0 15 Dec 2020
CHS-Net: A Deep learning approach for hierarchical segmentation of COVID-19 infected CT images Narinder Singh Punn Sonali Agarwal 37 31 0 13 Dec 2020
Periocular Embedding Learning with Consistent Knowledge Distillation from Face Yoon Gyo Jung Jaewoo Park C. Low Jacky Chen Long Chai Leslie Ching Ow Tiong Andrew Beng Jin Teoh CVBM 21 2 0 12 Dec 2020
ADD: Augmented Disentanglement Distillation Framework for Improving Stock Trend Forecasting H. Tang Lijun Wu Weiqing Liu Jiang Bian AIFin 19 4 0 11 Dec 2020
Learning Tubule-Sensitive CNNs for Pulmonary Airway and Artery-Vein Segmentation in CT Yulei Qin Hao Zheng Yun Gu Xiaolin Huang Jie Yang Lihui Wang Feng Yao Y. Zhu Guang-Zhong Yang 27 75 0 10 Dec 2020
Debiased-CAM to mitigate image perturbations with faithful visual explanations of machine learning Wencan Zhang Mariella Dimiccoli Brian Y. Lim FAtt 34 18 0 10 Dec 2020
Model Compression Using Optimal Transport Suhas Lohit Michael J. Jones 28 8 0 07 Dec 2020
MERANet: Facial Micro-Expression Recognition using 3D Residual Attention Network G. V. Reddy S. Reddy Snehasis Mukherjee S. Dubey 3DH CVBM 19 13 0 07 Dec 2020
Cross-Layer Distillation with Semantic Calibration Defang Chen Jian-Ping Mei Yuan Zhang Can Wang Yan Feng Chun-Yen Chen FedML 45 288 0 06 Dec 2020
Multi-head Knowledge Distillation for Model Compression Haiquan Wang Suhas Lohit Michael J. Jones Y. Fu 19 5 0 05 Dec 2020
Going Beyond Classification Accuracy Metrics in Model Compression Vinu Joseph Shoaib Ahmed Siddiqui Aditya Bhaskara Ganesh Gopalakrishnan Saurav Muralidharan M. Garland Sheraz Ahmed Andreas Dengel 45 17 0 03 Dec 2020
Visually Imperceptible Adversarial Patch Attacks on Digital Images Yaguan Qian Jiamin Wang Bin Wang Xiang Ling Zhaoquan Gu Chunming Wu Wassim Swaileh AAML 44 2 0 02 Dec 2020
Multi-level Knowledge Distillation via Knowledge Alignment and Correlation Fei Ding Yin Yang Hongxin Hu Venkat Krovi Feng Luo 22 4 0 01 Dec 2020
Data-Free Model Extraction Jean-Baptiste Truong Pratyush Maini R. Walls Nicolas Papernot MIACV 15 181 0 30 Nov 2020
How Well Do Self-Supervised Models Transfer? Linus Ericsson Henry Gouk Timothy M. Hospedales SSL 35 274 0 26 Nov 2020
Channel-wise Knowledge Distillation for Dense Prediction Changyong Shu Yifan Liu Jianfei Gao Zheng Yan Chunhua Shen 27 257 0 26 Nov 2020
torchdistill: A Modular, Configuration-Driven Framework for Knowledge Distillation Yoshitomo Matsubara 11 25 0 25 Nov 2020
Bringing AI To Edge: From Deep Learning's Perspective Di Liu Hao Kong Xiangzhong Luo Weichen Liu Ravi Subramaniam 52 116 0 25 Nov 2020
PLOP: Learning without Forgetting for Continual Semantic Segmentation Arthur Douillard Yifu Chen Arnaud Dapogny Matthieu Cord CLL 19 234 0 23 Nov 2020
Backdoor Attacks on the DNN Interpretation System Shihong Fang A. Choromańska FAtt AAML 29 19 0 21 Nov 2020
3D attention mechanism for fine-grained classification of table tennis strokes using a Twin Spatio-Temporal Convolutional Neural Networks Pierre-Etienne Martin J. Benois-Pineau Renaud Péteri J. Morlier 3DPC 24 12 0 20 Nov 2020
Digging Deeper into CRNN Model in Chinese Text Images Recognition Kunhong Yu Yuze Zhang 14 1 0 17 Nov 2020
Learning Efficient GANs for Image Translation via Differentiable Masks and co-Attention Distillation Shaojie Li Mingbao Lin Yan Wang Rongrong Ji Ling Shao Rongrong Ji 20 32 0 17 Nov 2020
Filter Pre-Pruning for Improved Fine-tuning of Quantized Deep Neural Networks Jun Nishikawa Ryoji Ikegaya MQ 23 1 0 13 Nov 2020
PAMS: Quantized Super-Resolution via Parameterized Max Scale Huixia Li Chenqian Yan Shaohui Lin Xiawu Zheng Yuchao Li Baochang Zhang Fan Yang Rongrong Ji MQ 35 84 0 09 Nov 2020
Ensemble Knowledge Distillation for CTR Prediction Jieming Zhu Jinyang Liu Weiqi Li Jincai Lai Xiuqiang He Liang Chen Zibin Zheng 36 56 0 08 Nov 2020
Cross-Modal Self-Attention Distillation for Prostate Cancer Segmentation Guokai Zhang Xiaoang Shen Ye Luo Jihao Luo Zeju Wang Weigang Wang Binghui Zhao Jianwei Lu MedIm 18 4 0 08 Nov 2020
Robustness and Diversity Seeking Data-Free Knowledge Distillation Pengchao Han Jihong Park Shiqiang Wang Yejun Liu 15 12 0 07 Nov 2020
On Self-Distilling Graph Neural Network Y. Chen Yatao Bian Xi Xiao Yu Rong Tingyang Xu Junzhou Huang FedML 27 48 0 04 Nov 2020