MUSE: Feature Self-Distillation with Mutual Information and Self-Information

25 October 2021

Papers citing "MUSE: Feature Self-Distillation with Mutual Information and Self-Information"

31 / 31 papers shown

Title
Generative Adversarial Networks Gilad Cohen Raja Giryes GAN 280 30,103 0 01 Mar 2022
Refine Myself by Teaching Myself: Feature Refinement via Self-Knowledge Distillation Mingi Ji Seungjae Shin Seunghyun Hwang Gibeom Park Il-Chul Moon 35 123 0 15 Mar 2021
Regularizing Class-wise Predictions via Self-knowledge Distillation Sukmin Yun Jongjin Park Kimin Lee Jinwoo Shin 62 279 0 31 Mar 2020
Feature-map-level Online Adversarial Knowledge Distillation Inseop Chung Seonguk Park Jangho Kim Nojun Kwak GAN 81 130 0 05 Feb 2020
Contrastive Representation Distillation Yonglong Tian Dilip Krishnan Phillip Isola 151 1,049 0 23 Oct 2019
Learning Lightweight Lane Detection CNNs by Self Attention Distillation Yuenan Hou Zheng Ma Chunxiao Liu Chen Change Loy 68 557 0 02 Aug 2019
On Mutual Information Maximization for Representation Learning Michael Tschannen Josip Djolonga Paul Kishan Rubenstein Sylvain Gelly Mario Lucic SSL 177 495 0 31 Jul 2019
Contrastive Multiview Coding Yonglong Tian Dilip Krishnan Phillip Isola SSL 169 2,403 0 13 Jun 2019
Learning Representations by Maximizing Mutual Information Across Views Philip Bachman R. Devon Hjelm William Buchwalter SSL 189 1,475 0 03 Jun 2019
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks Mingxing Tan Quoc V. Le 3DV MedIm 139 18,134 0 28 May 2019
Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation Linfeng Zhang Jiebo Song Anni Gao Jingwei Chen Chenglong Bao Kaisheng Ma FedML 73 861 0 17 May 2019
Variational Information Distillation for Knowledge Transfer SungSoo Ahn S. Hu Andreas C. Damianou Neil D. Lawrence Zhenwen Dai 89 620 0 11 Apr 2019
MEAL: Multi-Model Ensemble via Adversarial Learning Zhiqiang Shen Zhankui He Xiangyang Xue AAML FedML 65 146 0 06 Dec 2018
Snapshot Distillation: Teacher-Student Optimization in One Generation Chenglin Yang Lingxi Xie Chi Su Alan Yuille 75 193 0 01 Dec 2018
Learning deep representations by mutual information estimation and maximization R. Devon Hjelm A. Fedorov Samuel Lavoie-Marchildon Karan Grewal Phil Bachman Adam Trischler Yoshua Bengio SSL DRL 320 2,662 0 20 Aug 2018
Representation Learning with Contrastive Predictive Coding Aaron van den Oord Yazhe Li Oriol Vinyals DRL SSL 320 10,302 0 10 Jul 2018
Knowledge Distillation by On-the-Fly Native Ensemble Xu Lan Xiatian Zhu S. Gong 290 479 0 12 Jun 2018
MINE: Mutual Information Neural Estimation Mohamed Ishmael Belghazi A. Baratin Sai Rajeswar Sherjil Ozair Yoshua Bengio Aaron Courville R. Devon Hjelm DRL 191 1,279 0 12 Jan 2018
Learning Transferable Architectures for Scalable Image Recognition Barret Zoph Vijay Vasudevan Jonathon Shlens Quoc V. Le 177 5,603 0 21 Jul 2017
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices Xiangyu Zhang Xinyu Zhou Mengxiao Lin Jian Sun AI4TS 141 6,872 0 04 Jul 2017
Deep Mutual Learning Ying Zhang Tao Xiang Timothy M. Hospedales Huchuan Lu FedML 148 1,653 0 01 Jun 2017
Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer Sergey Zagoruyko N. Komodakis 118 2,581 0 12 Dec 2016
Densely Connected Convolutional Networks Gao Huang Zhuang Liu Laurens van der Maaten Kilian Q. Weinberger PINN 3DV 772 36,813 0 25 Aug 2016
Wide Residual Networks Sergey Zagoruyko N. Komodakis 340 7,985 0 23 May 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 194,020 0 10 Dec 2015
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 362 19,660 0 09 Mar 2015
Striving for Simplicity: The All Convolutional Net Jost Tobias Springenberg Alexey Dosovitskiy Thomas Brox Martin Riedmiller FAtt 248 4,672 0 21 Dec 2014
FitNets: Hints for Thin Deep Nets Adriana Romero Nicolas Ballas Samira Ebrahimi Kahou Antoine Chassang C. Gatta Yoshua Bengio FedML 308 3,887 0 19 Dec 2014
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 1.6K 100,386 0 04 Sep 2014
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 413 43,667 0 01 May 2014
Equitability, mutual information, and the maximal information coefficient J. Kinney G. Atwal 90 608 0 31 Jan 2013