TinyMIM: An Empirical Study of Distilling MIM Pre-trained Models

TinyMIM: An Empirical Study of Distilling MIM Pre-trained Models

3 January 2023

ArXiv (abs)PDF HTML Github (156★)

Papers citing "TinyMIM: An Empirical Study of Distilling MIM Pre-trained Models"

11 / 61 papers shown

Title
Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons Byeongho Heo Minsik Lee Sangdoo Yun J. Choi 55 526 0 08 Nov 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 94,891 0 11 Oct 2018
Unified Perceptual Parsing for Scene Understanding Tete Xiao Yingcheng Liu Bolei Zhou Yuning Jiang Jian Sun OCL VOS 192 1,887 0 26 Jul 2018
Self-supervised Knowledge Distillation Using Singular Value Decomposition Seunghyun Lee D. Kim B. Song 52 139 0 18 Jul 2018
Born Again Neural Networks Tommaso Furlanello Zachary Chase Lipton Michael Tschannen Laurent Itti Anima Anandkumar 68 1,031 0 12 May 2018
Paraphrasing Complex Network: Network Compression via Factor Transfer Jangho Kim Seonguk Park Nojun Kwak 77 550 0 14 Feb 2018
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 1.2K 20,858 0 17 Apr 2017
Semantic Understanding of Scenes through the ADE20K Dataset Bolei Zhou Hang Zhao Xavier Puig Tete Xiao Sanja Fidler Adela Barriuso Antonio Torralba SSeg 404 1,881 0 18 Aug 2016
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 362 19,660 0 09 Mar 2015
FitNets: Hints for Thin Deep Nets Adriana Romero Nicolas Ballas Samira Ebrahimi Kahou Antoine Chassang C. Gatta Yoshua Bengio FedML 308 3,887 0 19 Dec 2014
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 1.7K 39,547 0 01 Sep 2014