Prune Your Model Before Distill It

30 September 2021

Papers citing "Prune Your Model Before Distill It"

14 / 14 papers shown

Title
Attention-guided Feature Distillation for Semantic Segmentation Amir M. Mansourian Arya Jalali Rozhan Ahmadi S. Kasaei 28 0 0 08 Mar 2024
EPSD: Early Pruning with Self-Distillation for Efficient Model Compression Dong Chen Ning Liu Yichen Zhu Zhengping Che Rui Ma Fachao Zhang Xiaofeng Mou Yi Chang Jian Tang 31 3 0 31 Jan 2024
RdimKD: Generic Distillation Paradigm by Dimensionality Reduction Yi Guo Yiqian He Xiaoyang Li Haotong Qin Van Tung Pham Yang Zhang Shouda Liu 43 1 0 14 Dec 2023
Fantastic Gains and Where to Find Them: On the Existence and Prospect of General Knowledge Transfer between Any Pretrained Model Karsten Roth Lukas Thede Almut Sophia Koepke Oriol Vinyals Olivier J. Hénaff Zeynep Akata AAML 22 11 0 26 Oct 2023
LAPP: Layer Adaptive Progressive Pruning for Compressing CNNs from Scratch P. Zhai K. Guo F. Liu Xiaofen Xing Xiangmin Xu 26 3 0 25 Sep 2023
The Quest of Finding the Antidote to Sparse Double Descent Victor Quétu Marta Milovanović 31 0 0 31 Aug 2023
DSD $^2$ : Can We Dodge Sparse Double Descent and Compress the Neural Network Worry-Free? Victor Quétu Enzo Tartaglione 29 7 0 02 Mar 2023
Progressive Learning without Forgetting Tao Feng Hangjie Yuan Mang Wang Ziyuan Huang Ang Bian Jianzhou Zhang CLL KELM 44 4 0 28 Nov 2022
Join the High Accuracy Club on ImageNet with A Binary Neural Network Ticket Nianhui Guo Joseph Bethge Christoph Meinel Haojin Yang MQ 31 19 0 23 Nov 2022
BCNet: Searching for Network Width with Bilaterally Coupled Network Xiu Su Shan You Fei-Yue Wang Chao Qian Changshui Zhang Chang Xu 63 34 0 21 May 2021
Carbon Emissions and Large Neural Network Training David A. Patterson Joseph E. Gonzalez Quoc V. Le Chen Liang Lluís-Miquel Munguía D. Rothchild David R. So Maud Texier J. Dean AI4CE 244 644 0 21 Apr 2021
Learning Student-Friendly Teacher Networks for Knowledge Distillation D. Park Moonsu Cha C. Jeong Daesin Kim Bohyung Han 118 100 0 12 Feb 2021
Distilling Knowledge from Graph Convolutional Networks Yiding Yang Jiayan Qiu Xiuming Zhang Dacheng Tao Xinchao Wang 160 226 0 23 Mar 2020
Comparing Rewinding and Fine-tuning in Neural Network Pruning Alex Renda Jonathan Frankle Michael Carbin 224 383 0 05 Mar 2020