TT-MPD: Test Time Model Pruning and Distillation

TT-MPD: Test Time Model Pruning and Distillation

10 December 2024

T. Malepathirana

Sachith Seneviratne

Saman K. Halgamuge

ArXiv (abs)PDF HTML

Papers citing "TT-MPD: Test Time Model Pruning and Distillation"

18 / 18 papers shown

Title
HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers Chen Liang Haoming Jiang Zheng Li Xianfeng Tang Bin Yin Tuo Zhao VLM 115 25 0 19 Feb 2023
Efficient Test-Time Model Adaptation without Forgetting Shuaicheng Niu Jiaxiang Wu Yifan Zhang Yaofo Chen S. Zheng P. Zhao Mingkui Tan OOD VLM TTA 91 350 0 06 Apr 2022
Compressing Models with Few Samples: Mimicking then Replacing Huanyu Wang Junjie Liu Xin Ma Yang Yong Z. Chai Jianxin Wu VLM OffRL 53 13 0 07 Jan 2022
Group Fisher Pruning for Practical Network Compression Liyang Liu Shilong Zhang Zhanghui Kuang Aojun Zhou Jingliang Xue Xinjiang Wang Yimin Chen Wenming Yang Q. Liao Wayne Zhang 88 149 0 02 Aug 2021
Knowledge distillation: A good teacher is patient and consistent Lucas Beyer Xiaohua Zhai Amelie Royer L. Markeeva Rohan Anil Alexander Kolesnikov VLM 109 299 0 09 Jun 2021
Neuron Merging: Compensating for Pruned Neurons Woojeong Kim Suhyun Kim Mincheol Park Geonseok Jeon 70 32 0 25 Oct 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 684 41,563 0 22 Oct 2020
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization Dan Hendrycks Steven Basart Norman Mu Saurav Kadavath Frank Wang ... Samyak Parajuli Mike Guo Basel Alomair Jacob Steinhardt Justin Gilmer OOD 363 1,757 0 29 Jun 2020
DynaBERT: Dynamic BERT with Adaptive Width and Depth Lu Hou Zhiqi Huang Lifeng Shang Xin Jiang Xiao Chen Qun Liu MQ 84 322 0 08 Apr 2020
Neural Network Pruning with Residual-Connections and Limited-Data Jian-Hao Luo Jianxin Wu 68 114 0 19 Nov 2019
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter Victor Sanh Lysandre Debut Julien Chaumond Thomas Wolf 267 7,554 0 02 Oct 2019
Benchmarking Neural Network Robustness to Common Corruptions and Perturbations Dan Hendrycks Thomas G. Dietterich OOD VLM 196 3,458 0 28 Mar 2019
Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration Yang He Ping Liu Ziwei Wang Zhilan Hu Yi Yang AAML 3DPC 98 1,050 0 01 Nov 2018
MobileNetV2: Inverted Residuals and Linear Bottlenecks Mark Sandler Andrew G. Howard Menglong Zhu A. Zhmoginov Liang-Chieh Chen 220 19,353 0 13 Jan 2018
Residual Connections Encourage Iterative Inference Stanislaw Jastrzebski Devansh Arpit Nicolas Ballas Vikas Verma Tong Che Yoshua Bengio 74 155 0 13 Oct 2017
ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression Jian-Hao Luo Jianxin Wu Weiyao Lin 58 1,761 0 20 Jul 2017
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.3K 194,641 0 10 Dec 2015
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding Song Han Huizi Mao W. Dally 3DGS 263 8,864 0 01 Oct 2015