Vision Models Can Be Efficiently Specialized via Few-Shot Task-Aware Compression

25 March 2023

Denis Kuznedelev

Dan Alistarh

Papers citing "Vision Models Can Be Efficiently Specialized via Few-Shot Task-Aware Compression"

30 / 30 papers shown

Title
CAP: Correlation-Aware Pruning for Highly-Accurate Sparse Vision Models Denis Kuznedelev Eldar Kurtic Elias Frantar Dan Alistarh VLM ViT 55 13 0 14 Oct 2022
Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning Elias Frantar Sidak Pal Singh Dan Alistarh MQ 106 243 0 24 Aug 2022
BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizers Zhiliang Peng Li Dong Hangbo Bao QiXiang Ye Furu Wei 66 319 0 12 Aug 2022
A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities Yisheng Song Ting-Yuan Wang S. Mondal J. P. Sahoo SLR 118 376 0 13 May 2022
DeiT III: Revenge of the ViT Hugo Touvron Matthieu Cord Hervé Jégou ViT 127 414 0 14 Apr 2022
The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models Eldar Kurtic Daniel Fernando Campos Tuan Nguyen Elias Frantar Mark Kurtz Ben Fineran Michael Goin Dan Alistarh VLM MQ MedIm 92 126 0 14 Mar 2022
SPDY: Accurate Pruning with Speedup Guarantees Elias Frantar Dan Alistarh 58 35 0 31 Jan 2022
A ConvNet for the 2020s Zhuang Liu Hanzi Mao Chaozheng Wu Christoph Feichtenhofer Trevor Darrell Saining Xie ViT 178 5,213 0 10 Jan 2022
How Well Do Sparse Imagenet Models Transfer? Eugenia Iofinova Alexandra Peste Mark Kurtz Dan Alistarh 71 41 0 26 Nov 2021
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 475 7,819 0 11 Nov 2021
LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs Christoph Schuhmann Richard Vencu Romain Beaumont R. Kaczmarczyk Clayton Mullis Aarush Katta Theo Coombes J. Jitsev Aran Komatsuzaki VLM MLLM CLIP 243 1,442 0 03 Nov 2021
How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers Andreas Steiner Alexander Kolesnikov Xiaohua Zhai Ross Wightman Jakob Uszkoreit Lucas Beyer ViT 116 635 0 18 Jun 2021
Chasing Sparsity in Vision Transformers: An End-to-End Exploration Tianlong Chen Yu Cheng Zhe Gan Lu Yuan Lei Zhang Zhangyang Wang ViT 68 222 0 08 Jun 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 711 6,127 0 29 Apr 2021
Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks Itay Hubara Brian Chmiel Moshe Island Ron Banner S. Naor Daniel Soudry 109 119 0 16 Feb 2021
BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction Yuhang Li Ruihao Gong Xu Tan Yang Yang Peng Hu Qi Zhang F. Yu Wei Wang Shi Gu MQ 138 442 0 10 Feb 2021
Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks Torsten Hoefler Dan Alistarh Tal Ben-Nun Nikoli Dryden Alexandra Peste MQ 314 724 0 31 Jan 2021
The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models Tianlong Chen Jonathan Frankle Shiyu Chang Sijia Liu Yang Zhang Michael Carbin Zhangyang Wang 66 123 0 12 Dec 2020
Up or Down? Adaptive Rounding for Post-Training Quantization Markus Nagel Rana Ali Amjad M. V. Baalen Christos Louizos Tijmen Blankevoort MQ 90 586 0 22 Apr 2020
A Simple Framework for Contrastive Learning of Visual Representations Ting-Li Chen Simon Kornblith Mohammad Norouzi Geoffrey E. Hinton SSL 378 18,866 0 13 Feb 2020
Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation Linfeng Zhang Jiebo Song Anni Gao Jingwei Chen Chenglong Bao Kaisheng Ma FedML 76 865 0 17 May 2019
The State of Sparsity in Deep Neural Networks Trevor Gale Erich Elsen Sara Hooker 161 762 0 25 Feb 2019
Recognition in Terra Incognita Sara Beery Grant Van Horn Pietro Perona 97 852 0 13 Jul 2018
Do Better ImageNet Models Transfer Better? Simon Kornblith Jonathon Shlens Quoc V. Le OOD MLT 170 1,329 0 23 May 2018
Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon Xin Luna Dong Shangyu Chen Sinno Jialin Pan 183 506 0 22 May 2017
Rethinking the Inception Architecture for Computer Vision Christian Szegedy Vincent Vanhoucke Sergey Ioffe Jonathon Shlens Z. Wojna 3DV BDL 886 27,416 0 02 Dec 2015
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding Song Han Huizi Mao W. Dally 3DGS 263 8,859 0 01 Oct 2015
How transferable are features in deep neural networks? J. Yosinski Jeff Clune Yoshua Bengio Hod Lipson OOD 234 8,348 0 06 Nov 2014
Going Deeper with Convolutions Christian Szegedy Wei Liu Yangqing Jia P. Sermanet Scott E. Reed Dragomir Anguelov D. Erhan Vincent Vanhoucke Andrew Rabinovich 485 43,694 0 17 Sep 2014
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 1.7K 39,595 0 01 Sep 2014