Unified Visual Transformer Compression

15 March 2022

Papers citing "Unified Visual Transformer Compression"

27 / 27 papers shown

Title
MAS-Attention: Memory-Aware Stream Processing for Attention Acceleration on Resource-Constrained Edge Devices Mohammadali Shakerdargah Shan Lu Chao Gao Di Niu 70 0 0 20 Nov 2024
OATS: Outlier-Aware Pruning Through Sparse and Low Rank Decomposition Stephen Zhang V. Papyan VLM 51 1 0 20 Sep 2024
Compress and Compare: Interactively Evaluating Efficiency and Behavior Across ML Model Compression Experiments Angie Boggust Venkatesh Sivaraman Yannick Assogba Donghao Ren Dominik Moritz Fred Hohman VLM 55 3 0 06 Aug 2024
Isomorphic Pruning for Vision Models Gongfan Fang Xinyin Ma Michael Bi Mi Xinchao Wang VLM ViT 36 6 0 05 Jul 2024
LPViT: Low-Power Semi-structured Pruning for Vision Transformers Kaixin Xu Zhe Wang Chunyun Chen Xue Geng Jie Lin Xulei Yang Min-man Wu Min Wu Xiaoli Li Weisi Lin ViT VLM 51 7 0 02 Jul 2024
Torch2Chip: An End-to-end Customizable Deep Neural Network Compression and Deployment Toolkit for Prototype Hardware Accelerator Design Jian Meng Yuan Liao Anupreetham Anupreetham Ahmed Hassan Shixing Yu Han-Sok Suh Xiaofeng Hu Jae-sun Seo MQ 49 1 0 02 May 2024
Improved TokenPose with Sparsity Anning Li ViT 34 0 0 16 Nov 2023
CAIT: Triple-Win Compression towards High Accuracy, Fast Inference, and Favorable Transferability For ViTs Ao Wang Hui Chen Zijia Lin Sicheng Zhao J. Han Guiguang Ding ViT 31 6 0 27 Sep 2023
Resource Constrained Model Compression via Minimax Optimization for Spiking Neural Networks Jue Chen Huan Yuan Jianchao Tan Bin Chen Chengru Song Di Zhang 25 3 0 09 Aug 2023
COMCAT: Towards Efficient Compression and Customization of Attention-Based Vision Models Jinqi Xiao Miao Yin Yu Gong Xiao Zang Jian Ren Bo Yuan VLM ViT 40 9 0 26 May 2023
Boost Vision Transformer with GPU-Friendly Sparsity and Quantization Chong Yu Tao Chen Zhongxue Gan Jiayuan Fan MQ ViT 30 23 0 18 May 2023
Distilling Token-Pruned Pose Transformer for 2D Human Pose Estimation Feixiang Ren ViT 19 2 0 12 Apr 2023
GOHSP: A Unified Framework of Graph and Optimization-based Heterogeneous Structured Pruning for Vision Transformer Miao Yin Burak Uzkent Yilin Shen Hongxia Jin Bo Yuan ViT 26 13 0 13 Jan 2023
Peeling the Onion: Hierarchical Reduction of Data Redundancy for Efficient Vision Transformer Training Zhenglun Kong Haoyu Ma Geng Yuan Mengshu Sun Yanyue Xie ... Tianlong Chen Xiaolong Ma Xiaohui Xie Zhangyang Wang Yanzhi Wang ViT 31 22 0 19 Nov 2022
HeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformers Peiyan Dong Mengshu Sun Alec Lu Yanyue Xie Li-Yu Daisy Liu ... Xin Meng ZeLin Li Xue Lin Zhenman Fang Yanzhi Wang ViT 31 58 0 15 Nov 2022
ViTALiTy: Unifying Low-rank and Sparse Approximation for Vision Transformer Acceleration with a Linear Taylor Attention Jyotikrishna Dass Shang Wu Huihong Shi Chaojian Li Zhifan Ye Zhongfeng Wang Yingyan Lin 17 49 0 09 Nov 2022
PPT: token-Pruned Pose Transformer for monocular and multi-view human pose estimation Haoyu Ma Zhe Wang Yifei Chen Deying Kong Liangjian Chen Xingwei Liu Xiangyi Yan Hao Tang Xiaohui Xie ViT 35 47 0 16 Sep 2022
MiniViT: Compressing Vision Transformers with Weight Multiplexing Jinnian Zhang Houwen Peng Kan Wu Mengchen Liu Bin Xiao Jianlong Fu Lu Yuan ViT 28 123 0 14 Apr 2022
LightHuBERT: Lightweight and Configurable Speech Representation Learning with Once-for-All Hidden-Unit BERT Rui Wang Qibing Bai Junyi Ao Long Zhou Zhixiang Xiong Zhihua Wei Yu Zhang Tom Ko Haizhou Li 34 61 0 29 Mar 2022
Multi-Dimensional Model Compression of Vision Transformer Zejiang Hou S. Kung ViT 25 16 0 31 Dec 2021
Pruning Self-attentions into Convolutional Layers in Single Path Haoyu He Jianfei Cai Jing Liu Zizheng Pan Jing Zhang Dacheng Tao Bohan Zhuang ViT 34 40 0 23 Nov 2021
Is Space-Time Attention All You Need for Video Understanding? Gedas Bertasius Heng Wang Lorenzo Torresani ViT 280 1,982 0 09 Feb 2021
Hessian-Aware Pruning and Optimal Neural Implant Shixing Yu Z. Yao A. Gholami Zhen Dong Sehoon Kim Michael W. Mahoney Kurt Keutzer 54 59 0 22 Jan 2021
BinaryBERT: Pushing the Limit of BERT Quantization Haoli Bai Wei Zhang Lu Hou Lifeng Shang Jing Jin Xin Jiang Qun Liu Michael Lyu Irwin King MQ 142 221 0 31 Dec 2020
SCOP: Scientific Control for Reliable Neural Network Pruning Yehui Tang Yunhe Wang Yixing Xu Dacheng Tao Chunjing Xu Chao Xu Chang Xu AAML 50 166 0 21 Oct 2020
The Lottery Ticket Hypothesis for Pre-trained BERT Networks Tianlong Chen Jonathan Frankle Shiyu Chang Sijia Liu Yang Zhang Zhangyang Wang Michael Carbin 156 345 0 23 Jul 2020
BERT-of-Theseus: Compressing BERT by Progressive Module Replacing Canwen Xu Wangchunshu Zhou Tao Ge Furu Wei Ming Zhou 221 197 0 07 Feb 2020