Pruning Self-attentions into Convolutional Layers in Single Path

23 November 2021

Haoyu He

Jianfei Cai

Jing Liu

Bohan Zhuang

Papers citing "Pruning Self-attentions into Convolutional Layers in Single Path"

24 / 24 papers shown

Title
MDP: Multidimensional Vision Model Pruning with Latency Constraint Xinglong Sun Barath Lakshmanan Maying Shen Shiyi Lan Jingde Chen Jose M. Alvarez VLM 49 0 0 02 Apr 2025
Efficient Token Compression for Vision Transformer with Spatial Information Preserved Junzhu Mao Yang Shen Jinyang Guo Yazhou Yao Xiansheng Hua ViT 36 0 0 30 Mar 2025
Learning Content-Aware Multi-Modal Joint Input Pruning via Bird's-Eye-View Representation Yuxin Li Yiheng Li Xulei Yang Mengying Yu Zihang Huang Xiaojun Wu Chai Kiat Yeo 37 0 0 09 Oct 2024
Isomorphic Pruning for Vision Models Gongfan Fang Xinyin Ma Michael Bi Mi Xinchao Wang VLM ViT 34 6 0 05 Jul 2024
Speeding Up Image Classifiers with Little Companions Yang Liu Kowshik Thopalli Jayaraman J. Thiagarajan VLM 26 0 0 24 Jun 2024
Automatic Channel Pruning for Multi-Head Attention Eunho Lee Youngbae Hwang ViT 40 1 0 31 May 2024
Divide-Conquer Transformer Learning for Predicting Electric Vehicle Charging Events Using Smart Meter Data Fucai Ke Hao Wang 29 2 0 20 Mar 2024
CAIT: Triple-Win Compression towards High Accuracy, Fast Inference, and Favorable Transferability For ViTs Ao Wang Hui Chen Zijia Lin Sicheng Zhao J. Han Guiguang Ding ViT 31 6 0 27 Sep 2023
Transformer-based models and hardware acceleration analysis in autonomous driving: A survey J. Zhong Zheng Liu Xiangshan Chen ViT 44 17 0 21 Apr 2023
Towards Accurate Post-Training Quantization for Vision Transformer Yifu Ding Haotong Qin Qing-Yu Yan Z. Chai Junjie Liu Xiaolin K. Wei Xianglong Liu MQ 54 68 0 25 Mar 2023
Scratching Visual Transformer's Back with Uniform Attention Nam Hyeon-Woo Kim Yu-Ji Byeongho Heo Doonyoon Han Seong Joon Oh Tae-Hyun Oh 353 23 0 16 Oct 2022
FocusFormer: Focusing on What We Need via Architecture Sampler Jing Liu Jianfei Cai Bohan Zhuang 32 7 0 23 Aug 2022
Peripheral Vision Transformer Juhong Min Yucheng Zhao Chong Luo Minsu Cho ViT MDE 29 30 0 14 Jun 2022
Adaptive Split-Fusion Transformer Zixuan Su Hao Zhang Jingjing Chen Lei Pang Chong-Wah Ngo Yu-Gang Jiang ViT 21 7 0 26 Apr 2022
Dynamic Focus-aware Positional Queries for Semantic Segmentation Haoyu He Jianfei Cai Zizheng Pan Jing Liu Jing Zhang Dacheng Tao Bohan Zhuang 34 17 0 04 Apr 2022
Multi-Dimensional Model Compression of Vision Transformer Zejiang Hou S. Kung ViT 25 16 0 31 Dec 2021
Global Vision Transformer Pruning with Hessian-Aware Saliency Huanrui Yang Hongxu Yin Maying Shen Pavlo Molchanov Hai Helen Li Jan Kautz ViT 30 39 0 10 Oct 2021
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer Sachin Mehta Mohammad Rastegari ViT 218 1,213 0 05 Oct 2021
CMT: Convolutional Neural Networks Meet Vision Transformers Jianyuan Guo Kai Han Han Wu Yehui Tang Chunjing Xu Yunhe Wang Chang Xu ViT 351 633 0 13 Jul 2021
Intriguing Properties of Vision Transformers Muzammal Naseer Kanchana Ranasinghe Salman Khan Munawar Hayat F. Khan Ming-Hsuan Yang ViT 256 621 0 21 May 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 317 5,785 0 29 Apr 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 277 3,623 0 24 Feb 2021
The Lottery Ticket Hypothesis for Pre-trained BERT Networks Tianlong Chen Jonathan Frankle Shiyu Chang Sijia Liu Yang Zhang Zhangyang Wang Michael Carbin 153 345 0 23 Jul 2020
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 950 20,567 0 17 Apr 2017