Accelerating ViT Inference on FPGA through Static and Dynamic Pruning

21 March 2024

Dhruv Parikh

Rajgopal Kannan

Viktor Prasanna

Papers citing "Accelerating ViT Inference on FPGA through Static and Dynamic Pruning"

21 / 21 papers shown

Title
TPC-ViT: Token Propagation Controller for Efficient Vision Transformer Wentao Zhu 40 2 0 03 Jan 2024
Structured Pruning for Deep Convolutional Neural Networks: A survey Yang He Lingao Xiao 3DPC 69 136 0 01 Mar 2023
HeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformers Peiyan Dong Mengshu Sun Alec Lu Yanyue Xie Li-Yu Daisy Liu ... Xin Meng Zechao Li Xue Lin Zhenman Fang Yanzhi Wang ViT 71 68 0 15 Nov 2022
Auto-ViT-Acc: An FPGA-Aware Automatic Acceleration Framework for Vision Transformer with Mixed-Scheme Quantization Zechao Li Mengshu Sun Alec Lu Haoyu Ma Geng Yuan ... Yanyu Li M. Leeser Zhangyang Wang Xue Lin Zhenman Fang ViT MQ 42 54 0 10 Aug 2022
Unified Visual Transformer Compression Shixing Yu Tianlong Chen Jiayi Shen Huan Yuan Jianchao Tan Sen Yang Ji Liu Zhangyang Wang ViT 41 93 0 15 Mar 2022
Learned Token Pruning for Transformers Sehoon Kim Sheng Shen D. Thorsley A. Gholami Woosuk Kwon Joseph Hassoun Kurt Keutzer 48 154 0 02 Jul 2021
IA-RED $^2$ : Interpretability-Aware Redundancy Reduction for Vision Transformers Bowen Pan Yikang Shen Yi Ding Zhangyang Wang Rogerio Feris A. Oliva VLM ViT 82 160 0 23 Jun 2021
Chasing Sparsity in Vision Transformers: An End-to-End Exploration Tianlong Chen Yu Cheng Zhe Gan Lu Yuan Lei Zhang Zhangyang Wang ViT 60 221 0 08 Jun 2021
DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification Yongming Rao Wenliang Zhao Benlin Liu Jiwen Lu Jie Zhou Cho-Jui Hsieh ViT 76 697 0 03 Jun 2021
An Empirical Study of Training Self-Supervised Vision Transformers Xinlei Chen Saining Xie Kaiming He ViT 146 1,862 0 05 Apr 2021
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 385 4,919 0 24 Feb 2021
Transformers in Vision: A Survey Salman Khan Muzammal Naseer Munawar Hayat Syed Waqas Zamir Fahad Shahbaz Khan M. Shah ViT 274 2,503 0 04 Jan 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 361 6,757 0 23 Dec 2020
SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning Hanrui Wang Zhekai Zhang Song Han 103 388 0 17 Dec 2020
Efficient Transformer-based Large Scale Language Representations using Hardware-friendly Block Structured Pruning Bingbing Li Zhenglun Kong Tianyun Zhang Ji Li Zechao Li Hang Liu Caiwen Ding VLM 136 64 0 17 Sep 2020
End-to-End Object Detection with Transformers Nicolas Carion Francisco Massa Gabriel Synnaeve Nicolas Usunier Alexander Kirillov Sergey Zagoruyko ViT 3DV PINN 363 13,002 0 26 May 2020
A Unified DNN Weight Compression Framework Using Reweighted Optimization Methods Tianyun Zhang Xiaolong Ma Zheng Zhan Shangli Zhou Minghai Qin Fei Sun Yen-kuang Chen Caiwen Ding M. Fardad Yanzhi Wang 33 5 0 12 Apr 2020
What's Hidden in a Randomly Weighted Neural Network? Vivek Ramanujan Mitchell Wortsman Aniruddha Kembhavi Ali Farhadi Mohammad Rastegari 66 356 0 29 Nov 2019
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks Jonathan Frankle Michael Carbin 221 3,457 0 09 Mar 2018
Layer Normalization Jimmy Lei Ba J. Kiros Geoffrey E. Hinton 364 10,467 0 21 Jul 2016
Learning both Weights and Connections for Efficient Neural Networks Song Han Jeff Pool J. Tran W. Dally CVBM 304 6,660 0 08 Jun 2015