ViTCoD: Vision Transformer Acceleration via Dedicated Algorithm and Accelerator Co-Design

18 October 2022

Papers citing "ViTCoD: Vision Transformer Acceleration via Dedicated Algorithm and Accelerator Co-Design"

27 / 27 papers shown

Title
A Low-Power Streaming Speech Enhancement Accelerator For Edge Devices Ci-Hao Wu Tian-Sheuan Chang 61 1 0 27 Mar 2025
QuartDepth: Post-Training Quantization for Real-Time Depth Estimation on the Edge Xuan Shen Weize Ma Jing Liu Changdi Yang Rui Ding ... Wei Niu Yanzhi Wang Pu Zhao Jun Lin Jiuxiang Gu MQ 57 0 0 20 Mar 2025
T-REX: A 68-567 μs/token, 0.41-3.95 μJ/token Transformer Accelerator with Reduced External Memory Access and Enhanced Hardware Utilization in 16nm FinFET Seunghyun Moon Mao Li Gregory K. Chen Phil Knag R. Krishnamurthy Mingoo Seok 26 0 0 01 Mar 2025
PAPI: Exploiting Dynamic Parallelism in Large Language Model Decoding with a Processing-In-Memory-Enabled Computing System Yintao He Haiyu Mao Christina Giannoula Mohammad Sadrosadati Juan Gómez Luna Huawei Li Xiaowei Li Ying Wang O. Mutlu 43 5 0 21 Feb 2025
Communication-Efficient Sparsely-Activated Model Training via Sequence Migration and Token Condensation Fahao Chen Peng Li Zicong Hong Zhou Su Song Guo MoMe MoE 69 0 0 23 Nov 2024
M $^2$ -ViT: Accelerating Hybrid Vision Transformers with Two-Level Mixed Quantization Yanbiao Liang Huihong Shi Zhongfeng Wang MQ 21 0 0 10 Oct 2024
Foundation Models for Autonomous Robots in Unstructured Environments Hossein Naderi Alireza Shojaei Lifu Huang LM&Ro 47 0 0 19 Jul 2024
CHOSEN: Compilation to Hardware Optimization Stack for Efficient Vision Transformer Inference Mohammad Erfan Sadeghi A. Fayyazi Suhas Somashekar Massoud Pedram 34 2 0 17 Jul 2024
New Solutions on LLM Acceleration, Optimization, and Application Yingbing Huang Lily Jiaxin Wan Hanchen Ye Manvi Jha Jinghua Wang Yuhong Li Xiaofan Zhang Deming Chen 47 12 0 16 Jun 2024
When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models Haoran You Yichao Fu Zheng Wang Amir Yazdanbakhsh Yingyan Celine Lin 33 2 0 11 Jun 2024
P $^2$ -ViT: Power-of-Two Post-Training Quantization and Acceleration for Fully Quantized Vision Transformer Huihong Shi Xin Cheng Wendong Mao Zhongfeng Wang MQ 48 3 0 30 May 2024
SWAT: Scalable and Efficient Window Attention-based Transformers Acceleration on FPGAs Zhenyu Bai Pranav Dangi Huize Li Tulika Mitra 29 5 0 27 May 2024
Trio-ViT: Post-Training Quantization and Acceleration for Softmax-Free Efficient Vision Transformer Huihong Shi Haikuo Shao Wendong Mao Zhongfeng Wang ViT MQ 44 3 0 06 May 2024
Test-Time Model Adaptation with Only Forward Passes Shuaicheng Niu Chunyan Miao Guohao Chen Pengcheng Wu Peilin Zhao TTA 43 18 0 02 Apr 2024
An FPGA-Based Reconfigurable Accelerator for Convolution-Transformer Hybrid EfficientViT Haikuo Shao Huihong Shi Wendong Mao Zhongfeng Wang 23 2 0 29 Mar 2024
BESA: Pruning Large Language Models with Blockwise Parameter-Efficient Sparsity Allocation Peng-Tao Xu Wenqi Shao Mengzhao Chen Shitao Tang Kai-Chuang Zhang Peng Gao Fengwei An Yu Qiao Ping Luo MoE 27 27 0 18 Feb 2024
Accelerator-driven Data Arrangement to Minimize Transformers Run-time on Multi-core Architectures Alireza Amirshahi G. Ansaloni David Atienza 20 0 0 20 Dec 2023
YFlows: Systematic Dataflow Exploration and Code Generation for Efficient Neural Network Inference using SIMD Architectures on CPUs Cyrus Zhou Zack Hassman Ruize Xu Dhirpal Shah Vaughn Richard Yanjing Li 32 1 0 01 Oct 2023
A Survey of Techniques for Optimizing Transformer Inference Krishna Teja Chitty-Venkata Sparsh Mittal M. Emani V. Vishwanath Arun Somani 43 62 0 16 Jul 2023
ShiftAddViT: Mixture of Multiplication Primitives Towards Efficient Vision Transformer Haoran You Huihong Shi Yipin Guo Yingyan Lin Lin 31 16 0 10 Jun 2023
A Survey on Efficient Training of Transformers Bohan Zhuang Jing Liu Zizheng Pan Haoyu He Yuetian Weng Chunhua Shen 31 47 0 02 Feb 2023
Vision Transformer Computation and Resilience for Dynamic Inference Kavya Sreedhar Jason Clemons Rangharajan Venkatesan S. Keckler M. Horowitz 24 2 0 06 Dec 2022
Castling-ViT: Compressing Self-Attention via Switching Towards Linear-Angular Attention at Vision Transformer Inference Haoran You Yunyang Xiong Xiaoliang Dai Bichen Wu Peizhao Zhang Haoqi Fan Peter Vajda Yingyan Lin 35 32 0 18 Nov 2022
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer Sachin Mehta Mohammad Rastegari ViT 218 1,213 0 05 Oct 2021
Predicting Attention Sparsity in Transformers Marcos Vinícius Treviso António Góis Patrick Fernandes E. Fonseca André F. T. Martins 35 13 0 24 Sep 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 283 3,623 0 24 Feb 2021
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 282 2,015 0 28 Jul 2020