v1v2 (latest)

TinyFormer: Efficient Transformer Design and Deployment on Tiny Devices

3 November 2023

Papers citing "TinyFormer: Efficient Transformer Design and Deployment on Tiny Devices"

33 / 33 papers shown

Title
MCUFormer: Deploying Vision Transformers on Microcontrollers with Limited Memory Yinan Liang Ziwei Wang Xiuwei Xu Yansong Tang Jie Zhou Jiwen Lu 84 10 0 25 Oct 2023
TinyML: Tools, Applications, Challenges, and Future Research Directions Rakhee Kallimani K. Pai Prasoon Raghuwanshi S. Iyer O. López 103 43 0 23 Mar 2023
Hardware-Aware Graph Neural Network Automated Design for Edge Computing Platforms Ao Zhou Jianlei Yang Yingjie Qi Yumeng Shi Tong Qiao Weisheng Zhao Chunming Hu GNN 119 12 0 20 Mar 2023
On-Device Training Under 256KB Memory Ji Lin Ligeng Zhu Wei-Ming Chen Wei-Chen Wang Chuang Gan Song Han MQ 130 212 0 30 Jun 2022
Separable Self-attention for Mobile Vision Transformers Sachin Mehta Mohammad Rastegari ViT MQ 94 265 0 06 Jun 2022
MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning Ji Lin Wei-Ming Chen Han Cai Chuang Gan Song Han 92 159 0 28 Oct 2021
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer Sachin Mehta Mohammad Rastegari ViT 294 1,290 0 05 Oct 2021
Escaping the Big Data Paradigm with Compact Transformers Ali Hassani Steven Walton Nikhil Shah Abulikemu Abuduweili Jiachen Li Humphrey Shi 144 464 0 12 Apr 2021
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 693 41,663 0 22 Oct 2020
Leveraging Automated Mixed-Low-Precision Quantization for tiny edge microcontrollers Manuele Rusci Marco Fariselli Alessandro Capotondi Luca Benini MQ 65 17 0 12 Aug 2020
SparseTrain: Exploiting Dataflow Sparsity for Efficient Convolutional Neural Networks Training Pengcheng Dai Jianlei Yang Xucheng Ye Xingzhou Cheng Junyu Luo Linghao Song Yiran Chen Weisheng Zhao 74 22 0 21 Jul 2020
MCUNet: Tiny Deep Learning on IoT Devices Ji Lin Wei-Ming Chen Chengyue Wu J. Cohn Chuang Gan Song Han 162 494 0 20 Jul 2020
Conformer: Convolution-augmented Transformer for Speech Recognition Anmol Gulati James Qin Chung-Cheng Chiu Niki Parmar Yu Zhang ... Wei Han Shibo Wang Zhengdong Zhang Yonghui Wu Ruoming Pang 229 3,168 0 16 May 2020
Benchmarking TinyML Systems: Challenges and Direction Colby R. Banbury Vijay Janapa Reddi Max Lam William Fu A. Fazel ... Jae-sun Seo Jeff Sieracki Urmish Thakker Marian Verhelst Poonam Yadav 166 237 0 10 Mar 2020
Hardware/Software Co-Exploration of Neural Architectures Weiwen Jiang Lei Yang E. Sha Qingfeng Zhuge Shouzhen Gu Sakyasingha Dasgupta Yiyu Shi Jiaxi Hu 97 131 0 06 Jul 2019
Memory-Driven Mixed Low Precision Quantization For Enabling Deep Network Inference On Microcontrollers Manuele Rusci Alessandro Capotondi Luca Benini MQ 97 75 0 30 May 2019
SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers Igor Fedorov Ryan P. Adams Matthew Mattina P. Whatmough 87 168 0 28 May 2019
Single Path One-Shot Neural Architecture Search with Uniform Sampling Zichao Guo Xiangyu Zhang Haoyuan Mu Wen Heng Zechun Liu Yichen Wei Jian Sun 127 941 0 31 Mar 2019
FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search Bichen Wu Xiaoliang Dai Peizhao Zhang Yanghan Wang Fei Sun Yiming Wu Yuandong Tian Peter Vajda Yangqing Jia Kurt Keutzer MQ 108 1,307 0 09 Dec 2018
ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware Han Cai Ligeng Zhu Song Han 112 1,877 0 02 Dec 2018
Quantizing deep convolutional networks for efficient inference: A whitepaper Raghuraman Krishnamoorthi MQ 143 1,022 0 21 Jun 2018
Deep Learning using Rectified Linear Units (ReLU) Abien Fred Agarap 95 3,246 0 22 Mar 2018
CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs Liangzhen Lai Naveen Suda Vikas Chandra 83 381 0 19 Jan 2018
MobileNetV2: Inverted Residuals and Linear Bottlenecks Mark Sandler Andrew G. Howard Menglong Zhu A. Zhmoginov Liang-Chieh Chen 254 19,384 0 13 Jan 2018
Hello Edge: Keyword Spotting on Microcontrollers Yundong Zhang Naveen Suda Liangzhen Lai Vikas Chandra 94 437 0 20 Nov 2017
To prune, or not to prune: exploring the efficacy of pruning for model compression Michael Zhu Suyog Gupta 202 1,284 0 05 Oct 2017
A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets P. Chrabaszcz I. Loshchilov Frank Hutter SSeg OOD 179 649 0 27 Jul 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 842 132,854 0 12 Jun 2017
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 517 5,388 0 05 Nov 2016
TensorFlow: A system for large-scale machine learning Martín Abadi P. Barham Jianmin Chen Zhiwen Chen Andy Davis ... Vijay Vasudevan Pete Warden Martin Wicke Yuan Yu Xiaoqiang Zhang GNN AI4CE 435 18,361 0 27 May 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.3K 194,774 0 10 Dec 2015
Rethinking the Inception Architecture for Computer Vision Christian Szegedy Vincent Vanhoucke Sergey Ioffe Jonathon Shlens Z. Wojna 3DV BDL 888 27,453 0 02 Dec 2015
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification Kaiming He Xinming Zhang Shaoqing Ren Jian Sun VLM 372 18,666 0 06 Feb 2015