Separable Self-attention for Mobile Vision Transformers

6 June 2022

Papers citing "Separable Self-attention for Mobile Vision Transformers"

34 / 34 papers shown

Title
Image Recognition with Online Lightweight Vision Transformer: A Survey Zherui Zhang Rongtao Xu Jie Zhou Changwei Wang Xingtian Pei ... Jiguang Zhang Li Guo Longxiang Gao Wenyuan Xu Shibiao Xu ViT 223 0 0 06 May 2025
iFormer: Integrating ConvNet and Transformer for Mobile Application Chuanyang Zheng ViT 80 0 0 26 Jan 2025
A Separable Self-attention Inspired by the State Space Model for Computer Vision Juntao Zhang Shaogeng Liu Kun Bian You Zhou Pei Zhang Jianning Liu Jun Zhou Bingyan Liu Mamba 59 0 0 03 Jan 2025
Improving Accuracy and Generalization for Efficient Visual Tracking Ram J. Zaveri Shivang Patel Yu Gu Gianfranco Doretto VLM 91 0 0 28 Nov 2024
EfficientViM: Efficient Vision Mamba with Hidden State Mixer based State Space Duality Sanghyeok Lee Joonmyung Choi Hyunwoo J. Kim 118 3 0 22 Nov 2024
HDKD: Hybrid Data-Efficient Knowledge Distillation Network for Medical Image Classification Omar S. El-Assiouti Ghada Hamed Dina Khattab H. M. Ebied 47 1 0 10 Jul 2024
Fish-Vista: A Multi-Purpose Dataset for Understanding & Identification of Traits from Images Kazi Sajeed Mehrab M. Maruf Arka Daw Harish Babu Manogaran Abhilash Neog ... Paula Mabee Wasila Dahdul Anuj Karpatne Wasila M Dahdul Anuj Karpatne 41 4 0 10 Jul 2024
Efficient Modulation for Vision Networks Xu Ma Xiyang Dai Jianwei Yang Bin Xiao Yinpeng Chen Yun Fu Lu Yuan 45 17 0 29 Mar 2024
OnDev-LCT: On-Device Lightweight Convolutional Transformers towards federated learning Chu Myaet Thwal Minh N. H. Nguyen Ye Lin Tun Seongjin Kim My T. Thai Choong Seon Hong 67 5 0 22 Jan 2024
TinyFormer: Efficient Transformer Design and Deployment on Tiny Devices Jianlei Yang Jiacheng Liao Fanding Lei Meichen Liu Junyi Chen Lingkun Long Han Wan Bei Yu Weisheng Zhao MoE 35 2 0 03 Nov 2023
Separable Self and Mixed Attention Transformers for Efficient Object Tracking Goutam Yelluru Gopal Maria A. Amer VOT ViT 27 25 0 07 Sep 2023
TurboViT: Generating Fast Vision Transformers via Generative Architecture Search Alexander Wong Saad Abbasi Saeejith Nair ViT 35 1 0 22 Aug 2023
MiDaS v3.1 -- A Model Zoo for Robust Monocular Relative Depth Estimation R. Birkl Diana Wofk Matthias Muller MDE 29 134 0 26 Jul 2023
MobileViG: Graph-Based Sparse Attention for Mobile Vision Applications Mustafa Munir William Avery R. Marculescu ViT GNN 36 33 0 01 Jul 2023
CVSNet: A Computer Implementation for Central Visual System of The Brain Ruimin Gao Hao-Li Zou Zhekai Duan 31 3 0 31 May 2023
FR-Net:A Light-weight FFT Residual Net For Gaze Estimation Tao Xu Borimandafu Wu Ruilong Fan Yun Zhou Di Huang 38 2 0 04 May 2023
SwiftFormer: Efficient Additive Attention for Transformer-based Real-time Mobile Vision Applications Abdelrahman M. Shaker Muhammad Maaz H. Rasheed Salman Khan Ming Yang Fahad Shahbaz Khan ViT 53 84 0 27 Mar 2023
Efficient Activation Function Optimization through Surrogate Modeling G. Bingham Risto Miikkulainen 24 2 0 13 Jan 2023
Skip-Attention: Improving Vision Transformers by Paying Less Attention Shashanka Venkataramanan Amir Ghodrati Yuki M. Asano Fatih Porikli A. Habibian ViT 23 25 0 05 Jan 2023
Rethinking Vision Transformers for MobileNet Size and Speed Yanyu Li Ju Hu Yang Wen Georgios Evangelidis Kamyar Salahi Yanzhi Wang Sergey Tulyakov Jian Ren ViT 40 161 0 15 Dec 2022
Realistic Bokeh Effect Rendering on Mobile GPUs, Mobile AI & AIM 2022 challenge: Report Andrey D. Ignatov Radu Timofte Jin Zhang Feng Zhang G. Yu ... Mingyang Qian Huixin Ma Yanan Li Xiaotao Wang Lei Lei 15 10 0 07 Nov 2022
Grafting Vision Transformers Jong Sung Park Kumara Kahatapitiya Donghyun Kim Shivchander Sudalairaj Quanfu Fan Michael S. Ryoo ViT 29 2 0 28 Oct 2022
MOAT: Alternating Mobile Convolution and Attention Brings Strong Vision Models Chenglin Yang Siyuan Qiao Qihang Yu Xiaoding Yuan Yukun Zhu Alan Yuille Hartwig Adam Liang-Chieh Chen ViT MoE 41 60 0 04 Oct 2022
MobileViTv3: Mobile-Friendly Vision Transformer with Simple and Effective Fusion of Local, Global and Input Features S. Wadekar Abhishek Chaurasia ViT 106 89 0 30 Sep 2022
Next-ViT: Next Generation Vision Transformer for Efficient Deployment in Realistic Industrial Scenarios Jiashi Li Xin Xia W. Li Huixia Li Xing Wang Xuefeng Xiao Rui Wang Min Zheng Xin Pan ViT 17 149 0 12 Jul 2022
EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applications Muhammad Maaz Abdelrahman M. Shaker Hisham Cholakkal Salman Khan Syed Waqas Zamir Rao Muhammad Anwer Fahad Shahbaz Khan ViT 40 184 0 21 Jun 2022
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer Sachin Mehta Mohammad Rastegari ViT 218 1,219 0 05 Oct 2021
ResNet strikes back: An improved training procedure in timm Ross Wightman Hugo Touvron Hervé Jégou AI4TS 212 487 0 01 Oct 2021
Mobile-Former: Bridging MobileNet and Transformer Yinpeng Chen Xiyang Dai Dongdong Chen Mengchen Liu Xiaoyi Dong Lu Yuan Zicheng Liu ViT 183 477 0 12 Aug 2021
ImageNet-21K Pretraining for the Masses T. Ridnik Emanuel Ben-Baruch Asaf Noy Lihi Zelnik-Manor SSeg VLM CLIP 187 690 0 22 Apr 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 316 3,633 0 24 Feb 2021
LambdaNetworks: Modeling Long-Range Interactions Without Attention Irwan Bello 281 179 0 17 Feb 2021
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 950 20,599 0 17 Apr 2017
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 307 39,238 0 01 Sep 2014