CAS-ViT: Convolutional Additive Self-attention Vision Transformers for Efficient Mobile Applications

7 August 2024

Tianfang Zhang

Lei Li

Chen Qian

Papers citing "CAS-ViT: Convolutional Additive Self-attention Vision Transformers for Efficient Mobile Applications"

45 / 45 papers shown

Title
Neural Attention: A Novel Mechanism for Enhanced Expressive Power in Transformer Models Andrew DiGiugno Ausif Mahmood 87 0 0 24 Feb 2025
iFormer: Integrating ConvNet and Transformer for Mobile Application Chuanyang Zheng ViT 121 0 0 26 Jan 2025
PolaFormer: Polarity-aware Linear Attention for Vision Transformers Weikang Meng Yadan Luo Xin Li D. Jiang Zheng Zhang 390 3 0 25 Jan 2025
Scattering Vision Transformer: Spectral Mixing Matters Badri N. Patro Vijay Srinivas Agneeswaran 67 15 0 02 Nov 2023
FasterViT: Fast Vision Transformers with Hierarchical Attention Ali Hatamizadeh Greg Heinrich Hongxu Yin Andrew Tao J. Álvarez Jan Kautz Pavlo Molchanov ViT 100 71 0 09 Jun 2023
Lightweight Vision Transformer with Bidirectional Interaction Qihang Fan Huaibo Huang Xiaoqiang Zhou Ran He ViT 103 29 0 01 Jun 2023
SwiftFormer: Efficient Additive Attention for Transformer-based Real-time Mobile Vision Applications Abdelrahman M. Shaker Muhammad Maaz H. Rasheed Salman Khan Ming-Hsuan Yang Fahad Shahbaz Khan ViT 91 94 0 27 Mar 2023
BiFormer: Vision Transformer with Bi-Level Routing Attention Lei Zhu Xinjiang Wang Zhanghan Ke Wayne Zhang Rynson W. H. Lau 172 513 0 15 Mar 2023
Vision Transformer with Super Token Sampling Huaibo Huang Xiaoqiang Zhou Jie Cao Ran He Tieniu Tan ViT 38 59 0 21 Nov 2022
Next-ViT: Next Generation Vision Transformer for Efficient Deployment in Realistic Industrial Scenarios Jiashi Li Xin Xia W. Li Huixia Li Xing Wang Xuefeng Xiao Rui Wang Min Zheng Xin Pan ViT 47 152 0 12 Jul 2022
EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applications Muhammad Maaz Abdelrahman M. Shaker Hisham Cholakkal Salman Khan Syed Waqas Zamir Rao Muhammad Anwer Fahad Shahbaz Khan ViT 86 197 0 21 Jun 2022
Global Context Vision Transformers Ali Hatamizadeh Hongxu Yin Greg Heinrich Jan Kautz Pavlo Molchanov ViT 62 126 0 20 Jun 2022
MobileOne: An Improved One millisecond Mobile Backbone Pavan Kumar Anasosalu Vasu J. Gabriel Jeff J. Zhu Oncel Tuzel Anurag Ranjan 77 161 0 08 Jun 2022
EfficientFormer: Vision Transformers at MobileNet Speed Yanyu Li Geng Yuan Yang Wen Eric Hu Georgios Evangelidis Sergey Tulyakov Yanzhi Wang Jian Ren ViT 68 364 0 02 Jun 2022
EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformers Junting Pan Adrian Bulat Fuwen Tan Xiatian Zhu Łukasz Dudziak Hongsheng Li Georgios Tzimiropoulos Brais Martínez ViT 66 190 0 06 May 2022
DaViT: Dual Attention Vision Transformers Mingyu Ding Bin Xiao Noel Codella Ping Luo Jingdong Wang Lu Yuan ViT 104 250 0 07 Apr 2022
MetaFormer Is Actually What You Need for Vision Weihao Yu Mi Luo Pan Zhou Chenyang Si Yichen Zhou Xinchao Wang Jiashi Feng Shuicheng Yan 158 909 0 22 Nov 2021
AGPCNet: Attention-Guided Pyramid Context Networks for Infrared Small Target Detection Tianfang Zhang Siying Cao Tian Pu Zhenming Peng 47 56 0 05 Nov 2021
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer Sachin Mehta Mohammad Rastegari ViT 280 1,264 0 05 Oct 2021
CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows Xiaoyi Dong Jianmin Bao Dongdong Chen Weiming Zhang Nenghai Yu Lu Yuan Dong Chen B. Guo ViT 139 981 0 01 Jul 2021
PVT v2: Improved Baselines with Pyramid Vision Transformer Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT AI4TS 91 1,661 0 25 Jun 2021
XCiT: Cross-Covariance Image Transformers Alaaeldin El-Nouby Hugo Touvron Mathilde Caron Piotr Bojanowski Matthijs Douze ... Ivan Laptev Natalia Neverova Gabriel Synnaeve Jakob Verbeek Hervé Jégou ViT 134 512 0 17 Jun 2021
Twins: Revisiting the Design of Spatial Attention in Vision Transformers Xiangxiang Chu Zhi Tian Yuqing Wang Bo Zhang Haibing Ren Xiaolin K. Wei Huaxia Xia Chunhua Shen ViT 79 1,017 0 28 Apr 2021
CvT: Introducing Convolutions to Vision Transformers Haiping Wu Bin Xiao Noel Codella Mengchen Liu Xiyang Dai Lu Yuan Lei Zhang ViT 143 1,907 0 29 Mar 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng Zhang Stephen Lin B. Guo ViT 432 21,392 0 25 Mar 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 516 3,718 0 24 Feb 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 367 6,757 0 23 Dec 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 593 40,961 0 22 Oct 2020
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization Dan Hendrycks Steven Basart Norman Mu Saurav Kadavath Frank Wang ... Samyak Parajuli Mike Guo D. Song Jacob Steinhardt Justin Gilmer OOD 320 1,732 0 29 Jun 2020
Designing Network Design Spaces Ilija Radosavovic Raj Prateek Kosaraju Ross B. Girshick Kaiming He Piotr Dollár GNN 100 1,682 0 30 Mar 2020
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 439 42,393 0 03 Dec 2019
RandAugment: Practical automated data augmentation with a reduced search space E. D. Cubuk Barret Zoph Jonathon Shlens Quoc V. Le MQ 217 3,485 0 30 Sep 2019
Natural Adversarial Examples Dan Hendrycks Kevin Zhao Steven Basart Jacob Steinhardt D. Song OODD 195 1,469 0 16 Jul 2019
MMDetection: Open MMLab Detection Toolbox and Benchmark Kai-xiang Chen Jiaqi Wang Jiangmiao Pang Yuhang Cao Yu Xiong ... Jingdong Wang Jianping Shi Wanli Ouyang Chen Change Loy Dahua Lin VOS 135 2,866 0 17 Jun 2019
Learning Robust Global Representations by Penalizing Local Predictive Power Haohan Wang Songwei Ge Eric Xing Zachary Chase Lipton OOD 114 957 0 29 May 2019
Searching for MobileNetV3 Andrew G. Howard Mark Sandler Grace Chu Liang-Chieh Chen Bo Chen ... Yukun Zhu Ruoming Pang Vijay Vasudevan Quoc V. Le Hartwig Adam 340 6,772 0 06 May 2019
Panoptic Feature Pyramid Networks Alexander Kirillov Ross B. Girshick Kaiming He Piotr Dollár ISeg SSeg 110 1,285 0 08 Jan 2019
MobileNetV2: Inverted Residuals and Linear Bottlenecks Mark Sandler Andrew G. Howard Menglong Zhu A. Zhmoginov Liang-Chieh Chen 173 19,262 0 13 Jan 2018
Non-local Neural Networks Xinyu Wang Ross B. Girshick Abhinav Gupta Kaiming He OffRL 283 8,902 0 21 Nov 2017
Squeeze-and-Excitation Networks Jie Hu Li Shen Samuel Albanie Gang Sun Enhua Wu 412 26,465 0 05 Sep 2017
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Zhuowen Tu Kaiming He 506 10,318 0 16 Nov 2016
Semantic Understanding of Scenes through the ADE20K Dataset Bolei Zhou Hang Zhao Xavier Puig Tete Xiao Sanja Fidler Adela Barriuso Antonio Torralba SSeg 391 1,871 0 18 Aug 2016
SGDR: Stochastic Gradient Descent with Warm Restarts I. Loshchilov Frank Hutter ODL 307 8,114 0 13 Aug 2016
Rethinking the Inception Architecture for Computer Vision Christian Szegedy Vincent Vanhoucke Sergey Ioffe Jonathon Shlens Z. Wojna 3DV BDL 872 27,350 0 02 Dec 2015
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 398 43,619 0 01 May 2014