Advancing Vision Transformers with Group-Mix Attention

26 November 2023

Ping Luo

Papers citing "Advancing Vision Transformers with Group-Mix Attention"

49 / 49 papers shown

Title
BiFormer: Vision Transformer with Bi-Level Routing Attention Lei Zhu Xinjiang Wang Zhanghan Ke Wayne Zhang Rynson W. H. Lau 181 524 0 15 Mar 2023
Inception Transformer Chenyang Si Weihao Yu Pan Zhou Yichen Zhou Xinchao Wang Shuicheng Yan ViT 110 197 0 25 May 2022
DaViT: Dual Attention Vision Transformers Mingyu Ding Bin Xiao Noel Codella Ping Luo Jingdong Wang Lu Yuan ViT 146 251 0 07 Apr 2022
MaxViT: Multi-Axis Vision Transformer Zhengzhong Tu Hossein Talebi Han Zhang Feng Yang P. Milanfar A. Bovik Yinxiao Li ViT 140 670 0 04 Apr 2022
UniFormer: Unifying Convolution and Self-attention for Visual Recognition Kunchang Li Yali Wang Junhao Zhang Peng Gao Guanglu Song Yu Liu Hongsheng Li Yu Qiao ViT 200 381 0 24 Jan 2022
A ConvNet for the 2020s Zhuang Liu Hanzi Mao Chaozheng Wu Christoph Feichtenhofer Trevor Darrell Saining Xie ViT 189 5,226 0 10 Jan 2022
MPViT: Multi-Path Vision Transformer for Dense Prediction Youngwan Lee Jonghee Kim Jeffrey Willette Sung Ju Hwang ViT 94 252 0 21 Dec 2021
MViTv2: Improved Multiscale Vision Transformers for Classification and Detection Yanghao Li Chaoxia Wu Haoqi Fan K. Mangalam Bo Xiong Jitendra Malik Christoph Feichtenhofer ViT 155 693 0 02 Dec 2021
Shunted Self-Attention via Multi-Scale Token Aggregation Sucheng Ren Daquan Zhou Shengfeng He Jiashi Feng Xinchao Wang ViT 82 228 0 30 Nov 2021
Swin Transformer V2: Scaling Up Capacity and Resolution Ze Liu Han Hu Yutong Lin Zhuliang Yao Zhenda Xie ... Yue Cao Zheng Zhang Li Dong Furu Wei B. Guo ViT 221 1,831 0 18 Nov 2021
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer Sachin Mehta Mohammad Rastegari ViT 290 1,285 0 05 Oct 2021
Mobile-Former: Bridging MobileNet and Transformer Yinpeng Chen Xiyang Dai Dongdong Chen Mengchen Liu Xiaoyi Dong Lu Yuan Zicheng Liu ViT 258 491 0 12 Aug 2021
CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows Xiaoyi Dong Jianmin Bao Dongdong Chen Weiming Zhang Nenghai Yu Lu Yuan Dong Chen B. Guo ViT 163 990 0 01 Jul 2021
Focal Self-attention for Local-Global Interactions in Vision Transformers Jianwei Yang Chunyuan Li Pengchuan Zhang Xiyang Dai Bin Xiao Lu Yuan Jianfeng Gao ViT 82 436 0 01 Jul 2021
PVT v2: Improved Baselines with Pyramid Vision Transformer Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT AI4TS 122 1,682 0 25 Jun 2021
VOLO: Vision Outlooker for Visual Recognition Li-xin Yuan Qibin Hou Zihang Jiang Jiashi Feng Shuicheng Yan ViT 118 327 0 24 Jun 2021
Dynamic Head: Unifying Object Detection Heads with Attentions Xiyang Dai Yinpeng Chen Bin Xiao Dongdong Chen Mengchen Liu Lu Yuan Lei Zhang 76 583 0 15 Jun 2021
BEiT: BERT Pre-Training of Image Transformers Hangbo Bao Li Dong Songhao Piao Furu Wei ViT 297 2,845 0 15 Jun 2021
DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification Yongming Rao Wenliang Zhao Benlin Liu Jiwen Lu Jie Zhou Cho-Jui Hsieh ViT 104 708 0 03 Jun 2021
SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers Enze Xie Wenhai Wang Zhiding Yu Anima Anandkumar J. Álvarez Ping Luo ViT 332 5,095 0 31 May 2021
Vision Transformers are Robust Learners Sayak Paul Pin-Yu Chen ViT 67 312 0 17 May 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 735 6,139 0 29 Apr 2021
Multiscale Vision Transformers Haoqi Fan Bo Xiong K. Mangalam Yanghao Li Zhicheng Yan Jitendra Malik Christoph Feichtenhofer ViT 135 1,265 0 22 Apr 2021
All Tokens Matter: Token Labeling for Training Better Vision Transformers Zihang Jiang Qibin Hou Li-xin Yuan Daquan Zhou Yujun Shi Xiaojie Jin Anran Wang Jiashi Feng ViT 114 209 0 22 Apr 2021
Co-Scale Conv-Attentional Image Transformers Weijian Xu Yifan Xu Tyler A. Chang Zhuowen Tu ViT 59 377 0 13 Apr 2021
An Empirical Study of Training Self-Supervised Vision Transformers Xinlei Chen Saining Xie Kaiming He ViT 163 1,873 0 05 Apr 2021
CvT: Introducing Convolutions to Vision Transformers Haiping Wu Bin Xiao Noel Codella Mengchen Liu Xiyang Dai Lu Yuan Lei Zhang ViT 159 1,921 0 29 Mar 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng Zhang Stephen Lin B. Guo ViT 470 21,603 0 25 Mar 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 543 3,740 0 24 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 469 3,906 0 11 Feb 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 389 6,813 0 23 Dec 2020
Sharpness-Aware Minimization for Efficiently Improving Generalization Pierre Foret Ariel Kleiner H. Mobahi Behnam Neyshabur AAML 199 1,359 0 03 Oct 2020
On the Relationship between Self-Attention and Convolutional Layers Jean-Baptiste Cordonnier Andreas Loukas Martin Jaggi 116 535 0 08 Nov 2019
Cascade R-CNN: High Quality Object Detection and Instance Segmentation Zhaowei Cai Nuno Vasconcelos ObjD 84 1,360 0 24 Jun 2019
MMDetection: Open MMLab Detection Toolbox and Benchmark Kai-xiang Chen Jiaqi Wang Jiangmiao Pang Yuhang Cao Yu Xiong ... Jingdong Wang Jianping Shi Wanli Ouyang Chen Change Loy Dahua Lin VOS 184 2,887 0 17 Jun 2019
CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features Sangdoo Yun Dongyoon Han Seong Joon Oh Sanghyuk Chun Junsuk Choe Y. Yoo OOD 627 4,814 0 13 May 2019
Panoptic Feature Pyramid Networks Alexander Kirillov Ross B. Girshick Kaiming He Piotr Dollár ISeg SSeg 151 1,292 0 08 Jan 2019
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design Ningning Ma Xiangyu Zhang Haitao Zheng Jian Sun 181 5,019 0 30 Jul 2018
mixup: Beyond Empirical Risk Minimization Hongyi Zhang Moustapha Cissé Yann N. Dauphin David Lopez-Paz NoLa 316 9,815 0 25 Oct 2017
Random Erasing Data Augmentation Zhun Zhong Liang Zheng Guoliang Kang Shaozi Li Yi Yang 116 3,652 0 16 Aug 2017
Mask R-CNN Kaiming He Georgia Gkioxari Piotr Dollár Ross B. Girshick ObjD 389 27,275 0 20 Mar 2017
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Zhuowen Tu Kaiming He 524 10,358 0 16 Nov 2016
Xception: Deep Learning with Depthwise Separable Convolutions François Chollet MDE BDL PINN 1.5K 14,618 0 07 Oct 2016
Densely Connected Convolutional Networks Gao Huang Zhuang Liu Laurens van der Maaten Kilian Q. Weinberger PINN 3DV 859 36,910 0 25 Aug 2016
Semantic Understanding of Scenes through the ADE20K Dataset Bolei Zhou Hang Zhao Xavier Puig Tete Xiao Sanja Fidler Adela Barriuso Antonio Torralba SSeg 418 1,893 0 18 Aug 2016
Layer Normalization Jimmy Lei Ba J. Kiros Geoffrey E. Hinton 437 10,548 0 21 Jul 2016
Deep Networks with Stochastic Depth Gao Huang Yu Sun Zhuang Liu Daniel Sedra Kilian Q. Weinberger 217 2,365 0 30 Mar 2016
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 1.7K 39,637 0 01 Sep 2014
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 437 43,875 0 01 May 2014