DeiT III: Revenge of the ViT

14 April 2022

Papers citing "DeiT III: Revenge of the ViT"

32 / 82 papers shown

Title
Representation Separation for Semantic Segmentation with Vision Transformers Yuanduo Hong Huihui Pan Weichao Sun Xinghu Yu Huijun Gao ViT 28 5 0 28 Dec 2022
Rethinking Vision Transformers for MobileNet Size and Speed Yanyu Li Ju Hu Yang Wen Georgios Evangelidis Kamyar Salahi Yanzhi Wang Sergey Tulyakov Jian Ren ViT 35 159 0 15 Dec 2022
Co-training $2^L$ Submodels for Visual Recognition Hugo Touvron Matthieu Cord Maxime Oquab Piotr Bojanowski Jakob Verbeek Hervé Jégou VLM 35 9 0 09 Dec 2022
Deep Incubation: Training Large Models by Divide-and-Conquering Zanlin Ni Yulin Wang Jiangwei Yu Haojun Jiang Yu Cao Gao Huang VLM 18 11 0 08 Dec 2022
Learning Imbalanced Data with Vision Transformers Zhengzhuo Xu R. Liu Shuo Yang Zenghao Chai Chun Yuan 37 32 0 05 Dec 2022
Towards All-in-one Pre-training via Maximizing Multi-modal Mutual Information Weijie Su Xizhou Zhu Chenxin Tao Lewei Lu Bin Li Gao Huang Yu Qiao Xiaogang Wang Jie Zhou Jifeng Dai 42 41 0 17 Nov 2022
EVA: Exploring the Limits of Masked Visual Representation Learning at Scale Yuxin Fang Wen Wang Binhui Xie Quan-Sen Sun Ledell Yu Wu Xinggang Wang Tiejun Huang Xinlong Wang Yue Cao VLM CLIP 87 675 0 14 Nov 2022
ParCNetV2: Oversized Kernel with Enhanced Attention Ruihan Xu Haokui Zhang Wenze Hu Shiliang Zhang Xiaoyu Wang ViT 30 6 0 14 Nov 2022
BiViT: Extremely Compressed Binary Vision Transformer Yefei He Zhenyu Lou Luoming Zhang Jing Liu Weijia Wu Hong Zhou Bohan Zhuang ViT MQ 20 28 0 14 Nov 2022
InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions Wenhai Wang Jifeng Dai Zhe Chen Zhenhang Huang Zhiqi Li ... Tong Lu Lewei Lu Hongsheng Li Xiaogang Wang Yu Qiao VLM 36 657 0 10 Nov 2022
Effective Audio Classification Network Based on Paired Inverse Pyramid Structure and Dense MLP Block Yunhao Chen Yunjie Zhu Zihui Yan Yifan Huang Zhen Ren Jianlu Shen Lifang Chen 28 9 0 05 Nov 2022
Towards Sustainable Self-supervised Learning Shanghua Gao Pan Zhou Mingg-Ming Cheng Shuicheng Yan CLL 48 7 0 20 Oct 2022
lo-fi: distributed fine-tuning without communication Mitchell Wortsman Suchin Gururangan Shen Li Ali Farhadi Ludwig Schmidt Michael G. Rabbat Ari S. Morcos 32 24 0 19 Oct 2022
Token-Label Alignment for Vision Transformers Han Xiao Wenzhao Zheng Zhengbiao Zhu Jie Zhou Jiwen Lu 21 4 0 12 Oct 2022
OCD: Learning to Overfit with Conditional Diffusion Models Shahar Lutati Lior Wolf DiffM 20 8 0 02 Oct 2022
ViT-DD: Multi-Task Vision Transformer for Semi-Supervised Driver Distraction Detection Yunsheng Ma Ziran Wang ViT 41 14 0 19 Sep 2022
Accelerating Vision Transformer Training via a Patch Sampling Schedule Bradley McDanel C. Huynh ViT 25 1 0 19 Aug 2022
Vision Transformers: From Semantic Segmentation to Dense Prediction Li Zhang Jiachen Lu Sixiao Zheng Xinxuan Zhao Xiatian Zhu Yanwei Fu Tao Xiang Jianfeng Feng Philip H. S. Torr ViT 27 7 0 19 Jul 2022
EfficientFormer: Vision Transformers at MobileNet Speed Yanyu Li Geng Yuan Yang Wen Eric Hu Georgios Evangelidis Sergey Tulyakov Yanzhi Wang Jian Ren ViT 23 347 0 02 Jun 2022
CropMix: Sampling a Rich Input Distribution via Multi-Scale Cropping Junlin Han L. Petersson Hongdong Li Ian Reid 33 9 0 31 May 2022
SupMAE: Supervised Masked Autoencoders Are Efficient Vision Learners Feng Liang Yangguang Li Diana Marculescu SSL TPM ViT 51 22 0 28 May 2022
A Closer Look at Self-Supervised Lightweight Vision Transformers Shaoru Wang Jin Gao Zeming Li Jian Sun Weiming Hu ViT 67 41 0 28 May 2022
Vision Transformers in 2022: An Update on Tiny ImageNet Ethan Huynh ViT 31 11 0 21 May 2022
Better plain ViT baselines for ImageNet-1k Lucas Beyer Xiaohua Zhai Alexander Kolesnikov ViT VLM 33 111 0 03 May 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 305 7,443 0 11 Nov 2021
ResNet strikes back: An improved training procedure in timm Ross Wightman Hugo Touvron Hervé Jégou AI4TS 212 487 0 01 Oct 2021
MLP-Mixer: An all-MLP Architecture for Vision Ilya O. Tolstikhin N. Houlsby Alexander Kolesnikov Lucas Beyer Xiaohua Zhai ... Andreas Steiner Daniel Keysers Jakob Uszkoreit Mario Lucic Alexey Dosovitskiy 274 2,603 0 04 May 2021
Transformer in Transformer Kai Han An Xiao Enhua Wu Jianyuan Guo Chunjing Xu Yunhe Wang ViT 289 1,524 0 27 Feb 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 289 3,623 0 24 Feb 2021
Video Transformer Network Daniel Neimark Omri Bar Maya Zohar Dotan Asselmann ViT 204 422 0 01 Feb 2021
Fixing the train-test resolution discrepancy: FixEfficientNet Hugo Touvron Andrea Vedaldi Matthijs Douze Hervé Jégou AAML 196 110 0 18 Mar 2020
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 296 39,198 0 01 Sep 2014