Robust Transformer with Locality Inductive Bias and Feature Normalization

27 January 2023

Omid Nejati Manzari

Hossein Kashiani

Hojat Asgarian Dehkordi

S. B. Shokouhi

ViT

ArXiv (abs)PDF HTML

Papers citing "Robust Transformer with Locality Inductive Bias and Feature Normalization"

37 / 37 papers shown

Title
Road Traffic Sign Recognition method using Siamese network Combining Efficient-CNN based Encoder Zhenghao Xi Yuchao Shao Yang Zheng Xiang Liu Yaqi Liu Yitong Cai 206 0 0 24 Feb 2025
Pyramid Transformer for Traffic Sign Detection Omid Nejati Manzari A. Boudesh S. B. Shokouhi ViT 49 12 0 13 Jul 2022
Are Transformers More Robust Than CNNs? Yutong Bai Jieru Mei Alan Yuille Cihang Xie ViT AAML 256 266 0 10 Nov 2021
Adversarial Robustness Comparison of Vision Transformer and MLP-Mixer to CNNs Philipp Benz Soomin Ham Chaoning Zhang Adil Karjauv In So Kweon AAML ViT 88 80 0 06 Oct 2021
Uformer: A General U-Shaped Transformer for Image Restoration Zhendong Wang Xiaodong Cun Jianmin Bao Wengang Zhou Jianzhuang Liu Houqiang Li ViT 117 1,413 0 06 Jun 2021
Glance-and-Gaze Vision Transformer Qihang Yu Yingda Xia Yutong Bai Yongyi Lu Alan Yuille Wei Shen ViT 67 76 0 04 Jun 2021
ResT: An Efficient Transformer for Visual Recognition Qing-Long Zhang Yubin Yang ViT 70 234 0 28 May 2021
Towards Robust Vision Transformer Xiaofeng Mao Gege Qi YueFeng Chen Xiaodan Li Ranjie Duan Shaokai Ye Yuan He Hui Xue ViT 64 194 0 17 May 2021
Vision Transformers are Robust Learners Sayak Paul Pin-Yu Chen ViT 67 312 0 17 May 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 730 6,135 0 29 Apr 2021
Vision Transformer Pruning Mingjian Zhu Yehui Tang Kai Han ViT 77 92 0 17 Apr 2021
On the Robustness of Vision Transformers to Adversarial Examples Kaleel Mahmood Rigel Mahmood Marten van Dijk ViT 133 225 0 31 Mar 2021
CvT: Introducing Convolutions to Vision Transformers Haiping Wu Bin Xiao Noel Codella Mengchen Liu Xiyang Dai Lu Yuan Lei Zhang ViT 156 1,917 0 29 Mar 2021
Understanding Robustness of Transformers for Image Classification Srinadh Bhojanapalli Ayan Chakrabarti Daniel Glasner Daliang Li Thomas Unterthiner Andreas Veit ViT 96 386 0 26 Mar 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng Zhang Stephen Lin B. Guo ViT 467 21,603 0 25 Mar 2021
Incorporating Convolution Designs into Visual Transformers Kun Yuan Shaopeng Guo Ziwei Liu Aojun Zhou F. Yu Wei Wu ViT 113 480 0 22 Mar 2021
Transformer in Transformer Kai Han An Xiao Enhua Wu Jianyuan Guo Chunjing Xu Yunhe Wang ViT 391 1,574 0 27 Feb 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 535 3,740 0 24 Feb 2021
Conditional Positional Encodings for Vision Transformers Xiangxiang Chu Zhi Tian Bo Zhang Xinlong Wang Chunhua Shen ViT 97 622 0 22 Feb 2021
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet Li-xin Yuan Yunpeng Chen Tao Wang Weihao Yu Yujun Shi Zihang Jiang Francis E. H. Tay Jiashi Feng Shuicheng Yan ViT 146 1,942 0 28 Jan 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 389 6,805 0 23 Dec 2020
Do Wider Neural Networks Really Help Adversarial Robustness? Boxi Wu Jinghui Chen Deng Cai Xiaofei He Quanquan Gu AAML 69 95 0 03 Oct 2020
Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixup Jang-Hyun Kim Wonho Choo Hyun Oh Song AAML 94 390 0 15 Sep 2020
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization Dan Hendrycks Steven Basart Norman Mu Saurav Kadavath Frank Wang ... Samyak Parajuli Mike Guo Basel Alomair Jacob Steinhardt Justin Gilmer OOD 363 1,757 0 29 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 889 42,463 0 28 May 2020
DFNet: Discriminative feature extraction and integration network for salient object detection Mehrdad Noori S. Mohammadi Sina Ghofrani Majelan Ali Bahri Mohammad Havaei 47 34 0 03 Apr 2020
On Feature Normalization and Data Augmentation Boyi Li Felix Wu Ser-Nam Lim Serge J. Belongie Kilian Q. Weinberger 54 136 0 25 Feb 2020
AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty Dan Hendrycks Norman Mu E. D. Cubuk Barret Zoph Justin Gilmer Balaji Lakshminarayanan OOD UQCV 131 1,308 0 05 Dec 2019
CAGNet: Content-Aware Guidance for Salient Object Detection S. Mohammadi Mehrdad Noori Ali Bahri Sina Ghofrani Majelan Mohammad Havaei 65 102 0 29 Nov 2019
XLNet: Generalized Autoregressive Pretraining for Language Understanding Zhilin Yang Zihang Dai Yiming Yang J. Carbonell Ruslan Salakhutdinov Quoc V. Le AI4CE 236 8,451 0 19 Jun 2019
CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features Sangdoo Yun Dongyoon Han Seong Joon Oh Sanghyuk Chun Junsuk Choe Y. Yoo OOD 624 4,809 0 13 May 2019
Making Convolutional Networks Shift-Invariant Again Richard Y. Zhang OOD 93 798 0 25 Apr 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 260 3,747 0 09 Jan 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,229 0 11 Oct 2018
Towards Deep Learning Models Resistant to Adversarial Attacks Aleksander Madry Aleksandar Makelov Ludwig Schmidt Dimitris Tsipras Adrian Vladu SILM OOD 319 12,138 0 19 Jun 2017
Learning Deep Features for Discriminative Localization Bolei Zhou A. Khosla Àgata Lapedriza A. Oliva Antonio Torralba SSL SSeg FAtt 253 9,342 0 14 Dec 2015
Explaining and Harnessing Adversarial Examples Ian Goodfellow Jonathon Shlens Christian Szegedy AAML GAN 282 19,129 0 20 Dec 2014