When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations

3 June 2021

Papers citing "When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations"

41 / 91 papers shown

Title
Assaying Out-Of-Distribution Generalization in Transfer Learning F. Wenzel Andrea Dittadi Peter V. Gehler Carl-Johann Simon-Gabriel Max Horn ... Chris Russell Thomas Brox Bernt Schiele Bernhard Schölkopf Francesco Locatello OOD OODD AAML 60 71 0 19 Jul 2022
Eliminating Gradient Conflict in Reference-based Line-Art Colorization Zekun Li Zhengyang Geng Zhao Kang Wenyu Chen Yibo Yang 21 35 0 13 Jul 2022
Fast Finite Width Neural Tangent Kernel Roman Novak Jascha Narain Sohl-Dickstein S. Schoenholz AAML 22 53 0 17 Jun 2022
A Closer Look at Smoothness in Domain Adversarial Training Harsh Rangwani Sumukh K Aithal Mayank Mishra Arihant Jain R. Venkatesh Babu 32 119 0 16 Jun 2022
INDIGO: Intrinsic Multimodality for Domain Generalization Puneet Mangla Shivam Chandhok Milan Aggarwal V. Balasubramanian Balaji Krishnamurthy VLM 41 2 0 13 Jun 2022
Parotid Gland MRI Segmentation Based on Swin-Unet and Multimodal Images Zián Xu Yin Dai Fayu Liu Siqi Li Shengmei Liu Li-Li Shi Jun Fu ViT MedIm 16 3 0 07 Jun 2022
Which models are innately best at uncertainty estimation? Ido Galil Mohammed Dabbah Ran El-Yaniv UQCV 34 5 0 05 Jun 2022
PSO-Convolutional Neural Networks with Heterogeneous Learning Rate N. H. Phong A. Santos B. Ribeiro 24 8 0 20 May 2022
Temporally Precise Action Spotting in Soccer Videos Using Dense Detection Anchors J. C. V. Soares Avijit Shah Topojoy Biswas 35 32 0 20 May 2022
Simpler is Better: off-the-shelf Continual Learning Through Pretrained Backbones Francesco Pelosin VLM 16 11 0 03 May 2022
Multimodal Transformer for Nursing Activity Recognition Momal Ijaz Renato Diaz Cheng Chen ViT 24 26 0 09 Apr 2022
Improving Vision Transformers by Revisiting High-frequency Components Jiawang Bai Liuliang Yuan Shutao Xia Shuicheng Yan Zhifeng Li Wei Liu ViT 16 90 0 03 Apr 2022
TransGeo: Transformer Is All You Need for Cross-view Image Geo-localization Sijie Zhu M. Shah Cheng Chen ViT 24 147 0 31 Mar 2022
AudioTagging Done Right: 2nd comparison of deep learning methods for environmental sound classification Juncheng Billy Li Shuhui Qu Po-Yao (Bernie) Huang Florian Metze VLM 36 9 0 25 Mar 2022
The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy Tianlong Chen Zhenyu (Allen) Zhang Yu Cheng Ahmed Hassan Awadallah Zhangyang Wang ViT 41 37 0 12 Mar 2022
How Do Vision Transformers Work? Namuk Park Songkuk Kim ViT 47 465 0 14 Feb 2022
Iterative Self Knowledge Distillation -- From Pothole Classification to Fine-Grained and COVID Recognition Kuan-Chuan Peng 41 2 0 04 Feb 2022
When Do Flat Minima Optimizers Work? Jean Kaddour Linqing Liu Ricardo M. A. Silva Matt J. Kusner ODL 24 58 0 01 Feb 2022
VAQF: Fully Automatic Software-Hardware Co-Design Framework for Low-Bit Vision Transformer Mengshu Sun Haoyu Ma Guoliang Kang Yi Ding Tianlong Chen Xiaolong Ma Zhangyang Wang Yanzhi Wang ViT 33 45 0 17 Jan 2022
Video Transformers: A Survey Javier Selva A. S. Johansen Sergio Escalera Kamal Nasrollahi T. Moeslund Albert Clapés ViT 22 103 0 16 Jan 2022
A Simple Single-Scale Vision Transformer for Object Localization and Instance Segmentation Wuyang Chen Xianzhi Du Fan Yang Lucas Beyer Xiaohua Zhai ... Huizhong Chen Jing Li Xiaodan Song Zhangyang Wang Denny Zhou ViT 29 20 0 17 Dec 2021
Sharpness-Aware Minimization with Dynamic Reweighting Wenxuan Zhou Fangyu Liu Huan Zhang Muhao Chen AAML 19 8 0 16 Dec 2021
Scaling Up Influence Functions Andrea Schioppa Polina Zablotskaia David Vilar Artem Sokolov TDI 30 90 0 06 Dec 2021
How Well Do Sparse Imagenet Models Transfer? Eugenia Iofinova Alexandra Peste Mark Kurtz Dan Alistarh 27 38 0 26 Nov 2021
Sharpness-aware Quantization for Deep Neural Networks Jing Liu Jianfei Cai Bohan Zhuang MQ 27 24 0 24 Nov 2021
Discrete Representations Strengthen Vision Transformer Robustness Chengzhi Mao Lu Jiang Mostafa Dehghani Carl Vondrick Rahul Sukthankar Irfan Essa ViT 27 43 0 20 Nov 2021
Are Transformers More Robust Than CNNs? Yutong Bai Jieru Mei Alan Yuille Cihang Xie ViT AAML 192 257 0 10 Nov 2021
Are we ready for a new paradigm shift? A Survey on Visual Deep MLP Ruiyang Liu Hai-Tao Zheng Li Tao Dun Liang Haitao Zheng 85 97 0 07 Nov 2021
Can Vision Transformers Perform Convolution? Shanda Li Xiangning Chen Di He Cho-Jui Hsieh ViT 46 19 0 02 Nov 2021
Deep Active Learning by Leveraging Training Dynamics Haonan Wang Wei Huang Ziwei Wu A. Margenot Hanghang Tong Jingrui He AI4CE 27 33 0 16 Oct 2021
Sharpness-Aware Minimization Improves Language Model Generalization Dara Bahri H. Mobahi Yi Tay 133 98 0 16 Oct 2021
Is Attention Better Than Matrix Decomposition? Zhengyang Geng Meng-Hao Guo Hongxu Chen Xia Li Ke Wei Zhouchen Lin 62 137 0 09 Sep 2021
Scaled ReLU Matters for Training Vision Transformers Pichao Wang Xue Wang Haowen Luo Jingkai Zhou Zhipeng Zhou Fan Wang Hao Li R. L. Jin 19 41 0 08 Sep 2021
Vision Transformers are Robust Learners Sayak Paul Pin-Yu Chen ViT 28 304 0 17 May 2021
MLP-Mixer: An all-MLP Architecture for Vision Ilya O. Tolstikhin N. Houlsby Alexander Kolesnikov Lucas Beyer Xiaohua Zhai ... Andreas Steiner Daniel Keysers Jakob Uszkoreit Mario Lucic Alexey Dosovitskiy 274 2,603 0 04 May 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 347 5,785 0 29 Apr 2021
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text Hassan Akbari Liangzhe Yuan Rui Qian Wei-Hong Chuang Shih-Fu Chang Huayu Chen Boqing Gong ViT 251 577 0 22 Apr 2021
On the Adversarial Robustness of Vision Transformers Rulin Shao Zhouxing Shi Jinfeng Yi Pin-Yu Chen Cho-Jui Hsieh ViT 33 137 0 29 Mar 2021
Is Space-Time Attention All You Need for Video Understanding? Gedas Bertasius Heng Wang Lorenzo Torresani ViT 280 1,982 0 09 Feb 2021
Transformers in Vision: A Survey Salman Khan Muzammal Naseer Munawar Hayat Syed Waqas Zamir Fahad Shahbaz Khan M. Shah ViT 227 2,430 0 04 Jan 2021
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 308 2,890 0 15 Sep 2016