How to Train Vision Transformer on Small-scale Datasets?

How to Train Vision Transformer on Small-scale Datasets?

13 October 2022

Muzammal Naseer

Papers citing "How to Train Vision Transformer on Small-scale Datasets?"

12 / 12 papers shown

Title
Do not trust what you trust: Miscalibration in Semi-supervised Learning Shambhavi Mishra Balamurali Murugesan Ismail Ben Ayed M. Pedersoli Jose Dolz 48 2 0 22 Mar 2024
TULIP: Transformer for Upsampling of LiDAR Point Clouds Bin Yang Patrick Pfreundschuh Roland Siegwart Marco Hutter Peyman Moghadam Vaishakh Patil 3DPC 26 6 0 11 Dec 2023
Initializing Models with Larger Ones Zhiqiu Xu Yanjie Chen Kirill Vishniakov Yida Yin Zhiqiang Shen Trevor Darrell Lingjie Liu Zhuang Liu 33 17 0 30 Nov 2023
Mimetic Initialization of Self-Attention Layers Asher Trockman J. Zico Kolter 30 30 0 16 May 2023
MPCViT: Searching for Accurate and Efficient MPC-Friendly Vision Transformer with Heterogeneous Attention Wenyuan Zeng Meng Li Wenjie Xiong Tong Tong Wen-jie Lu Jin Tan Runsheng Wang Ru Huang 24 20 0 25 Nov 2022
Explicitly Increasing Input Information Density for Vision Transformers on Small Datasets Xiangyu Chen Ying Qin Wenju Xu A. Bur Cuncong Zhong Guanghui Wang ViT 38 3 0 25 Oct 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 305 7,443 0 11 Nov 2021
Intriguing Properties of Vision Transformers Muzammal Naseer Kanchana Ranasinghe Salman Khan Munawar Hayat F. Khan Ming-Hsuan Yang ViT 256 621 0 21 May 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 317 5,785 0 29 Apr 2021
Transformers in Vision: A Survey Salman Khan Muzammal Naseer Munawar Hayat Syed Waqas Zamir F. Khan M. Shah ViT 227 2,430 0 04 Jan 2021
Deep High-Resolution Representation Learning for Visual Recognition Jingdong Wang Ke Sun Tianheng Cheng Borui Jiang Chaorui Deng ... Yadong Mu Mingkui Tan Xinggang Wang Wenyu Liu Bin Xiao 195 3,531 0 20 Aug 2019
Densely Connected Convolutional Networks Gao Huang Zhuang Liu L. V. D. van der Maaten Kilian Q. Weinberger PINN 3DV 264 36,371 0 25 Aug 2016