Vision Transformer for Small-Size Datasets

27 December 2021

Papers citing "Vision Transformer for Small-Size Datasets"

50 / 109 papers shown

Title
Limited Data, Unlimited Potential: A Study on ViTs Augmented by Masked Autoencoders Srijan Das Tanmay Jain Dominick Reilly P. Balaji Soumyajit Karmakar Shyam Marjit Xiang Li Abhijit Das Michael S. Ryoo 39 16 0 31 Oct 2023
Distilling Inductive Bias: Knowledge Distillation Beyond Model Compression Gousia Habib Tausifa Jan Saleem Brejesh Lall VLM 25 0 0 30 Sep 2023
Multi-Dimensional Hyena for Spatial Inductive Bias Itamar Zimerman Lior Wolf ViT 30 4 0 24 Sep 2023
Asca: less audio data is more insightful Xiang Li Jing Chen Chao Li Hongwu Lv 12 0 0 23 Sep 2023
Increasing diversity of omni-directional images generated from single image using cGAN based on MLPMixer Atsuya Nakata Ryuto Miyazaki Takao Yamanaka 32 1 0 15 Sep 2023
Toward a Deeper Understanding: RetNet Viewed through Convolution Chenghao Li Chaoning Zhang ViT 35 7 0 11 Sep 2023
Boosting Residual Networks with Group Knowledge Shengji Tang Peng Ye Baopu Li Wei Lin Tao Chen Tong He Chong Yu Wanli Ouyang 46 5 0 26 Aug 2023
FedSOL: Stabilized Orthogonal Learning with Proximal Restrictions in Federated Learning Gihun Lee Minchan Jeong Sangmook Kim Jaehoon Oh Se-Young Yun FedML 26 8 0 24 Aug 2023
Denoising Diffusion Probabilistic Model for Retinal Image Generation and Segmentation Alnur Alimanov Md Baharul Islam DiffM MedIm 11 7 0 16 Aug 2023
Experts Weights Averaging: A New General Training Scheme for Vision Transformers Yongqian Huang Peng Ye Xiaoshui Huang Sheng Li Tao Chen Tong He Wanli Ouyang MoMe 26 8 0 11 Aug 2023
Pretrained Deep 2.5D Models for Efficient Predictive Modeling from Retinal OCT T. Emre Marzieh Oghbaie A. Chakravarty Antoine Rivail Sophie Riedl ... S. Sivaprasad Daniel Rueckert A. Lotery U. Schmidt-Erfurth Hrvoje Bogunović MedIm 23 1 0 25 Jul 2023
Image Reconstruction using Enhanced Vision Transformer N. Verma Deepkamal Kaur Lydia W Chau ViT 24 1 0 11 Jul 2023
A Reliable and Interpretable Framework of Multi-view Learning for Liver Fibrosis Staging Zheyao Gao Yuanye Liu Fuping Wu N. Shi Yuxin Shi Xiahai Zhuang EDL 19 11 0 21 Jun 2023
2-D SSM: A General Spatial Layer for Visual Transformers Ethan Baron Itamar Zimerman Lior Wolf 28 14 0 11 Jun 2023
Deep Classifier Mimicry without Data Access Steven Braun Martin Mundt Kristian Kersting DiffM 11 4 0 03 Jun 2023
HUB: Guiding Learned Optimizers with Continuous Prompt Tuning Gaole Dai Wei Yu Wu Ziyu Wang Jie Fu Shanghang Zhang Tiejun Huang AIFin 14 0 0 26 May 2023
Mimetic Initialization of Self-Attention Layers Asher Trockman J. Zico Kolter 30 30 0 16 May 2023
Salient Mask-Guided Vision Transformer for Fine-Grained Classification Dmitry Demidov M.H. Sharif Aliakbar Abdurahimov Hisham Cholakkal F. Khan 29 8 0 11 May 2023
The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learning Micah Goldblum Marc Finzi K. Rowan A. Wilson UQCV FedML 24 37 0 11 Apr 2023
Mask Detection and Classification in Thermal Face Images N. Kowalczyk J. Rumiński 3DH CVBM 11 9 0 06 Apr 2023
Ethylene Leak Detection Based on Infrared Imaging: A Benchmark Xuanchao Ma Yuchen Liu 16 0 0 04 Apr 2023
Astroformer: More Data Might not be all you need for Classification Rishit Dagli 28 7 0 03 Apr 2023
Supervised Masked Knowledge Distillation for Few-Shot Transformers Hanxi Lin G. Han Jiawei Ma Shiyuan Huang Xudong Lin Shih-Fu Chang 24 35 0 25 Mar 2023
Fast offset corrected in-memory training Malte J. Rasch F. Carta Omobayode Fagbohungbe Tayfun Gokmen 26 5 0 08 Mar 2023
Understanding Why ViT Trains Badly on Small Datasets: An Intuitive Perspective Haoran Zhu Boyuan Chen Carter Yang ViT 30 28 0 07 Feb 2023
V1T: large-scale mouse V1 response prediction using a Vision Transformer Bryan M. Li I. M. Cornacchia Nathalie L Rochefort A. Onken 26 8 0 06 Feb 2023
Diffusion Models as Artists: Are we Closing the Gap between Humans and Machines? Victor Boutin Thomas Fel Lakshya Singhal Rishav Mukherji Akash Nagaraj Julien Colin Thomas Serre DiffM 30 6 0 27 Jan 2023
Local Learning on Transformers via Feature Reconstruction P. Pathak Jingwei Zhang Dimitris Samaras ViT 20 5 0 29 Dec 2022
Rethinking Vision Transformers for MobileNet Size and Speed Yanyu Li Ju Hu Yang Wen Georgios Evangelidis Kamyar Salahi Yanzhi Wang Sergey Tulyakov Jian Ren ViT 35 159 0 15 Dec 2022
Masked autoencoders are effective solution to transformer data-hungry Jia-ju Mao Honggu Zhou Xuesong Yin Binling Nie MedIm 32 6 0 12 Dec 2022
ObjectStitch: Generative Object Compositing Yi-Zhe Song Zhifei Zhang Zhe-nan Lin Scott D. Cohen Brian L. Price Jianming Zhang S. Kim Daniel G. Aliaga DiffM 20 31 0 02 Dec 2022
TFormer: A throughout fusion transformer for multi-modal skin lesion diagnosis Yilan Zhang Feng-ying Xie Jianqing Chen MedIm 20 32 0 21 Nov 2022
Spikeformer: A Novel Architecture for Training High-Performance Low-Latency Spiking Neural Network Yudong Li Yunlin Lei Xu Yang 24 26 0 19 Nov 2022
Training a Vision Transformer from scratch in less than 24 hours with 1 GPU Saghar Irandoust Thibaut Durand Yunduz Rakhmangulova Wenjie Zi Hossein Hajimirsadeghi ViT 33 6 0 09 Nov 2022
Explicitly Increasing Input Information Density for Vision Transformers on Small Datasets Xiangyu Chen Ying Qin Wenju Xu A. Bur Cuncong Zhong Guanghui Wang ViT 38 3 0 25 Oct 2022
Accumulated Trivial Attention Matters in Vision Transformers on Small Datasets Xiangyu Chen Qinghao Hu Kaidong Li Cuncong Zhong Guanghui Wang ViT 35 11 0 22 Oct 2022
Neural Attentive Circuits Nasim Rahaman M. Weiß Francesco Locatello C. Pal Yoshua Bengio Bernhard Schölkopf Erran L. Li Nicolas Ballas 32 6 0 14 Oct 2022
How to Train Vision Transformer on Small-scale Datasets? Hanan Gani Muzammal Naseer Mohammad Yaqub ViT 17 49 0 13 Oct 2022
Bridging the Gap Between Vision Transformers and Convolutional Neural Networks on Small Datasets Zhiying Lu Hongtao Xie Chuanbin Liu Yongdong Zhang ViT 25 57 0 12 Oct 2022
Introducing Vision Transformer for Alzheimer's Disease classification task with 3D input Zilun Zhang Farzad Khalvati MedIm ViT 22 9 0 03 Oct 2022
Attention Distillation: self-supervised vision transformer students need more guidance Kai Wang Fei Yang Joost van de Weijer ViT 22 16 0 03 Oct 2022
Neural Integral Equations E. Zappala Antonio H. O. Fonseca J. O. Caro David van Dijk 25 10 0 30 Sep 2022
Augraphy: A Data Augmentation Library for Document Images Alexander Groleau Kok Wei Chee Stefan Larson Samay Maini Jonathan Boarman 24 10 0 30 Aug 2022
Your ViT is Secretly a Hybrid Discriminative-Generative Diffusion Model Xiulong Yang Sheng-Min Shih Yinlin Fu Xiaoting Zhao Shihao Ji DiffM 27 56 0 16 Aug 2022
Class-attention Video Transformer for Engagement Intensity Prediction Xusheng Ai Victor S. Sheng Chunhua Li Zhiming Cui ViT 9 5 0 12 Aug 2022
Improving Deep Neural Network Random Initialization Through Neuronal Rewiring Leonardo F. S. Scabini B. De Baets Odemir M. Bruno AI4CE 23 6 0 17 Jul 2022
EfficientFormer: Vision Transformers at MobileNet Speed Yanyu Li Geng Yuan Yang Wen Eric Hu Georgios Evangelidis Sergey Tulyakov Yanzhi Wang Jian Ren ViT 23 347 0 02 Jun 2022
Few-Shot Diffusion Models Giorgio Giannone Didrik Nielsen Ole Winther DiffM 183 49 0 30 May 2022
Temporal Latent Bottleneck: Synthesis of Fast and Slow Processing Mechanisms in Sequence Learning Aniket Didolkar Kshitij Gupta Anirudh Goyal Nitesh B. Gundavarapu Alex Lamb Nan Rosemary Ke Yoshua Bengio AI4CE 112 17 0 30 May 2022
MDMLP: Image Classification from Scratch on Small Datasets with MLP Tianxu Lv Chongyang Bai Chaojie Wang 24 5 0 28 May 2022