An Empirical Study of Training Self-Supervised Vision Transformers

5 April 2021

Papers citing "An Empirical Study of Training Self-Supervised Vision Transformers"

50 / 469 papers shown

Title
How Well Do Vision Transformers (VTs) Transfer To The Non-Natural Image Domain? An Empirical Study Involving Art Classification Vincent Tonkes M. Sabatelli ViT 30 6 0 09 Aug 2022
MVSFormer: Multi-View Stereo by Learning Robust Image Features and Temperature-based Depth Chenjie Cao Xinlin Ren Yanwei Fu 31 46 0 04 Aug 2022
Multi-Feature Vision Transformer via Self-Supervised Representation Learning for Improvement of COVID-19 Diagnosis Xiao Qi D. Foran J. Nosher I. Hacihaliloglu ViT MedIm 30 3 0 03 Aug 2022
A Survey on Masked Autoencoder for Self-supervised Learning in Vision and Beyond Chaoning Zhang Chenshuang Zhang Junha Song John Seon Keun Yi Kang Zhang In So Kweon SSL 57 71 0 30 Jul 2022
Contrastive Masked Autoencoders are Stronger Vision Learners Zhicheng Huang Xiaojie Jin Cheng Lu Qibin Hou Mingg-Ming Cheng Dongmei Fu Xiaohui Shen Jiashi Feng 50 148 0 27 Jul 2022
Jigsaw-ViT: Learning Jigsaw Puzzles in Vision Transformer Yingyi Chen Xiaoke Shen Yahui Liu Qinghua Tao Johan A. K. Suykens AAML ViT 28 22 0 25 Jul 2022
Improved Super Resolution of MR Images Using CNNs and Vision Transformers Dwarikanath Mahapatra SupR ViT MedIm 27 5 0 24 Jul 2022
High-Resolution Swin Transformer for Automatic Medical Image Segmentation Chen Wei Shenghan Ren Kaitai Guo Haihong Hu Jimin Liang ViT OOD MedIm 25 36 0 23 Jul 2022
Emotion Separation and Recognition from a Facial Expression by Generating the Poker Face with Vision Transformers Jia Li Jian‐Hui Nie Dan Guo Richang Hong Meng Wang ViT 32 14 0 22 Jul 2022
Decoupled Adversarial Contrastive Learning for Self-supervised Adversarial Robustness Chaoning Zhang Kang Zhang Chenshuang Zhang Axi Niu Jiu Feng Chang D. Yoo In So Kweon SSL 35 24 0 22 Jul 2022
Contrastive Self-Supervised Learning Leads to Higher Adversarial Susceptibility Rohit Gupta Naveed Akhtar Ajmal Mian M. Shah AAML SSL 26 5 0 22 Jul 2022
Negative Samples are at Large: Leveraging Hard-distance Elastic Loss for Re-identification Hyungtae Lee Sungmin Eum H. Kwon VLM 22 4 0 20 Jul 2022
Position Prediction as an Effective Pretraining Strategy Shuangfei Zhai Navdeep Jaitly Jason Ramapuram Dan Busbridge Tatiana Likhomanenko Joseph Y. Cheng Walter A. Talbott Chen Huang Hanlin Goh J. Susskind ViT 46 23 0 15 Jul 2022
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval Yiwei Ma Guohai Xu Xiaoshuai Sun Ming Yan Ji Zhang Rongrong Ji CLIP VLM 34 271 0 15 Jul 2022
Bootstrapped Masked Autoencoders for Vision BERT Pretraining Xiaoyi Dong Jianmin Bao Ting Zhang Dongdong Chen Weiming Zhang Lu Yuan Dong Chen Fang Wen Nenghai Yu 22 75 0 14 Jul 2022
Masked Autoencoders that Listen Po-Yao (Bernie) Huang Hu Xu Juncheng Billy Li Alexei Baevski Michael Auli Wojciech Galuba Florian Metze Christoph Feichtenhofer 21 268 0 13 Jul 2022
Eliminating Gradient Conflict in Reference-based Line-Art Colorization Zekun Li Zhengyang Geng Zhao Kang Wenyu Chen Yibo Yang 21 35 0 13 Jul 2022
LaT: Latent Translation with Cycle-Consistency for Video-Text Retrieval Jinbin Bai Chunhui Liu Feiyue Ni Haofan Wang Mengying Hu Xiaofeng Guo Lele Cheng 45 11 0 11 Jul 2022
Facilitated machine learning for image-based fruit quality assessment Manuel Knott Fernando Perez-Cruz T. Defraeye 21 47 0 10 Jul 2022
Big Learning Yulai Cong Miaoyun Zhao AI4CE 32 0 0 08 Jul 2022
Revisiting Classifier: Transferring Vision-Language Models for Video Recognition Wenhao Wu Zhun Sun Wanli Ouyang VLM 103 93 0 04 Jul 2022
Reading and Writing: Discriminative and Generative Modeling for Self-Supervised Text Recognition Mingkun Yang Minghui Liao Pu Lu Jing Wang Shenggao Zhu Hualin Luo Qingzhen Tian X. Bai SSL 33 55 0 01 Jul 2022
Masked World Models for Visual Control Younggyo Seo Danijar Hafner Hao Liu Fangchen Liu Stephen James Kimin Lee Pieter Abbeel OffRL 93 147 0 28 Jun 2022
EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithm Jiangning Zhang Xiangtai Li Yabiao Wang Chengjie Wang Yibo Yang Yong Liu Dacheng Tao ViT 34 32 0 19 Jun 2022
Bootstrapped Transformer for Offline Reinforcement Learning Kerong Wang Hanye Zhao Xufang Luo Kan Ren Weinan Zhang Dongsheng Li OffRL 16 37 0 17 Jun 2022
Rectify ViT Shortcut Learning by Visual Saliency Chong Ma Lin Zhao Yuzhong Chen David Liu Xi Jiang Tuo Zhang Xintao Hu Dinggang Shen Dajiang Zhu Tianming Liu ViT 36 20 0 17 Jun 2022
OmniMAE: Single Model Masked Pretraining on Images and Videos Rohit Girdhar Alaaeldin El-Nouby Mannat Singh Kalyan Vasudev Alwala Armand Joulin Ishan Misra ViT 37 97 0 16 Jun 2022
Evaluating Self-Supervised Learning for Molecular Graph Embeddings Hanchen Wang Jean Kaddour Shengchao Liu Jian Tang Joan Lasenby Qi Liu 30 20 0 16 Jun 2022
Patch-level Representation Learning for Self-supervised Vision Transformers Sukmin Yun Hankook Lee Jaehyung Kim Jinwoo Shin ViT 22 64 0 16 Jun 2022
Masked Siamese ConvNets L. Jing Jiachen Zhu Yann LeCun SSL 35 34 0 15 Jun 2022
Rethinking Generalization in Few-Shot Classification Markus Hiller Rongkai Ma Mehrtash Harandi Tom Drummond OCL VLM 30 55 0 15 Jun 2022
Multimodal Learning with Transformers: A Survey P. Xu Xiatian Zhu David A. Clifton ViT 72 528 0 13 Jun 2022
Discovering Object Masks with Transformers for Unsupervised Semantic Segmentation Wouter Van Gansbeke Simon Vandenhende Luc Van Gool 44 55 0 13 Jun 2022
Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels? Xiang Li Jinghuan Shang Srijan Das Michael S. Ryoo SSL 27 31 0 10 Jun 2022
Extreme Masking for Learning Instance and Distributed Visual Representations Zhirong Wu Zihang Lai Xiao Sun Stephen Lin 35 22 0 09 Jun 2022
Efficient Self-supervised Vision Pretraining with Local Masked Reconstruction Jun Chen Ming Hu Boyang Albert Li Mohamed Elhoseiny 47 36 0 01 Jun 2022
Contrastive Centroid Supervision Alleviates Domain Shift in Medical Image Classification Wenshuo Zhou Dalu Yang Binghong Wu Yehui Yang Junde Wu Xiaorong Wang Lei Wang Haifeng Huang Yanwu Xu OOD 47 1 0 31 May 2022
Self-Supervised Pre-training of Vision Transformers for Dense Prediction Tasks Jaonary Rabarisoa Velentin Belissen Florian Chabot Q. C. Pham VLM ViT SSL MDE 26 2 0 30 May 2022
GMML is All you Need Sara Atito Muhammad Awais J. Kittler ViT VLM 46 18 0 30 May 2022
HiViT: Hierarchical Vision Transformer Meets Masked Image Modeling Xiaosong Zhang Yunjie Tian Wei Huang QiXiang Ye Qi Dai Lingxi Xie Qi Tian 64 26 0 30 May 2022
SupMAE: Supervised Masked Autoencoders Are Efficient Vision Learners Feng Liang Yangguang Li Diana Marculescu SSL TPM ViT 51 22 0 28 May 2022
A Closer Look at Self-Supervised Lightweight Vision Transformers Shaoru Wang Jin Gao Zeming Li Jian Sun Weiming Hu ViT 67 41 0 28 May 2022
Contrastive Learning Rivals Masked Image Modeling in Fine-tuning via Feature Distillation Yixuan Wei Han Hu Zhenda Xie Zheng-Wei Zhang Yue Cao Jianmin Bao Dong Chen B. Guo CLIP 88 124 0 27 May 2022
Architecture-Agnostic Masked Image Modeling -- From ViT back to CNN Siyuan Li Di Wu Fang Wu Lei Shang Stan.Z.Li 34 48 0 27 May 2022
Green Hierarchical Vision Transformer for Masked Image Modeling Lang Huang Shan You Mingkai Zheng Fei Wang Chao Qian T. Yamasaki 35 68 0 26 May 2022
Improvements to Self-Supervised Representation Learning for Masked Image Modeling Jia-ju Mao Xuesong Yin Yuan Chang Honggu Zhou SSL 27 1 0 21 May 2022
Self-supervised 3D anatomy segmentation using self-distilled masked image transformer (SMIT) Jue Jiang N. Tyagi K. Tringale C. Crane Harini Veeraraghavan MedIm 36 34 0 20 May 2022
Masked Image Modeling with Denoising Contrast Kun Yi Yixiao Ge Xiaotong Li Shusheng Yang Dian Li Jianping Wu Ying Shan Xiaohu Qie VLM 30 51 0 19 May 2022
Label-Efficient Self-Supervised Federated Learning for Tackling Data Heterogeneity in Medical Imaging Rui Yan Liangqiong Qu Qingyue Wei Shih-Cheng Huang Liyue Shen D. Rubin Lei Xing Yuyin Zhou FedML 78 90 0 17 May 2022
Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localization Luke Melas-Kyriazi Christian Rupprecht Iro Laina Andrea Vedaldi 30 160 0 16 May 2022