An Empirical Study of Training Self-Supervised Vision Transformers

5 April 2021

Papers citing "An Empirical Study of Training Self-Supervised Vision Transformers"

50 / 469 papers shown

Title
SSL4EO-S12: A Large-Scale Multi-Modal, Multi-Temporal Dataset for Self-Supervised Learning in Earth Observation Yi Wang Nassim Ait Ali Braham Zhitong Xiong Chenying Liu C. Albrecht Xiao Xiang Zhu 34 71 0 13 Nov 2022
Token Transformer: Can class token help window-based transformer build better long-range interactions? Jia-ju Mao Yuan Chang Xuesong Yin 34 0 0 11 Nov 2022
Masked Contrastive Representation Learning Yuan Yao Nandakishor Desai M. Palaniswami SSL 22 8 0 11 Nov 2022
Contrastive Self-Supervised Learning for Skeleton Representations N. Lingg Miguel Sarabia Luca Zappella B. Theobald SSL 21 0 0 10 Nov 2022
Distilling Representations from GAN Generator via Squeeze and Span Yu Yang Xiaotian Cheng Chang-rui Liu Hakan Bilen Xiang Ji GAN 31 0 0 06 Nov 2022
Pixel-Wise Contrastive Distillation Junqiang Huang Zichao Guo 42 4 0 01 Nov 2022
Max Pooling with Vision Transformers reconciles class and shape in weakly supervised semantic segmentation Simone Rossetti Damiano Zappia Marta Sanzari M. Schaerf F. Pirri ViT 36 57 0 31 Oct 2022
A simple, efficient and scalable contrastive masked autoencoder for learning visual representations Shlok Kumar Mishra Joshua Robinson Huiwen Chang David Jacobs Aaron Sarna Aaron Maschinot Dilip Krishnan DiffM 43 30 0 30 Oct 2022
Open-vocabulary Semantic Segmentation with Frozen Vision-Language Models Chaofan Ma Yu-Hao Yang Yanfeng Wang Ya Zhang Weidi Xie VLM 31 48 0 27 Oct 2022
Exploiting Features and Logits in Heterogeneous Federated Learning Yun-Hin Chan Edith C.H. Ngai FedML 32 2 0 27 Oct 2022
Learning Explicit Object-Centric Representations with Vision Transformers Oscar Vikström Alexander Ilin OCL ViT 38 4 0 25 Oct 2022
Deep Model Reassembly Xingyi Yang Zhou Daquan Songhua Liu Jingwen Ye Xinchao Wang MoMe 20 120 0 24 Oct 2022
Foreground Guidance and Multi-Layer Feature Fusion for Unsupervised Object Discovery with Transformers Zhiwei Lin Ze Yang Yongtao Wang ViT 39 2 0 24 Oct 2022
Adversarial Pretraining of Self-Supervised Deep Networks: Past, Present and Future Guo-Jun Qi M. Shah SSL 23 8 0 23 Oct 2022
Boosting vision transformers for image retrieval Chull Hwan Song Jooyoung Yoon Shunghyun Choi Yannis Avrithis ViT 34 32 0 21 Oct 2022
Self-Supervised Learning via Maximum Entropy Coding Xin Liu Zhongdao Wang Yali Li Shengjin Wang SSL 29 40 0 20 Oct 2022
Similarity of Neural Architectures using Adversarial Attack Transferability Jaehui Hwang Dongyoon Han Byeongho Heo Song Park Sanghyuk Chun Jong-Seok Lee AAML 32 1 0 20 Oct 2022
Towards Sustainable Self-supervised Learning Shanghua Gao Pan Zhou Mingg-Ming Cheng Shuicheng Yan CLL 48 7 0 20 Oct 2022
SSiT: Saliency-guided Self-supervised Image Transformer for Diabetic Retinopathy Grading Yijin Huang Junyan Lyu Pujin Cheng Roger Tam Xiaoying Tang ViT MedIm 19 20 0 20 Oct 2022
A Unified View of Masked Image Modeling Zhiliang Peng Li Dong Hangbo Bao QiXiang Ye Furu Wei VLM 56 35 0 19 Oct 2022
Learning Self-Regularized Adversarial Views for Self-Supervised Vision Transformers Tao Tang Changlin Li Guangrun Wang Kaicheng Yu Xiaojun Chang Xiaodan Liang ViT 31 1 0 16 Oct 2022
When Adversarial Training Meets Vision Transformers: Recipes from Training to Architecture Yi Mo Dongxian Wu Yifei Wang Yiwen Guo Yisen Wang ViT 45 52 0 14 Oct 2022
Holo-Dex: Teaching Dexterity with Immersive Mixed Reality Sridhar Pandian Arunachalam Irmak Güzey Soumith Chintala Lerrel Pinto 43 68 0 12 Oct 2022
Bridging the Gap Between Vision Transformers and Convolutional Neural Networks on Small Datasets Zhiying Lu Hongtao Xie Chuanbin Liu Yongdong Zhang ViT 28 57 0 12 Oct 2022
HiCo: Hierarchical Contrastive Learning for Ultrasound Video Model Pretraining Chunhui Zhang Yixiong Chen Li Liu Qiong Liu Xiaoping Zhou VLM 45 8 0 10 Oct 2022
Env-Aware Anomaly Detection: Ignore Style Changes, Stay True to Content! Stefan Smeu Elena Burceanu Andrei Liviu Nicolicioiu Emanuela Haller 35 4 0 06 Oct 2022
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng-Zhen Zhang Yuxiao Dong Jie Tang BDL LRM 253 1,073 0 05 Oct 2022
RankMe: Assessing the downstream performance of pretrained self-supervised representations by their rank Q. Garrido Randall Balestriero Laurent Najman Yann LeCun SSL 63 73 0 05 Oct 2022
Learning Hierarchical Image Segmentation For Recognition and By Recognition Tsung-Wei Ke Sangwoo Mo Stella X. Yu VLM 37 9 0 01 Oct 2022
Slimmable Networks for Contrastive Self-supervised Learning Shuai Zhao Xiaohan Wang Linchao Zhu Yi Yang 35 1 0 30 Sep 2022
Understanding Collapse in Non-Contrastive Siamese Representation Learning Alexander C. Li Alexei A. Efros Deepak Pathak SSL 55 33 0 29 Sep 2022
Bridging the Gap to Real-World Object-Centric Learning Maximilian Seitzer Max Horn Andrii Zadaianchuk Dominik Zietlow Tianjun Xiao ... Tong He Zheng-Wei Zhang Bernhard Schölkopf Thomas Brox Francesco Locatello OCL 45 140 0 29 Sep 2022
Audio Barlow Twins: Self-Supervised Audio Representation Learning Jonah Anton H. Coppock Pancham Shukla Bjorn W. Schuller BDL SSL 43 8 0 28 Sep 2022
Multimodal Channel-Mixing: Channel and Spatial Masked AutoEncoder on Facial Action Unit Detection Xiang Zhang Huiyuan Yang Taoyue Wang Xiaotian Li L. Yin 21 7 0 25 Sep 2022
Pretraining the Vision Transformer using self-supervised methods for vision based Deep Reinforcement Learning Manuel Goulão Arlindo L. Oliveira ViT 41 6 0 22 Sep 2022
Lightweight Transformers for Human Activity Recognition on Mobile Devices Sannara Ek François Portet P. Lalanda 34 28 0 22 Sep 2022
A Simple and Powerful Global Optimization for Unsupervised Video Object Segmentation Georgy Ponimatkin Nermin Samet Yanghua Xiao Yuming Du Renaud Marlet Vincent Lepetit VOS 74 20 0 19 Sep 2022
PointCAT: Contrastive Adversarial Training for Robust Point Cloud Recognition Qidong Huang Xiaoyi Dong Dongdong Chen Hang Zhou Weiming Zhang Kui Zhang Gang Hua Nenghai Yu 3DPC 32 12 0 16 Sep 2022
Enhance the Visual Representation via Discrete Adversarial Training Xiaofeng Mao YueFeng Chen Ranjie Duan Yao Zhu Gege Qi Shaokai Ye Xiaodan Li Rong Zhang Hui Xue 44 31 0 16 Sep 2022
Exploring Target Representations for Masked Autoencoders Xingbin Liu Jinghao Zhou Tao Kong Xianming Lin Rongrong Ji 100 50 0 08 Sep 2022
Design of the topology for contrastive visual-textual alignment Zhun Sun 30 1 0 05 Sep 2022
TokenCut: Segmenting Objects in Images and Videos with Self-supervised Transformer and Normalized Cut Yangtao Wang Xiaoke Shen Yuan. Yuan Yuming Du Maomao Li S. Hu James L. Crowley Dominique Vaufreydaz VOS ViT 27 78 0 01 Sep 2022
CMD: Self-supervised 3D Action Representation Learning with Cross-modal Mutual Distillation Yunyao Mao Wen-gang Zhou Zhenbo Lu Jiajun Deng Houqiang Li 30 38 0 26 Aug 2022
MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretraining Xiaoyi Dong Jianmin Bao Yinglin Zheng Ting Zhang Dongdong Chen ... Weiming Zhang Lu Yuan Dong Chen Fang Wen Nenghai Yu CLIP VLM 54 158 0 25 Aug 2022
Masked Autoencoders Enable Efficient Knowledge Distillers Yutong Bai Zeyu Wang Junfei Xiao Chen Wei Huiyu Wang Alan Yuille Yuyin Zhou Cihang Xie CLL 32 39 0 25 Aug 2022
Refine and Represent: Region-to-Object Representation Learning Akash Gokul Konstantinos Kallidromitis Shufang Li Yu Kato Kazuki Kozuka Trevor Darrell Colorado Reed SSeg 31 5 0 25 Aug 2022
Federated Self-Supervised Contrastive Learning and Masked Autoencoder for Dermatological Disease Diagnosis Yawen Wu Dewen Zeng Zhepeng Wang Yi Sheng Lei Yang A. James Yiyu Shi Jingtong Hu 20 7 0 24 Aug 2022
BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizers Zhiliang Peng Li Dong Hangbo Bao QiXiang Ye Furu Wei 29 306 0 12 Aug 2022
RenyiCL: Contrastive Representation Learning with Skew Renyi Divergence Kyungmin Lee Jinwoo Shin SSL DRL 31 10 0 12 Aug 2022
On the Pros and Cons of Momentum Encoder in Self-Supervised Visual Representation Learning T. Pham Chaoning Zhang Axi Niu Kang Zhang Chang D. Yoo 36 11 0 11 Aug 2022