An Empirical Study of Training Self-Supervised Vision Transformers

5 April 2021

Papers citing "An Empirical Study of Training Self-Supervised Vision Transformers"

50 / 469 papers shown

Title
LDReg: Local Dimensionality Regularized Self-Supervised Learning Hanxun Huang R. Campello S. Erfani Xingjun Ma Michael E. Houle James Bailey 38 5 0 19 Jan 2024
Collaboratively Self-supervised Video Representation Learning for Action Recognition Jie Zhang Zhifan Wan Lanqing Hu Stephen Lin Shuzhe Wu Shiguang Shan TTA 67 1 0 15 Jan 2024
MIMIC: Mask Image Pre-training with Mix Contrastive Fine-tuning for Facial Expression Recognition Fan Zhang Xiaobao Guo Xiaojiang Peng Alex C. Kot 27 0 0 14 Jan 2024
A Study on Self-Supervised Pretraining for Vision Problems in Gastrointestinal Endoscopy Edward Sanderson B. Matuszewski 23 2 0 11 Jan 2024
Morphing Tokens Draw Strong Masked Image Models Taekyung Kim Byeongho Heo Dongyoon Han 54 3 0 30 Dec 2023
Learning Anatomically Consistent Embedding for Chest Radiography Ziyu Zhou Haozhe Luo Jiaxuan Pang Xiaowei Ding Michael B. Gotway Jianming Liang SSL 17 5 0 01 Dec 2023
Text Attribute Control via Closed-Loop Disentanglement Lei Sha Thomas Lukasiewicz DRL 43 2 0 01 Dec 2023
FALCON: Fairness Learning via Contrastive Attention Approach to Continual Semantic Scene Understanding Thanh-Dat Truong Utsav Prabhu Bhiksha Raj Jackson Cothren Khoa Luu CLL 40 3 0 27 Nov 2023
CUCL: Codebook for Unsupervised Continual Learning Chen Cheng Jingkuan Song Xiaosu Zhu Junchen Zhu Lianli Gao H. Shen CLL 25 2 0 25 Nov 2023
Stable Cluster Discrimination for Deep Clustering Qi Qian OOD 33 21 0 24 Nov 2023
Event Camera Data Dense Pre-training Yan Yang Liyuan Pan Liu Liu 30 4 0 20 Nov 2023
What Makes Pre-Trained Visual Representations Successful for Robust Manipulation? Kaylee Burns Zach Witzel Jubayer Ibn Hamid Tianhe Yu Chelsea Finn Karol Hausman OOD SSL 32 23 0 03 Nov 2023
COSTAR: Improved Temporal Counterfactual Estimation with Self-Supervised Learning Chuizheng Meng Yihe Dong Sercan Ö. Arik Yan Liu Tomas Pfister CML AI4TS 29 0 0 01 Nov 2023
Combating Representation Learning Disparity with Geometric Harmonization Zhihan Zhou Jiangchao Yao Feng Hong Ya Zhang Bo Han Yanfeng Wang 33 12 0 26 Oct 2023
Causal Unsupervised Semantic Segmentation Junho Kim Byung-Kwan Lee Yonghyun Ro 33 18 0 11 Oct 2023
Transferable Availability Poisoning Attacks Yiyong Liu Michael Backes Xiao Zhang AAML 21 3 0 08 Oct 2023
Enhancing Representations through Heterogeneous Self-Supervised Learning Zhongyu Li Bo-Wen Yin Yongxiang Liu Li Liu Ming-Ming Cheng SSL 28 2 0 08 Oct 2023
Small-scale proxies for large-scale Transformer training instabilities Mitchell Wortsman Peter J. Liu Lechao Xiao Katie Everett A. Alemi ... Jascha Narain Sohl-Dickstein Kelvin Xu Jaehoon Lee Justin Gilmer Simon Kornblith 40 84 0 25 Sep 2023
SCOB: Universal Text Understanding via Character-wise Supervised Contrastive Learning with Online Text Rendering for Bridging Domain Gap Daehee Kim Yoon Kim Donghyun Kim Yumin Lim Geewook Kim Taeho Kil 34 3 0 21 Sep 2023
DimCL: Dimensional Contrastive Learning For Improving Self-Supervised Learning Thanh Nguyen T. Pham Chaoning Zhang Tung M. Luu Thang Vu Chang D. Yoo 27 9 0 21 Sep 2023
SCT: A Simple Baseline for Parameter-Efficient Fine-Tuning via Salient Channels Henry Hengyuan Zhao Pichao Wang Yuyang Zhao Hao Luo F. Wang Mike Zheng Shou ViT 37 14 0 15 Sep 2023
$360$^\circ$ from a Single Camera: A Few-Shot Approach for LiDAR Segmentation$ 360 $^\circ$ from a Single Camera: A Few-Shot Approach for LiDAR Segmentation Laurenz Reichardt Nikolas Ebert Oliver Wasenmüller 3DPC 51 10 0 12 Sep 2023
Dynamic Visual Prompt Tuning for Parameter Efficient Transfer Learning Chunqing Ruan Hongjian Wang VLM VPVLM 32 1 0 12 Sep 2023
Representation Synthesis by Probabilistic Many-Valued Logic Operation in Self-Supervised Learning Hiroki Nakamura Masashi Okada T. Taniguchi SSL NAI 31 0 0 08 Sep 2023
Efficient Model Personalization in Federated Learning via Client-Specific Prompt Generation Fu-En Yang Chien-Yi Wang Yu-Chiang Frank Wang VLM FedML 34 59 0 29 Aug 2023
Forensic Histopathological Recognition via a Context-Aware MIL Network Powered by Self-Supervised Contrastive Learning Chen Shen Jun Zhang Xinggong Liang Zeyi Hao Ke Li Fan Wang Zhenyuan Wang C. Lian 22 2 0 27 Aug 2023
Masked Feature Modelling: Feature Masking for the Unsupervised Pre-training of a Graph Attention Network Block for Bottom-up Video Event Recognition Dimitrios Daskalakis Nikolaos Gkalelis Vasileios Mezaris 38 0 0 24 Aug 2023
Masked Momentum Contrastive Learning for Zero-shot Semantic Understanding Jiantao Wu Shentong Mo Muhammad Awais Sara Atito Zhenhua Feng J. Kittler VLM 30 4 0 22 Aug 2023
Jumping through Local Minima: Quantization in the Loss Landscape of Vision Transformers N. Frumkin Dibakar Gope Diana Marculescu MQ 41 16 0 21 Aug 2023
CoNe: Contrast Your Neighbours for Supervised Image Classification Mingkai Zheng Shan You Lang Huang Xiu Su Fei Wang Chao Qian Xiaogang Wang Chang Xu VLM 26 0 0 21 Aug 2023
SSTFormer: Bridging Spiking Neural Network and Memory Support Transformer for Frame-Event based Recognition Tianlin Li Zong-Yao Wu Yao Rong Lin Zhu Bowei Jiang Jin Tang Yonghong Tian ViT 77 17 0 08 Aug 2023
Exploring Visual Pre-training for Robot Manipulation: Datasets, Models and Methods Ya Jing Xuelin Zhu Xingbin Liu Qie Sima Taozheng Yang Yunhai Feng Tao Kong LM&Ro 45 16 0 07 Aug 2023
Stochastic positional embeddings improve masked image modeling Amir Bar Florian Bordes Assaf Shocher Mahmoud Assran Pascal Vincent Nicolas Ballas Trevor Darrell Amir Globerson Yann LeCun 36 3 0 31 Jul 2023
MC-JEPA: A Joint-Embedding Predictive Architecture for Self-Supervised Learning of Motion and Content Features Adrien Bardes Jean Ponce Yann LeCun MDE 39 25 0 24 Jul 2023
PRIOR: Prototype Representation Joint Learning from Medical Images and Reports Pujin Cheng Li Lin Junyan Lyu Yijin Huang Wenhan Luo Xiaoying Tang MedIm 37 45 0 24 Jul 2023
Why Does Little Robustness Help? Understanding and Improving Adversarial Transferability from Surrogate Training Yechao Zhang Shengshan Hu Leo Yu Zhang Junyu Shi Minghui Li Xiaogeng Liu Wei Wan Hai Jin AAML 22 21 0 15 Jul 2023
Inter-Instance Similarity Modeling for Contrastive Learning Chen Shen Dawei Liu H. Tang Zhe Qu Jianxin Wang SSL 31 4 0 21 Jun 2023
Understanding Contrastive Learning Through the Lens of Margins Daniel Rho Taesoo Kim Sooill Park Jaehyun Park Jaehan Park SSL 32 2 0 20 Jun 2023
ExpPoint-MAE: Better interpretability and performance for self-supervised point cloud transformers Ioannis Romanelis Vlassis Fotis Konstantinos Moustakas Adrian Munteanu ViT 3DPC 23 4 0 19 Jun 2023
Learning to Mask and Permute Visual Tokens for Vision Transformer Pre-Training Lorenzo Baraldi Roberto Amoroso Marcella Cornia Lorenzo Baraldi Andrea Pilzer Rita Cucchiara 38 2 0 12 Jun 2023
Joint Channel Estimation and Feedback with Masked Token Transformers in Massive MIMO Systems Mingming Zhao Lin Liu Lifu Liu Mengke Li Qing-min Tian 16 0 0 08 Jun 2023
Object-Centric Learning for Real-World Videos by Predicting Temporal Feature Similarities Andrii Zadaianchuk Maximilian Seitzer Georg Martius OCL 23 39 0 07 Jun 2023
Context-Aware Self-Supervised Learning of Whole Slide Images Milan Aryal Nasim Yahya Soltani 22 5 0 07 Jun 2023
VIPriors 3: Visual Inductive Priors for Data-Efficient Deep Learning Challenges Robert-Jan Bruintjes A. Lengyel Marcos Baptista-Rios O. Kayhan Davide Zambrano Nergis Tomen Jan van Gemert 25 9 0 31 May 2023
Instance-based Max-margin for Practical Few-shot Recognition Minghao Fu Kevin Zhu Jianxin Wu 35 2 0 27 May 2023
Investigating how ReLU-networks encode symmetries Georg Bökman Fredrik Kahl 29 6 0 26 May 2023
Image as First-Order Norm+Linear Autoregression: Unveiling Mathematical Invariance Yinpeng Chen Xiyang Dai Dongdong Chen Mengchen Liu Lu Yuan Zicheng Liu Youzuo Lin 26 2 0 25 May 2023
Medical supervised masked autoencoders: Crafting a better masking strategy and efficient fine-tuning schedule for medical image classification Jia-ju Mao Shu-Hua Guo Yuan Chang Xuesong Yin Binling Nie 28 2 0 10 May 2023
What Do Self-Supervised Vision Transformers Learn? Namuk Park Wonjae Kim Byeongho Heo Taekyung Kim Sangdoo Yun SSL 71 76 1 01 May 2023
LEMaRT: Label-Efficient Masked Region Transform for Image Harmonization Sheng Liu C. P. Huynh Congmin Chen Maxim Arap Raffay Hamid 33 19 0 25 Apr 2023