Emerging Properties in Self-Supervised Vision Transformers

29 April 2021

Papers citing "Emerging Properties in Self-Supervised Vision Transformers"

50 / 1,385 papers shown

Title
The SSL Interplay: Augmentations, Inductive Bias, and Generalization Vivien A. Cabannes B. Kiani Randall Balestriero Yann LeCun A. Bietti SSL 24 31 0 06 Feb 2023
Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretraining Zekun Qi Runpei Dong Guo Fan Zheng Ge Xiangyu Zhang Kaisheng Ma Li Yi 43 118 0 05 Feb 2023
Self-Supervised Transformer Architecture for Change Detection in Radio Access Networks Igor Kozlov D. Rivkin Wei-Di Chang Di Wu Xue Liu Gregory Dudek AI4TS 13 1 0 03 Feb 2023
Show me your NFT and I tell you how it will perform: Multimodal representation learning for NFT selling price prediction Davide Costa Lucio La Cava Andrea Tagarelli 29 22 0 03 Feb 2023
Energy-Inspired Self-Supervised Pretraining for Vision Models Ze Wang Jiang Wang Zicheng Liu Qiang Qiu 26 8 0 02 Feb 2023
Vision Transformer-based Feature Extraction for Generalized Zero-Shot Learning Jiseob Kim Kyuhong Shim Junhan Kim B. Shim ViT 32 12 0 02 Feb 2023
Real Estate Property Valuation using Self-Supervised Vision Transformers Mahdieh Yazdani M. Raissi ViT 18 1 0 31 Jan 2023
ViewCo: Discovering Text-Supervised Segmentation Masks via Multi-View Semantic Consistency Pengzhen Ren Changlin Li Hang Xu Yi Zhu Guangrun Wang Jian-zhuo Liu Xiaojun Chang Xiaodan Liang 42 43 0 31 Jan 2023
Inference Time Evidences of Adversarial Attacks for Forensic on Transformers Hugo Lemarchant Liang Li Yiming Qian Yuta Nakashima Hajime Nagahara ViT AAML 43 0 0 31 Jan 2023
Continuous Spatiotemporal Transformers Antonio H. O. Fonseca E. Zappala J. O. Caro David van Dijk 26 7 0 31 Jan 2023
Don't Play Favorites: Minority Guidance for Diffusion Models Soo Bin Um Suhyeon Lee Jong Chul Ye DiffM 31 22 0 29 Jan 2023
A Closer Look at Few-shot Classification Again Xu Luo Hao Wu Ji Zhang Lianli Gao Jing Xu Jingkuan Song 24 48 0 28 Jan 2023
Deciphering the Projection Head: Representation Evaluation Self-supervised Learning Jiajun Ma Tianyang Hu Wei Cao 23 8 0 28 Jan 2023
Cross-Architectural Positive Pairs improve the effectiveness of Self-Supervised Learning P. Singh Jacopo Cirrone SSL 48 0 0 27 Jan 2023
Understanding Self-Supervised Pretraining with Part-Aware Representation Learning Jie Zhu Jiyang Qi Mingyu Ding Xiaokang Chen Ping Luo Xinggang Wang Wenyu Liu Leye Wang Jingdong Wang SSL 38 8 0 27 Jan 2023
Cut and Learn for Unsupervised Object Detection and Instance Segmentation Xudong Wang Rohit Girdhar Stella X. Yu Ishan Misra VLM 64 162 0 26 Jan 2023
Discovering and Mitigating Visual Biases through Keyword Explanation Younghyun Kim Sangwoo Mo Minkyu Kim Kyungmin Lee Jaeho Lee Jinwoo Shin 40 33 0 26 Jan 2023
Self-Supervised Curricular Deep Learning for Chest X-Ray Image Classification Iván de Andrés Tamé Kirill Sirotkin Pablo Carballeira Marcos Escudero-Viñolo 31 2 0 25 Jan 2023
RangeViT: Towards Vision Transformers for 3D Semantic Segmentation in Autonomous Driving Angelika Ando Spyros Gidaris Andrei Bursuc Gilles Puy Alexandre Boulch Renaud Marlet ViT 3DPC 26 72 0 24 Jan 2023
StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesis Axel Sauer Tero Karras S. Laine Andreas Geiger Timo Aila 37 209 0 23 Jan 2023
Open-Set Likelihood Maximization for Few-Shot Learning Malik Boudiaf Etienne Bennequin Myriam Tami Antoine Toubhans Pablo Piantanida C´eline Hudelot Ismail Ben Ayed BDL 34 10 0 20 Jan 2023
PIRLNav: Pretraining with Imitation and RL Finetuning for ObjectNav Ram Ramrakhya Dhruv Batra Erik Wijmans Abhishek Das OffRL 31 53 0 18 Jan 2023
RILS: Masked Visual Reconstruction in Language Semantic Space Shusheng Yang Yixiao Ge Kun Yi Dian Li Ying Shan Xiaohu Qie Xinggang Wang CLIP 43 11 0 17 Jan 2023
Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Models Zhiqiu Lin Samuel Yu Zhiyi Kuang Deepak Pathak Deva Ramana VLM 20 100 0 16 Jan 2023
Learning to Summarize Videos by Contrasting Clips Ivan Sosnovik A. Moskalev Cees Kaandorp A. Smeulders 19 0 0 12 Jan 2023
SemPPL: Predicting pseudo-labels for better contrastive representations Matko Bovsnjak Pierre Harvey Richemond Nenad Tomašev Florian Strub Jacob Walker Felix Hill Lars Buesing Razvan Pascanu Charles Blundell Jovana Mitrović SSL VLM 49 9 0 12 Jan 2023
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks Xinsong Zhang Yan Zeng Jipeng Zhang Hang Li VLM AI4CE LRM 22 17 0 12 Jan 2023
Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling Keyu Tian Yi-Xin Jiang Qishuai Diao Chen Lin Liwei Wang Zehuan Yuan 36 100 0 09 Jan 2023
Learning the Relation between Similarity Loss and Clustering Loss in Self-Supervised Learning Jidong Ge YuXiang Liu Jie Gui Lanting Fang Ming Lin James T. Kwok LiGuo Huang B. Luo SSL 22 5 0 08 Jan 2023
Exploring Efficient Few-shot Adaptation for Vision Transformers C. Xu Siqian Yang Yabiao Wang Zhanxiong Wang Yanwei Fu Xiangyang Xue 35 16 0 06 Jan 2023
CiT: Curation in Training for Effective Vision-Language Data Hu Xu Saining Xie Po-Yao (Bernie) Huang Licheng Yu Russ Howes Gargi Ghosh Luke Zettlemoyer Christoph Feichtenhofer VLM DiffM 33 25 0 05 Jan 2023
Skip-Attention: Improving Vision Transformers by Paying Less Attention Shashanka Venkataramanan Amir Ghodrati Yuki M. Asano Fatih Porikli A. Habibian ViT 23 25 0 05 Jan 2023
Learning by Sorting: Self-supervised Learning with Group Ordering Constraints Nina Shvetsova Felix Petersen Anna Kukleva Bernt Schiele Hilde Kuehne SSL 49 12 0 05 Jan 2023
Event Camera Data Pre-training Yan Yang Liyuan Pan Liu Liu 28 32 0 05 Jan 2023
PACO: Parts and Attributes of Common Objects Vignesh Ramanathan Anmol Kalia Vladan Petrovic Yiqian Wen Baixue Zheng ... Abhishek Kadian Amir Mousavi Yi-Zhe Song Abhimanyu Dubey D. Mahajan VLM 30 95 0 04 Jan 2023
MoBYv2AL: Self-supervised Active Learning for Image Classification Razvan Caramalau Binod Bhattarai Danail Stoyanov Tae-Kyun Kim SSL 35 7 0 04 Jan 2023
Towards the Identifiability in Noisy Label Learning: A Multinomial Mixture Approach Cuong C. Nguyen Thanh-Toan Do G. Carneiro NoLa 42 0 0 04 Jan 2023
Ego-Only: Egocentric Action Detection without Exocentric Transferring Huiyu Wang Mitesh Singh Lorenzo Torresani EgoV 77 24 0 03 Jan 2023
TinyMIM: An Empirical Study of Distilling MIM Pre-trained Models Sucheng Ren Fangyun Wei Zheng-Wei Zhang Han Hu 42 35 0 03 Jan 2023
A New Perspective to Boost Vision Transformer for Medical Image Classification Yuexiang Li Yawen Huang Nanjun He Kai Ma Yefeng Zheng ViT MedIm 23 3 0 03 Jan 2023
ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders Sanghyun Woo Shoubhik Debnath Ronghang Hu Xinlei Chen Zhuang Liu In So Kweon Saining Xie SyDa 85 729 0 02 Jan 2023
Deep Learning Technique for Human Parsing: A Survey and Outlook Lu Yang Wenhe Jia Shane Li Q. Song ViT 55 17 0 01 Jan 2023
Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning Colorado Reed Ritwik Gupta Shufan Li S. Brockman Christopher Funk Brian Clipp Kurt Keutzer Salvatore Candido M. Uyttendaele Trevor Darrell 121 170 0 30 Dec 2022
Improving Visual Representation Learning through Perceptual Understanding Samyakh Tukra Frederick Hoffman Ken Chatfield 33 5 0 30 Dec 2022
Swin MAE: Masked Autoencoders for Small Datasets Zián Xu Yin Dai Fayu Liu Weibin Chen Yue Liu Li-Li Shi Sheng Liu Yuhang Zhou SyDa MedIm ViT 36 28 0 28 Dec 2022
MVTN: Learning Multi-View Transformations for 3D Understanding Abdullah Hamdi Faisal AlZahrani Silvio Giancola Guohao Li 3DPC 3DV 34 6 0 27 Dec 2022
GEDI: GEnerative and DIscriminative Training for Self-Supervised Learning Emanuele Sansone Robin Manhaeve SSL 20 9 0 27 Dec 2022
Metadata-guided Consistency Learning for High Content Images Johan Fredin Haslum Christos Matsoukas Karl‐Johan Leuchowius Erik Mullers Kevin Smith 30 8 0 22 Dec 2022
A Mathematical Framework for Learning Probability Distributions Hongkang Yang 31 7 0 22 Dec 2022
Similarity Contrastive Estimation for Image and Video Soft Contrastive Self-Supervised Learning J. Denize Jaonary Rabarisoa Astrid Orcesi Romain Hérault SSL 19 6 0 21 Dec 2022