Emerging Properties in Self-Supervised Vision Transformers

29 April 2021

Papers citing "Emerging Properties in Self-Supervised Vision Transformers"

49 / 1,249 papers shown

Title
Vision Transformer for Learning Driving Policies in Complex Multi-Agent Environments E. Kargar Ville Kyrki ViT 26 13 0 14 Sep 2021
Self supervised learning improves dMMR/MSI detection from histology slides across multiple cancers C. Saillard Olivier Dehaene Tanguy Marchand O. Moindrot A. Kamoun B. Schmauch S. Jégou 38 39 0 13 Sep 2021
Scaled ReLU Matters for Training Vision Transformers Pichao Wang Xue Wang Haowen Luo Jingkai Zhou Zhipeng Zhou Fan Wang Hao Li R. L. Jin 19 41 0 08 Sep 2021
The Functional Correspondence Problem Zihang Lai Senthil Purushwalkam Abhinav Gupta 38 13 0 02 Sep 2021
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision Zirui Wang Jiahui Yu Adams Wei Yu Zihang Dai Yulia Tsvetkov Yuan Cao VLM MLLM 51 779 0 24 Aug 2021
How Transferable Are Self-supervised Features in Medical Image Classification Tasks? T. Truong Sadegh Mohammadi Matthias Lenga 42 45 0 23 Aug 2021
Is it Time to Replace CNNs with Transformers for Medical Images? Christos Matsoukas Johan Fredin Haslum Magnus P Soderberg Kevin Smith ViT OOD MedIm 11 146 0 20 Aug 2021
Do Vision Transformers See Like Convolutional Neural Networks? M. Raghu Thomas Unterthiner Simon Kornblith Chiyuan Zhang Alexey Dosovitskiy ViT 67 924 0 19 Aug 2021
A Low Rank Promoting Prior for Unsupervised Contrastive Learning Yu Wang Jingyang Lin Qi Cai Yingwei Pan Ting Yao Hongyang Chao Tao Mei SSL 35 16 0 05 Aug 2021
Focal Self-attention for Local-Global Interactions in Vision Transformers Jianwei Yang Chunyuan Li Pengchuan Zhang Xiyang Dai Bin Xiao Lu Yuan Jianfeng Gao ViT 42 428 0 01 Jul 2021
IA-RED $^2$ : Interpretability-Aware Redundancy Reduction for Vision Transformers Bowen Pan Rameswar Panda Yi Ding Zhangyang Wang Rogerio Feris A. Oliva VLM ViT 39 153 0 23 Jun 2021
Encoder-Decoder Architectures for Clinically Relevant Coronary Artery Segmentation Joao Lourencco Silva M. Menezes T. Rodrigues B. Silva F. Pinto Arlindo L. Oliveira MedIm 31 17 0 21 Jun 2021
How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers Andreas Steiner Alexander Kolesnikov Xiaohua Zhai Ross Wightman Jakob Uszkoreit Lucas Beyer ViT 54 616 0 18 Jun 2021
Efficient Self-supervised Vision Transformers for Representation Learning Chunyuan Li Jianwei Yang Pengchuan Zhang Mei Gao Bin Xiao Xiyang Dai Lu Yuan Jianfeng Gao ViT 37 209 0 17 Jun 2021
XCiT: Cross-Covariance Image Transformers Alaaeldin El-Nouby Hugo Touvron Mathilde Caron Piotr Bojanowski Matthijs Douze ... Ivan Laptev Natalia Neverova Gabriel Synnaeve Jakob Verbeek Hervé Jégou ViT 42 499 0 17 Jun 2021
The 2021 Image Similarity Dataset and Challenge Matthijs Douze Giorgos Tolias Ed Pizzi Zoe Papakipos L. Chanussot ... Maxim Maximov Laura Leal-Taixé Ismail Elezi Ondřej Chum Cristian Canton Ferrer 28 59 0 17 Jun 2021
BEiT: BERT Pre-Training of Image Transformers Hangbo Bao Li Dong Songhao Piao Furu Wei ViT 66 2,749 0 15 Jun 2021
Dynamic Distillation Network for Cross-Domain Few-Shot Recognition with Unlabeled Data Ashraful Islam Chun-Fu Chen Rameswar Panda Leonid Karlinsky Rogerio Feris Richard J. Radke 30 84 0 14 Jun 2021
Revisiting Model Stitching to Compare Neural Representations Yamini Bansal Preetum Nakkiran Boaz Barak FedML 44 105 0 14 Jun 2021
Delving Deep into the Generalization of Vision Transformers under Distribution Shifts Chongzhi Zhang Mingyuan Zhang Shanghang Zhang Daisheng Jin Qiang-feng Zhou Zhongang Cai Haiyu Zhao Xianglong Liu Ziwei Liu 21 102 0 14 Jun 2021
Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learning Liangqiong Qu Yuyin Zhou Paul Pu Liang Yingda Xia Feifei Wang Ehsan Adeli L. Fei-Fei D. Rubin FedML AI4CE 19 175 0 10 Jun 2021
MST: Masked Self-Supervised Transformer for Visual Representation Zhaowen Li Zhiyang Chen Fan Yang Wei Li Yousong Zhu ... Rui Deng Liwei Wu Rui Zhao Ming Tang Jinqiao Wang ViT 42 162 0 10 Jun 2021
Keeping Your Eye on the Ball: Trajectory Attention in Video Transformers Mandela Patrick Dylan Campbell Yuki M. Asano Ishan Misra Ishan Misra Florian Metze Christoph Feichtenhofer Andrea Vedaldi João F. Henriques 18 274 0 09 Jun 2021
Scaling Vision Transformers Xiaohua Zhai Alexander Kolesnikov N. Houlsby Lucas Beyer ViT 55 1,060 0 08 Jun 2021
Aligning Pretraining for Detection via Object-Level Contrastive Learning Fangyun Wei Yue Gao Zhirong Wu Han Hu Stephen Lin ObjD 22 144 0 04 Jun 2021
CATs: Cost Aggregation Transformers for Visual Correspondence Seokju Cho Sunghwan Hong Sangryul Jeon Yunsung Lee Kwanghoon Sohn Seungryong Kim ViT 26 86 0 04 Jun 2021
KVT: k-NN Attention for Boosting Vision Transformers Pichao Wang Xue Wang F. Wang Ming Lin Shuning Chang Hao Li R. L. Jin ViT 51 105 0 28 May 2021
Intriguing Properties of Vision Transformers Muzammal Naseer Kanchana Ranasinghe Salman Khan Munawar Hayat F. Khan Ming-Hsuan Yang ViT 265 621 0 21 May 2021
Vision Transformers are Robust Learners Sayak Paul Pin-Yu Chen ViT 28 304 0 17 May 2021
Contrastive Attraction and Contrastive Repulsion for Representation Learning Huangjie Zheng Xu Chen Jiangchao Yao Hongxia Yang Chunyuan Li Ya-Qin Zhang Hao Zhang Ivor Tsang Jingren Zhou Mingyuan Zhou SSL 42 12 0 08 May 2021
ResMLP: Feedforward networks for image classification with data-efficient training Hugo Touvron Piotr Bojanowski Mathilde Caron Matthieu Cord Alaaeldin El-Nouby ... Gautier Izacard Armand Joulin Gabriel Synnaeve Jakob Verbeek Hervé Jégou VLM 21 656 0 07 May 2021
DisCo: Remedy Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning Yuting Gao Jia-Xin Zhuang Xiaowei Guo Hao Cheng Xing Sun Ke Li Feiyue Huang 39 40 0 19 Apr 2021
SiT: Self-supervised vIsion Transformer Sara Atito Ali Ahmed Muhammad Awais J. Kittler ViT 36 139 0 08 Apr 2021
Characterizing and Improving the Robustness of Self-Supervised Learning through Background Augmentations Chaitanya K. Ryali D. Schwab Ari S. Morcos SSL 32 9 0 23 Mar 2021
UNETR: Transformers for 3D Medical Image Segmentation Ali Hatamizadeh Yucheng Tang Vishwesh Nath Dong Yang Andriy Myronenko Bennett Landman H. Roth Daguang Xu ViT MedIm 92 1,535 0 18 Mar 2021
CoDeGAN: Contrastive Disentanglement for Generative Adversarial Network Lili Pan Peijun Tang Zhiyong Chen Zenglin Xu GAN DRL 18 6 0 05 Mar 2021
Predicting post-operative right ventricular failure using video-based deep learning R. Shad Nicolas Quach R. Fong P. Kasinpila C. Bowles ... Y. Woo J. Teuteberg John P. Cunningham C. Langlotz W. Hiesinger 22 40 0 28 Feb 2021
S2-BNN: Bridging the Gap Between Self-Supervised Real and 1-bit Neural Networks via Guided Distribution Calibration Zhiqiang Shen Zechun Liu Jie Qin Lei Huang Kwang-Ting Cheng Marios Savvides UQCV SSL MQ 248 22 0 17 Feb 2021
Deep Learning with Label Differential Privacy Badih Ghazi Noah Golowich Ravi Kumar Pasin Manurangsi Chiyuan Zhang 42 144 0 11 Feb 2021
SEED: Self-supervised Distillation For Visual Representation Zhiyuan Fang Jianfeng Wang Lijuan Wang Lei Zhang Yezhou Yang Zicheng Liu SSL 242 190 0 12 Jan 2021
Transformers in Vision: A Survey Salman Khan Muzammal Naseer Munawar Hayat Syed Waqas Zamir F. Khan M. Shah ViT 227 2,430 0 04 Jan 2021
Concept Generalization in Visual Representation Learning Mert Bulent Sariyildiz Yannis Kalantidis Diane Larlus Alahari Karteek SSL 28 50 0 10 Dec 2020
Exploring Simple Siamese Representation Learning Xinlei Chen Kaiming He SSL 22 3,960 0 20 Nov 2020
BYOL works even without batch statistics Pierre Harvey Richemond Jean-Bastien Grill Florent Altché Corentin Tallec Florian Strub ... Samuel L. Smith Soham De Razvan Pascanu Bilal Piot Michal Valko SSL 250 114 0 20 Oct 2020
Meta Pseudo Labels Hieu H. Pham Zihang Dai Qizhe Xie Minh-Thang Luong Quoc V. Le VLM 262 656 0 23 Mar 2020
Improved Baselines with Momentum Contrastive Learning Xinlei Chen Haoqi Fan Ross B. Girshick Kaiming He SSL 270 3,375 0 09 Mar 2020
Boosting Self-Supervised Learning via Knowledge Transfer M. Noroozi Ananth Vinjimoor Paolo Favaro Hamed Pirsiavash SSL 209 292 0 01 May 2018
Large scale distributed neural network training through online distillation Rohan Anil Gabriel Pereyra Alexandre Passos Róbert Ormándi George E. Dahl Geoffrey E. Hinton FedML 278 404 0 09 Apr 2018
OpenNMT: Open-Source Toolkit for Neural Machine Translation Guillaume Klein Yoon Kim Yuntian Deng Jean Senellart Alexander M. Rush 262 1,896 0 10 Jan 2017