Emerging Properties in Self-Supervised Vision Transformers

29 April 2021

Papers citing "Emerging Properties in Self-Supervised Vision Transformers"

42 / 1,287 papers shown

Title
A Low Rank Promoting Prior for Unsupervised Contrastive Learning Yu Wang Jingyang Lin Qi Cai Yingwei Pan Ting Yao Hongyang Chao Tao Mei SSL 35 16 0 05 Aug 2021
Focal Self-attention for Local-Global Interactions in Vision Transformers Jianwei Yang Chunyuan Li Pengchuan Zhang Xiyang Dai Bin Xiao Lu Yuan Jianfeng Gao ViT 42 428 0 01 Jul 2021
IA-RED $^2$ : Interpretability-Aware Redundancy Reduction for Vision Transformers Bowen Pan Yikang Shen Yi Ding Zhangyang Wang Rogerio Feris A. Oliva VLM ViT 39 153 0 23 Jun 2021
Encoder-Decoder Architectures for Clinically Relevant Coronary Artery Segmentation Joao Lourencco Silva M. Menezes T. Rodrigues B. Silva F. Pinto Arlindo L. Oliveira MedIm 31 17 0 21 Jun 2021
How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers Andreas Steiner Alexander Kolesnikov Xiaohua Zhai Ross Wightman Jakob Uszkoreit Lucas Beyer ViT 66 616 0 18 Jun 2021
Efficient Self-supervised Vision Transformers for Representation Learning Chunyuan Li Jianwei Yang Pengchuan Zhang Mei Gao Bin Xiao Xiyang Dai Lu Yuan Jianfeng Gao ViT 37 209 0 17 Jun 2021
XCiT: Cross-Covariance Image Transformers Alaaeldin El-Nouby Hugo Touvron Mathilde Caron Piotr Bojanowski Matthijs Douze ... Ivan Laptev Natalia Neverova Gabriel Synnaeve Jakob Verbeek Hervé Jégou ViT 42 499 0 17 Jun 2021
The 2021 Image Similarity Dataset and Challenge Matthijs Douze Giorgos Tolias Ed Pizzi Zoe Papakipos L. Chanussot ... Maxim Maximov Laura Leal-Taixé Ismail Elezi Ondřej Chum Cristian Canton Ferrer 28 59 0 17 Jun 2021
BEiT: BERT Pre-Training of Image Transformers Hangbo Bao Li Dong Songhao Piao Furu Wei ViT 68 2,749 0 15 Jun 2021
Dynamic Distillation Network for Cross-Domain Few-Shot Recognition with Unlabeled Data Ashraful Islam Chun-Fu Chen Yikang Shen Leonid Karlinsky Rogerio Feris Richard J. Radke 30 84 0 14 Jun 2021
Revisiting Model Stitching to Compare Neural Representations Yamini Bansal Preetum Nakkiran Boaz Barak FedML 44 105 0 14 Jun 2021
Delving Deep into the Generalization of Vision Transformers under Distribution Shifts Chongzhi Zhang Mingyuan Zhang Shanghang Zhang Daisheng Jin Qiang-feng Zhou Zhongang Cai Haiyu Zhao Xianglong Liu Ziwei Liu 21 102 0 14 Jun 2021
Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learning Liangqiong Qu Yuyin Zhou Paul Pu Liang Yingda Xia Feifei Wang Ehsan Adeli L. Fei-Fei D. Rubin FedML AI4CE 19 175 0 10 Jun 2021
MST: Masked Self-Supervised Transformer for Visual Representation Zhaowen Li Zhiyang Chen Fan Yang Wei Li Yousong Zhu ... Rui Deng Liwei Wu Rui Zhao Ming Tang Jinqiao Wang ViT 42 162 0 10 Jun 2021
Keeping Your Eye on the Ball: Trajectory Attention in Video Transformers Mandela Patrick Dylan Campbell Yuki M. Asano Ishan Misra Ishan Misra Florian Metze Christoph Feichtenhofer Andrea Vedaldi João F. Henriques 18 274 0 09 Jun 2021
Scaling Vision Transformers Xiaohua Zhai Alexander Kolesnikov N. Houlsby Lucas Beyer ViT 73 1,060 0 08 Jun 2021
Aligning Pretraining for Detection via Object-Level Contrastive Learning Fangyun Wei Yue Gao Zhirong Wu Han Hu Stephen Lin ObjD 22 144 0 04 Jun 2021
CATs: Cost Aggregation Transformers for Visual Correspondence Seokju Cho Sunghwan Hong Sangryul Jeon Yunsung Lee Kwanghoon Sohn Seungryong Kim ViT 26 86 0 04 Jun 2021
KVT: k-NN Attention for Boosting Vision Transformers Pichao Wang Xue Wang F. Wang Ming Lin Shuning Chang Hao Li R. L. Jin ViT 51 105 0 28 May 2021
Intriguing Properties of Vision Transformers Muzammal Naseer Kanchana Ranasinghe Salman Khan Munawar Hayat Fahad Shahbaz Khan Ming-Hsuan Yang ViT 265 621 0 21 May 2021
Vision Transformers are Robust Learners Sayak Paul Pin-Yu Chen ViT 28 307 0 17 May 2021
Contrastive Attraction and Contrastive Repulsion for Representation Learning Huangjie Zheng Xu Chen Jiangchao Yao Hongxia Yang Chunyuan Li Ya Zhang Hao Zhang Ivor Tsang Jingren Zhou Mingyuan Zhou SSL 42 12 0 08 May 2021
ResMLP: Feedforward networks for image classification with data-efficient training Hugo Touvron Piotr Bojanowski Mathilde Caron Matthieu Cord Alaaeldin El-Nouby ... Gautier Izacard Armand Joulin Gabriel Synnaeve Jakob Verbeek Hervé Jégou VLM 30 656 0 07 May 2021
If your data distribution shifts, use self-learning E. Rusak Steffen Schneider George Pachitariu L. Eck Peter V. Gehler Oliver Bringmann Wieland Brendel Matthias Bethge VLM OOD TTA 79 29 0 27 Apr 2021
DisCo: Remedy Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning Yuting Gao Jia-Xin Zhuang Xiaowei Guo Hao Cheng Xing Sun Ke Li Feiyue Huang 39 40 0 19 Apr 2021
SiT: Self-supervised vIsion Transformer Sara Atito Ali Ahmed Muhammad Awais J. Kittler ViT 39 139 0 08 Apr 2021
Characterizing and Improving the Robustness of Self-Supervised Learning through Background Augmentations Chaitanya K. Ryali D. Schwab Ari S. Morcos SSL 32 9 0 23 Mar 2021
UNETR: Transformers for 3D Medical Image Segmentation Ali Hatamizadeh Yucheng Tang Vishwesh Nath Dong Yang Andriy Myronenko Bennett Landman H. Roth Daguang Xu ViT MedIm 95 1,535 0 18 Mar 2021
CoDeGAN: Contrastive Disentanglement for Generative Adversarial Network Lili Pan Peijun Tang Zhiyong Chen Zenglin Xu GAN DRL 18 6 0 05 Mar 2021
Predicting post-operative right ventricular failure using video-based deep learning R. Shad Nicolas Quach R. Fong P. Kasinpila C. Bowles ... Y. Woo J. Teuteberg John P. Cunningham C. Langlotz W. Hiesinger 22 40 0 28 Feb 2021
S2-BNN: Bridging the Gap Between Self-Supervised Real and 1-bit Neural Networks via Guided Distribution Calibration Zhiqiang Shen Zechun Liu Jie Qin Lei Huang Kwang-Ting Cheng Marios Savvides UQCV SSL MQ 248 22 0 17 Feb 2021
Deep Learning with Label Differential Privacy Badih Ghazi Noah Golowich Ravi Kumar Pasin Manurangsi Chiyuan Zhang 42 144 0 11 Feb 2021
SEED: Self-supervised Distillation For Visual Representation Zhiyuan Fang Jianfeng Wang Lijuan Wang Lei Zhang Yezhou Yang Zicheng Liu SSL 245 190 0 12 Jan 2021
Transformers in Vision: A Survey Salman Khan Muzammal Naseer Munawar Hayat Syed Waqas Zamir Fahad Shahbaz Khan M. Shah ViT 227 2,431 0 04 Jan 2021
Concept Generalization in Visual Representation Learning Mert Bulent Sariyildiz Yannis Kalantidis Diane Larlus Alahari Karteek SSL 28 50 0 10 Dec 2020
Exploring Simple Siamese Representation Learning Xinlei Chen Kaiming He SSL 31 3,960 0 20 Nov 2020
BYOL works even without batch statistics Pierre Harvey Richemond Jean-Bastien Grill Florent Altché Corentin Tallec Florian Strub ... Samuel L. Smith Soham De Razvan Pascanu Bilal Piot Michal Valko SSL 250 114 0 20 Oct 2020
Meta Pseudo Labels Hieu H. Pham Zihang Dai Qizhe Xie Minh-Thang Luong Quoc V. Le VLM 262 656 0 23 Mar 2020
Improved Baselines with Momentum Contrastive Learning Xinlei Chen Haoqi Fan Ross B. Girshick Kaiming He SSL 273 3,375 0 09 Mar 2020
Boosting Self-Supervised Learning via Knowledge Transfer M. Noroozi Ananth Vinjimoor Paolo Favaro Hamed Pirsiavash SSL 212 292 0 01 May 2018
Large scale distributed neural network training through online distillation Rohan Anil Gabriel Pereyra Alexandre Passos Róbert Ormándi George E. Dahl Geoffrey E. Hinton FedML 278 404 0 09 Apr 2018
OpenNMT: Open-Source Toolkit for Neural Machine Translation Guillaume Klein Yoon Kim Yuntian Deng Jean Senellart Alexander M. Rush 273 1,896 0 10 Jan 2017