O-ViT: Orthogonal Vision Transformer

O-ViT: Orthogonal Vision Transformer

28 January 2022

Papers citing "O-ViT: Orthogonal Vision Transformer"

17 / 17 papers shown

Title
Attention Approximates Sparse Distributed Memory Trenton Bricken Cengiz Pehlevan 57 34 0 10 Nov 2021
Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges M. Bronstein Joan Bruna Taco S. Cohen Petar Velivcković GNN 267 1,137 0 27 Apr 2021
Going deeper with Image Transformers Hugo Touvron Matthieu Cord Alexandre Sablayrolles Gabriel Synnaeve Hervé Jégou ViT 112 998 0 31 Mar 2021
DeepViT: Towards Deeper Vision Transformer Daquan Zhou Bingyi Kang Xiaojie Jin Linjie Yang Xiaochen Lian Zihang Jiang Qibin Hou Jiashi Feng ViT 60 517 0 22 Mar 2021
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet Li-xin Yuan Yunpeng Chen Tao Wang Weihao Yu Yujun Shi Zihang Jiang Francis E. H. Tay Jiashi Feng Shuicheng Yan ViT 84 1,918 0 28 Jan 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 260 6,657 0 23 Dec 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 251 40,217 0 22 Oct 2020
On Translation Invariance in CNNs: Convolutional Layers can Exploit Absolute Spatial Location O. Kayhan Jan van Gemert 278 234 0 16 Mar 2020
Orthogonal Convolutional Neural Networks Jiayun Wang Yubei Chen Rudrasis Chakraborty Stella X. Yu 42 188 0 27 Nov 2019
TENER: Adapting Transformer Encoder for Named Entity Recognition Hang Yan Bocao Deng Xiaonan Li Xipeng Qiu 32 270 0 10 Nov 2019
Trivializations for Gradient-Based Optimization on Manifolds Mario Lezcano Casado 82 124 0 20 Sep 2019
Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Group Mario Lezcano Casado David Martínez-Rubio 38 195 0 24 Jan 2019
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 354 129,831 0 12 Jun 2017
Building Deep Networks on Grassmann Manifolds Zhiwu Huang Jiqing Wu Luc Van Gool 39 160 0 17 Nov 2016
A Riemannian Network for SPD Matrix Learning Zhiwu Huang Luc Van Gool 40 393 0 15 Aug 2016
Unitary Evolution Recurrent Neural Networks Martín Arjovsky Amar Shah Yoshua Bengio ODL 38 763 0 20 Nov 2015
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks Andrew M. Saxe James L. McClelland Surya Ganguli ODL 100 1,830 0 20 Dec 2013