Horizontal and Vertical Attention in Transformers

10 July 2022

Papers citing "Horizontal and Vertical Attention in Transformers"

26 / 26 papers shown

Title
Efficient Training of Visual Transformers with Small Datasets Yahui Liu E. Sangineto Wei Bi N. Sebe Bruno Lepri Marco De Nadai ViT 68 173 0 07 Jun 2021
CvT: Introducing Convolutions to Vision Transformers Haiping Wu Bin Xiao Noel Codella Mengchen Liu Xiyang Dai Lu Yuan Lei Zhang ViT 154 1,917 0 29 Mar 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng Zhang Stephen Lin B. Guo ViT 465 21,566 0 25 Mar 2021
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet Li-xin Yuan Yunpeng Chen Tao Wang Weihao Yu Yujun Shi Zihang Jiang Francis E. H. Tay Jiashi Feng Shuicheng Yan ViT 141 1,942 0 28 Jan 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 389 6,802 0 23 Dec 2020
Dual Attention on Pyramid Feature Maps for Image Captioning Litao Yu Jian Zhang Qiang Wu 89 50 0 02 Nov 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 670 41,430 0 22 Oct 2020
Linformer: Self-Attention with Linear Complexity Sinong Wang Belinda Z. Li Madian Khabsa Han Fang Hao Ma 216 1,713 0 08 Jun 2020
Masked Language Modeling for Proteins via Linearly Scalable Long-Context Transformers K. Choromanski Valerii Likhosherstov David Dohan Xingyou Song Andreea Gane ... Peter Hawkins Jared Davis David Belanger Lucy J. Colwell Adrian Weller 88 85 0 05 Jun 2020
Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere Tongzhou Wang Phillip Isola SSL 162 1,855 0 20 May 2020
ResNeSt: Split-Attention Networks Hang Zhang Chongruo Wu Zhongyue Zhang Yi Zhu Yanghua Peng ... Tong He Jonas W. Mueller R. Manmatha Mu Li Alex Smola 114 1,480 0 19 Apr 2020
Reformer: The Efficient Transformer Nikita Kitaev Lukasz Kaiser Anselm Levskaya VLM 201 2,332 0 13 Jan 2020
Meshed-Memory Transformer for Image Captioning Marcella Cornia Matteo Stefanini Lorenzo Baraldi Rita Cucchiara 78 884 0 17 Dec 2019
Improving Multi-Head Attention with Capsule Networks Shuhao Gu Yang Feng 47 13 0 31 Aug 2019
Adaptive Attention Span in Transformers Sainbayar Sukhbaatar Edouard Grave Piotr Bojanowski Armand Joulin 76 286 0 19 May 2019
An Attentive Survey of Attention Models S. Chaudhari Varun Mithal Gungor Polatkan R. Ramanath 146 662 0 05 Apr 2019
Multi-Head Attention with Disagreement Regularization Jian Li Zhaopeng Tu Baosong Yang Michael R. Lyu Tong Zhang 68 147 0 24 Oct 2018
Squeeze-and-Excitation Networks Jie Hu Li Shen Samuel Albanie Gang Sun Enhua Wu 427 26,557 0 05 Sep 2017
The University of Edinburgh's Neural MT Systems for WMT17 Rico Sennrich Alexandra Birch Anna Currey Ulrich Germann Barry Haddow Kenneth Heafield Antonio Valerio Miceli Barone Philip Williams MoE 84 182 0 02 Aug 2017
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering Peter Anderson Xiaodong He Chris Buehler Damien Teney Mark Johnson Stephen Gould Lei Zhang AIMat 123 4,221 0 25 Jul 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 786 132,363 0 12 Jun 2017
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Zhuowen Tu Kaiming He 522 10,347 0 16 Nov 2016
Edinburgh Neural Machine Translation Systems for WMT 16 Rico Sennrich Barry Haddow Alexandra Birch 78 524 0 09 Jun 2016
Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning Christian Szegedy Sergey Ioffe Vincent Vanhoucke Alexander A. Alemi 381 14,263 0 23 Feb 2016
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention Ke Xu Jimmy Ba Ryan Kiros Kyunghyun Cho Aaron Courville Ruslan Salakhutdinov R. Zemel Yoshua Bengio DiffM 350 10,079 0 10 Feb 2015
Deep Visual-Semantic Alignments for Generating Image Descriptions A. Karpathy Li Fei-Fei 152 5,591 0 07 Dec 2014