Dynamic Grained Encoder for Vision Transformers

10 January 2023

Papers citing "Dynamic Grained Encoder for Vision Transformers"

50 / 51 papers shown

Title
DyDiT++: Dynamic Diffusion Transformers for Efficient Visual Generation Wangbo Zhao Yizeng Han Jiasheng Tang Kai Wang Hao Luo Yibing Song Gao Huang Fan Wang Yang You 124 0 0 09 Apr 2025
SparseFormer: Detecting Objects in HRW Shots via Sparse Vision Transformer Wenxi Li Yuchen Guo Jilai Zheng Haozhe Lin Chao Ma Lu Fang Xiaokang Yang ViT 93 4 0 11 Feb 2025
AutoFormer: Searching Transformers for Visual Recognition Minghao Chen Houwen Peng Jianlong Fu Haibin Ling ViT 80 264 0 01 Jul 2021
Twins: Revisiting the Design of Spatial Attention in Vision Transformers Xiangxiang Chu Zhi Tian Yuqing Wang Bo Zhang Haibing Ren Xiaolin K. Wei Huaxia Xia Chunhua Shen ViT 79 1,013 0 28 Apr 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng Zhang Stephen Lin B. Guo ViT 396 21,281 0 25 Mar 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 481 3,699 0 24 Feb 2021
Dynamic Neural Networks: A Survey Yizeng Han Gao Huang Shiji Song Le Yang Honghui Wang Yulin Wang 3DH AI4TS AI4CE 75 645 0 09 Feb 2021
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet Li-xin Yuan Yunpeng Chen Tao Wang Weihao Yu Yujun Shi Zihang Jiang Francis E. H. Tay Jiashi Feng Shuicheng Yan ViT 117 1,931 0 28 Jan 2021
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity W. Fedus Barret Zoph Noam M. Shazeer MoE 81 2,168 0 11 Jan 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 339 6,731 0 23 Dec 2020
Transformer Interpretability Beyond Attention Visualization Hila Chefer Shir Gur Lior Wolf 103 659 0 17 Dec 2020
End-to-End Object Detection with Fully Convolutional Network Jianfeng Wang Lin Song Zeming Li Hongbin Sun Jian Sun N. Zheng ObjD 80 196 0 07 Dec 2020
Fine-Grained Dynamic Head for Object Detection Lin Song Yanwei Li Zhengkai Jiang Zeming Li Hongbin Sun Jian Sun N. Zheng ObjD 51 41 0 07 Dec 2020
Rethinking Learnable Tree Filter for Generic Feature Transform Lin Song Yanwei Li Zhengkai Jiang Zeming Li Xiangyu Zhang Hongbin Sun Jian Sun N. Zheng 36 17 0 07 Dec 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 524 40,739 0 22 Oct 2020
Deformable DETR: Deformable Transformers for End-to-End Object Detection Xizhou Zhu Weijie Su Lewei Lu Bin Li Xiaogang Wang Jifeng Dai ViT 191 5,046 0 08 Oct 2020
Rethinking Attention with Performers K. Choromanski Valerii Likhosherstov David Dohan Xingyou Song Andreea Gane ... Afroz Mohiuddin Lukasz Kaiser David Belanger Lucy J. Colwell Adrian Weller 165 1,570 0 30 Sep 2020
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 499 2,074 0 28 Jul 2020
Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention Angelos Katharopoulos Apoorv Vyas Nikolaos Pappas Franccois Fleuret 166 1,755 0 29 Jun 2020
Linformer: Self-Attention with Linear Complexity Sinong Wang Belinda Z. Li Madian Khabsa Han Fang Hao Ma 185 1,694 0 08 Jun 2020
Synthesizer: Rethinking Self-Attention in Transformer Models Yi Tay Dara Bahri Donald Metzler Da-Cheng Juan Zhe Zhao Che Zheng 53 335 0 02 May 2020
Longformer: The Long-Document Transformer Iz Beltagy Matthew E. Peters Arman Cohan RALM VLM 128 4,048 0 10 Apr 2020
Learning Dynamic Routing for Semantic Segmentation Yanwei Li Lin Song Yukang Chen Zeming Li Xinming Zhang Xingang Wang Jian Sun SSeg 113 163 0 23 Mar 2020
Resolution Adaptive Networks for Efficient Inference Le Yang Yizeng Han Xi Chen Shiji Song Jifeng Dai Gao Huang 55 219 0 16 Mar 2020
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 294 596 0 12 Mar 2020
Sparse Sinkhorn Attention Yi Tay Dara Bahri Liu Yang Donald Metzler Da-Cheng Juan 76 339 0 26 Feb 2020
Reformer: The Efficient Transformer Nikita Kitaev Lukasz Kaiser Anselm Levskaya VLM 181 2,307 0 13 Jan 2020
Axial Attention in Multidimensional Transformers Jonathan Ho Nal Kalchbrenner Dirk Weissenborn Tim Salimans 98 528 0 20 Dec 2019
Dynamic Convolutions: Exploiting Spatial Sparsity for Faster Inference Thomas Verelst Tinne Tuytelaars 44 152 0 06 Dec 2019
Learnable Tree Filter for Structure-preserving Feature Transform Lin Song Yanwei Li Zeming Li Gang Yu Hongbin Sun Jian Sun Nanning Zheng 50 48 0 27 Sep 2019
Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networks Zhonghui You Kun Yan Jinmian Ye Meng Ma Ping Wang 3DPC 44 250 0 18 Sep 2019
TACNet: Transition-Aware Context Network for Spatio-Temporal Action Detection Lin Song Shiwei Zhang Gang Yu Hongbin Sun 129 82 0 31 May 2019
Generating Long Sequences with Sparse Transformers R. Child Scott Gray Alec Radford Ilya Sutskever 93 1,894 0 23 Apr 2019
Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution Yunpeng Chen Haoqi Fan Bing Xu Zhicheng Yan Yannis Kalantidis Marcus Rohrbach Shuicheng Yan Jiashi Feng 98 558 0 10 Apr 2019
Panoptic Feature Pyramid Networks Alexander Kirillov Ross B. Girshick Kaiming He Piotr Dollár ISeg SSeg 106 1,282 0 08 Jan 2019
Context Encoding for Semantic Segmentation Hang Zhang Kristin J. Dana Jianping Shi Zhongyue Zhang Xiaogang Wang A. Tyagi Amit Agrawal SSeg 84 1,251 0 23 Mar 2018
Convolutional Networks with Adaptive Inference Graphs Andreas Veit Serge J. Belongie OOD GNN 93 383 0 30 Nov 2017
SkipNet: Learning Dynamic Routing in Convolutional Networks Xin Wang Feng Yu Zi-Yi Dou Trevor Darrell Joseph E. Gonzalez 66 633 0 26 Nov 2017
BlockDrop: Dynamic Inference Paths in Residual Networks Zuxuan Wu Tushar Nagarajan Abhishek Kumar Steven J. Rennie L. Davis Kristen Grauman Rogerio Feris 84 463 0 22 Nov 2017
mixup: Beyond Empirical Risk Minimization Hongyi Zhang Moustapha Cissé Yann N. Dauphin David Lopez-Paz NoLa 269 9,743 0 25 Oct 2017
Segmentation-Aware Convolutional Networks Using Local Attention Masks Adam W. Harley Konstantinos G. Derpanis Iasonas Kokkinos SSeg 177 142 0 15 Aug 2017
Focal Loss for Dense Object Detection Nayeon Lee Priya Goyal Ross B. Girshick Kaiming He Piotr Dollár ObjD 112 2,997 0 07 Aug 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 622 130,942 0 12 Jun 2017
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer Noam M. Shazeer Azalia Mirhoseini Krzysztof Maziarz Andy Davis Quoc V. Le Geoffrey E. Hinton J. Dean MoE 219 2,630 0 23 Jan 2017
Feature Pyramid Networks for Object Detection Nayeon Lee Piotr Dollár Ross B. Girshick Kaiming He Bharath Hariharan Serge J. Belongie ObjD 437 22,040 0 09 Dec 2016
Categorical Reparameterization with Gumbel-Softmax Eric Jang S. Gu Ben Poole BDL 273 5,360 0 03 Nov 2016
SGDR: Stochastic Gradient Descent with Warm Restarts I. Loshchilov Frank Hutter ODL 288 8,091 0 13 Aug 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.8K 193,426 0 10 Dec 2015
Going Deeper with Convolutions Christian Szegedy Wei Liu Yangqing Jia P. Sermanet Scott E. Reed Dragomir Anguelov D. Erhan Vincent Vanhoucke Andrew Rabinovich 395 43,589 0 17 Sep 2014
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 365 43,524 0 01 May 2014