v1v2v3 (latest)

Transformer in Transformer

27 February 2021

ArXiv (abs)PDF HTML Github (4228★)

Papers citing "Transformer in Transformer"

50 / 558 papers shown

Title
ParZC: Parametric Zero-Cost Proxies for Efficient NAS Peijie Dong Lujun Li Xinglin Pan Zimian Wei Xiang Liu Qiang-qiang Wang Xiaowen Chu 95 3 0 03 Feb 2024
Investigating Recurrent Transformers with Dynamic Halt Jishnu Ray Chowdhury Cornelia Caragea 190 1 0 01 Feb 2024
A Manifold Representation of the Key in Vision Transformers Li Meng Morten Goodwin Anis Yazidi P. Engelstad 93 0 0 01 Feb 2024
SHViT: Single-Head Vision Transformer with Memory Efficient Macro Design Seokju Yun Youngmin Ro ViT 138 36 0 29 Jan 2024
Do deep neural networks utilize the weight space efficiently? Onur Can Koyun B. U. Toreyin 54 0 0 26 Jan 2024
Speech Swin-Transformer: Exploring a Hierarchical Transformer with Shifted Windows for Speech Emotion Recognition Yong Wang Cheng Lu Hailun Lian Yan Zhao Bjorn Schuller Yuan Zong Wenming Zheng 67 11 0 19 Jan 2024
Setting the Record Straight on Transformer Oversmoothing G. Dovonon M. Bronstein Matt J. Kusner 88 6 0 09 Jan 2024
PanGu- $π$ : Enhancing Language Model Architectures via Nonlinearity Compensation Yunhe Wang Hanting Chen Yehui Tang Tianyu Guo Kai Han ... Qinghua Xu Qun Liu Jun Yao Chao Xu Dacheng Tao 128 20 0 27 Dec 2023
PDiT: Interleaving Perception and Decision-making Transformers for Deep Reinforcement Learning Hangyu Mao Rui Zhao Ziyue Li Zhiwei Xu Hao Chen Yiqun Chen Bin Zhang Zhen Xiao Junge Zhang Jiangjin Yin OffRL 47 8 0 26 Dec 2023
A Survey on Open-Set Image Recognition Qiulei Dong Qiulei Dong BDL ObjD 92 7 0 25 Dec 2023
Resource-efficient Generative Mobile Edge Networks in 6G Era: Fundamentals, Framework and Case Study Bingkun Lai Jinbo Wen Jiawen Kang Hongyang Du Jiangtian Nie Changyan Yi Dong In Kim Shengli Xie 56 15 0 19 Dec 2023
Domain adaption and physical constrains transfer learning for shale gas production Zhao-zhong Yang Liangjie Gou Chao Min Duo Yi Xiaogang Li Guo-quan Wen AI4CE 78 0 0 18 Dec 2023
Auto-Prox: Training-Free Vision Transformer Architecture Search via Automatic Proxy Discovery Zimian Wei Lujun Li Peijie Dong Zheng Hui Anggeng Li Menglong Lu H. Pan Zhiliang Tian Dongsheng Li ViT 73 17 0 14 Dec 2023
Factorization Vision Transformer: Modeling Long Range Dependency with Local Window Cost Haolin Qin Daquan Zhou Tingfa Xu Ziyang Bian Jianan Li 76 9 0 14 Dec 2023
A Comprehensive Survey on Multi-modal Conversational Emotion Recognition with Deep Learning Yuntao Shou Tao Meng Wei Ai Nan Yin Keqin Li 112 31 0 10 Dec 2023
A Review of Hybrid and Ensemble in Deep Learning for Natural Language Processing Jianguo Jia Wen-Chieh Liang Youzhi Liang VLM 54 20 0 09 Dec 2023
DocBinFormer: A Two-Level Transformer Network for Effective Document Image Binarization Risab Biswas Swalpa Kumar Roy Ning Wang Umapada Pal Guang-Bin Huang ViT 29 1 0 06 Dec 2023
QuadraNet: Improving High-Order Neural Interaction Efficiency with Hardware-Aware Quadratic Neural Networks Chenhui Xu Fuxun Yu Zirui Xu Chenchen Liu Jinjun Xiong Xiang Chen 81 5 0 29 Nov 2023
Large Language Models in Law: A Survey Jinqi Lai Wensheng Gan Jiayang Wu Zhenlian Qi Philip S. Yu ELM AILaw 121 91 0 26 Nov 2023
Transformer-based Named Entity Recognition in Construction Supply Chain Risk Management in Australia Milad Baghalzadeh Shishehgarkhaneh R. Moehler Yihai Fang Amer A. Hijazi Hamed Aboutorab 102 10 0 23 Nov 2023
Bitformer: An efficient Transformer with bitwise operation-based attention for Big Data Analytics at low-cost low-precision devices Gaoxiang Duan Junkai Zhang Xiaoying Zheng Yongxin Zhu 63 2 0 22 Nov 2023
TSegFormer: 3D Tooth Segmentation in Intraoral Scans with Geometry Guided Transformer Huimin Xiong Kunle Li Kaiyuan Tan Yang Feng Qiufeng Wang Jinxiang Hao Haochao Ying Jian Wu Zuo-Qiang Liu MedIm 60 12 0 22 Nov 2023
HEViTPose: High-Efficiency Vision Transformer for Human Pose Estimation Chengpeng Wu Guangxing Tan Chunyu Li ViT 79 0 0 22 Nov 2023
ChemScraper: Leveraging PDF Graphics Instructions for Molecular Diagram Parsing A. Shah Bryan Amador Abhisek Dey Ming Creekmore Blake Ocampo Scott Denmark R. Zanibbi GNN 192 1 0 20 Nov 2023
Deep Tensor Network Yifan Zhang 116 0 0 18 Nov 2023
Recursion in Recursion: Two-Level Nested Recursion for Length Generalization with Scalability Jishnu Ray Chowdhury Cornelia Caragea 78 5 0 08 Nov 2023
Scattering Vision Transformer: Spectral Mixing Matters Badri N. Patro Vijay Srinivas Agneeswaran 112 15 0 02 Nov 2023
An Improved Transformer-based Model for Detecting Phishing, Spam, and Ham: A Large Language Model Approach Suhaima Jamal H. Wimmer 90 22 0 01 Nov 2023
Improving Robustness for Vision Transformer with a Simple Dynamic Scanning Augmentation Shashank Kotyan Danilo Vasconcellos Vargas ViT 75 4 0 01 Nov 2023
Limited Data, Unlimited Potential: A Study on ViTs Augmented by Masked Autoencoders Srijan Das Tanmay Jain Dominick Reilly P. Balaji Soumyajit Karmakar Shyam Marjit Xiang Li Abhijit Das Michael S. Ryoo 108 16 0 31 Oct 2023
One-for-All: Bridge the Gap Between Heterogeneous Architectures in Knowledge Distillation Zhiwei Hao Jianyuan Guo Kai Han Yehui Tang Han Hu Yunhe Wang Chang Xu 110 72 0 30 Oct 2023
MultiScale Spectral-Spatial Convolutional Transformer for Hyperspectral Image Classification Zhiqiang Gong Xian Zhou Wen Yao ViT 61 1 0 28 Oct 2023
Deep Intrinsic Decomposition with Adversarial Learning for Hyperspectral Image Classification Zhiqiang Gong Xian Zhou Wen Yao 39 1 0 28 Oct 2023
Gramian Attention Heads are Strong yet Efficient Vision Learners Jongbin Ryu Dongyoon Han J. Lim 102 2 0 25 Oct 2023
Exploring Driving Behavior for Autonomous Vehicles Based on Gramian Angular Field Vision Transformer Junwei You Ying Chen Zhuoyu Jiang Zhangchi Liu Zilin Huang Yifeng Ding Bin Ran 80 2 0 21 Oct 2023
Accelerating Vision Transformers Based on Heterogeneous Attention Patterns Deli Yu Teng Xi Jianwei Li Baopu Li Gang Zhang Haocheng Feng Junyu Han Jingtuo Liu Errui Ding Jingdong Wang ViT 81 1 0 11 Oct 2023
Multi-domain improves out-of-distribution and data-limited scenarios for medical image analysis Ece Ozkan Xavier Boix OOD 48 0 0 10 Oct 2023
EViT: An Eagle Vision Transformer with Bi-Fovea Self-Attention Yulong Shi Mingwei Sun Yongshuai Wang Hui Sun Zengqiang Chen 101 4 0 10 Oct 2023
Efficient Adaptation of Large Vision Transformer via Adapter Re-Composing Wei Dong Dawei Yan Zhijun Lin Peng Wang 80 24 0 10 Oct 2023
Anchor-Intermediate Detector: Decoupling and Coupling Bounding Boxes for Accurate Object Detection Yilong Lv Min Li Yujie He Shaopeng Li Zhuzhen He Aitao Yang 45 1 0 09 Oct 2023
TransCC: Transformer Network for Coronary Artery CCTA Segmentation Chenchu Xu Meng Li Xue Wu ViT MedIm 42 1 0 07 Oct 2023
GET: Group Event Transformer for Event-Based Vision Yansong Peng Yueyi Zhang Zhiwei Xiong Xiaoyan Sun Feng Wu 88 41 0 04 Oct 2023
Navigating Cultural Chasms: Exploring and Unlocking the Cultural POV of Text-To-Image Models Mor Ventura Eyal Ben-David Anna Korhonen Roi Reichart 86 13 0 03 Oct 2023
PPT: Token Pruning and Pooling for Efficient Vision Transformers Xinjian Wu Fanhu Zeng Xiudong Wang Xinghao Chen ViT 95 27 0 03 Oct 2023
You Do Not Need Additional Priors in Camouflage Object Detection Yuchen Dong Heng Zhou Chengyang Li Junjie Xie Yongqiang Xie Zhongbo Li 84 1 0 01 Oct 2023
PixArt- $α$ : Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis Junsong Chen Jincheng Yu Chongjian Ge Lewei Yao Enze Xie ... Zhongdao Wang James T. Kwok Ping Luo Huchuan Lu Zhenguo Li DiffM 127 461 0 30 Sep 2023
Deep Model Fusion: A Survey Weishi Li Yong Peng Miao Zhang Liang Ding Han Hu Li Shen FedML MoMe 117 62 0 27 Sep 2023
MLPST: MLP is All You Need for Spatio-Temporal Prediction Zijian Zhang Ze Huang Zhiwei Hu Xiangyu Zhao Wanyu Wang Zitao Liu Junbo Zhang S. Qin Hongwei Zhao AI4TS 46 28 0 23 Sep 2023
On Separate Normalization in Self-supervised Transformers Xiaohui Chen Yinkai Wang Yuanqi Du S. Hassoun Liping Liu ViT 68 2 0 22 Sep 2023
DualToken-ViT: Position-aware Efficient Vision Transformer with Dual Token Fusion Zhenzhen Chu Jiayu Chen Cen Chen Chengyu Wang Ziheng Wu Jun Huang Weining Qian ViT 60 3 0 21 Sep 2023