Transformer in Transformer

27 February 2021

Papers citing "Transformer in Transformer"

50 / 553 papers shown

Title
Setting the Record Straight on Transformer Oversmoothing G. Dovonon M. Bronstein Matt J. Kusner 28 5 0 09 Jan 2024
PanGu- $π$ : Enhancing Language Model Architectures via Nonlinearity Compensation Yunhe Wang Hanting Chen Yehui Tang Tianyu Guo Kai Han ... Qinghua Xu Qun Liu Jun Yao Chao Xu Dacheng Tao 73 16 0 27 Dec 2023
PDiT: Interleaving Perception and Decision-making Transformers for Deep Reinforcement Learning Hangyu Mao Rui Zhao Ziyue Li Zhiwei Xu Hao Chen Yiqun Chen Bin Zhang Zhen Xiao Junge Zhang Jiangjin Yin OffRL 19 8 0 26 Dec 2023
A Survey on Open-Set Image Recognition Jiaying Sun Qiulei Dong BDL ObjD 34 3 0 25 Dec 2023
Resource-efficient Generative Mobile Edge Networks in 6G Era: Fundamentals, Framework and Case Study Bingkun Lai Jinbo Wen Jiawen Kang Hongyang Du Jiangtian Nie Changyan Yi Dong In Kim Shengli Xie 21 14 0 19 Dec 2023
Domain adaption and physical constrains transfer learning for shale gas production Zhao-zhong Yang Liangjie Gou Chao Min Duo Yi Xiaogang Li Guo-quan Wen AI4CE 36 0 0 18 Dec 2023
Auto-Prox: Training-Free Vision Transformer Architecture Search via Automatic Proxy Discovery Zimian Wei Lujun Li Peijie Dong Zheng Hui Anggeng Li Menglong Lu H. Pan Zhiliang Tian Dongsheng Li ViT 45 16 0 14 Dec 2023
Factorization Vision Transformer: Modeling Long Range Dependency with Local Window Cost Haolin Qin Daquan Zhou Tingfa Xu Ziyang Bian Jianan Li 29 9 0 14 Dec 2023
A Comprehensive Survey on Multi-modal Conversational Emotion Recognition with Deep Learning Yuntao Shou Tao Meng Wei Ai Nan Yin Keqin Li 37 30 0 10 Dec 2023
A Review of Hybrid and Ensemble in Deep Learning for Natural Language Processing Jianguo Jia Wen-Chieh Liang Youzhi Liang VLM 17 17 0 09 Dec 2023
DocBinFormer: A Two-Level Transformer Network for Effective Document Image Binarization Risab Biswas Swalpa Kumar Roy Ning Wang Umapada Pal Guang-Bin Huang ViT 22 1 0 06 Dec 2023
QuadraNet: Improving High-Order Neural Interaction Efficiency with Hardware-Aware Quadratic Neural Networks Chenhui Xu Fuxun Yu Zirui Xu Chenchen Liu Jinjun Xiong Xiang Chen 35 4 0 29 Nov 2023
Large Language Models in Law: A Survey Jinqi Lai Wensheng Gan Jiayang Wu Zhenlian Qi Philip S. Yu ELM AILaw 34 72 0 26 Nov 2023
Transformer-based Named Entity Recognition in Construction Supply Chain Risk Management in Australia Milad Baghalzadeh Shishehgarkhaneh R. Moehler Yihai Fang Amer A. Hijazi Hamed Aboutorab 36 6 0 23 Nov 2023
Bitformer: An efficient Transformer with bitwise operation-based attention for Big Data Analytics at low-cost low-precision devices Gaoxiang Duan Junkai Zhang Xiaoying Zheng Yongxin Zhu 36 2 0 22 Nov 2023
TSegFormer: 3D Tooth Segmentation in Intraoral Scans with Geometry Guided Transformer Huimin Xiong Kunle Li Kaiyuan Tan Yang Feng Qiufeng Wang Jinxiang Hao Haochao Ying Jian Wu Zuo-Qiang Liu MedIm 17 11 0 22 Nov 2023
HEViTPose: High-Efficiency Vision Transformer for Human Pose Estimation Chengpeng Wu Guangxing Tan Chunyu Li ViT 21 0 0 22 Nov 2023
ChemScraper: Leveraging PDF Graphics Instructions for Molecular Diagram Parsing A. Shah Bryan Amador Abhisek Dey Ming Creekmore Blake Ocampo Scott Denmark R. Zanibbi GNN 42 1 0 20 Nov 2023
Deep Tensor Network Yifan Zhang 32 0 0 18 Nov 2023
Recursion in Recursion: Two-Level Nested Recursion for Length Generalization with Scalability Jishnu Ray Chowdhury Cornelia Caragea 37 5 0 08 Nov 2023
Scattering Vision Transformer: Spectral Mixing Matters Badri N. Patro Vijay Srinivas Agneeswaran 39 14 0 02 Nov 2023
An Improved Transformer-based Model for Detecting Phishing, Spam, and Ham: A Large Language Model Approach Suhaima Jamal H. Wimmer 24 19 0 01 Nov 2023
Improving Robustness for Vision Transformer with a Simple Dynamic Scanning Augmentation Shashank Kotyan Danilo Vasconcellos Vargas ViT 27 2 0 01 Nov 2023
Limited Data, Unlimited Potential: A Study on ViTs Augmented by Masked Autoencoders Srijan Das Tanmay Jain Dominick Reilly P. Balaji Soumyajit Karmakar Shyam Marjit Xiang Li Abhijit Das Michael S. Ryoo 39 16 0 31 Oct 2023
One-for-All: Bridge the Gap Between Heterogeneous Architectures in Knowledge Distillation Zhiwei Hao Jianyuan Guo Kai Han Yehui Tang Han Hu Yunhe Wang Chang Xu 46 59 0 30 Oct 2023
MultiScale Spectral-Spatial Convolutional Transformer for Hyperspectral Image Classification Zhiqiang Gong Xian Zhou Wen Yao ViT 16 1 0 28 Oct 2023
Deep Intrinsic Decomposition with Adversarial Learning for Hyperspectral Image Classification Zhiqiang Gong Xian Zhou Wen Yao 26 1 0 28 Oct 2023
Gramian Attention Heads are Strong yet Efficient Vision Learners Jongbin Ryu Dongyoon Han J. Lim 35 1 0 25 Oct 2023
Exploring Driving Behavior for Autonomous Vehicles Based on Gramian Angular Field Vision Transformer Junwei You Ying Chen Zhuoyu Jiang Zhangchi Liu Zilin Huang Yifeng Ding Bin Ran 27 0 0 21 Oct 2023
Accelerating Vision Transformers Based on Heterogeneous Attention Patterns Deli Yu Teng Xi Jianwei Li Baopu Li Gang Zhang Haocheng Feng Junyu Han Jingtuo Liu Errui Ding Jingdong Wang ViT 31 0 0 11 Oct 2023
Multi-domain improves out-of-distribution and data-limited scenarios for medical image analysis Ece Ozkan Xavier Boix OOD 28 0 0 10 Oct 2023
EViT: An Eagle Vision Transformer with Bi-Fovea Self-Attention Yulong Shi Mingwei Sun Yongshuai Wang Hui Sun Zengqiang Chen 34 4 0 10 Oct 2023
Efficient Adaptation of Large Vision Transformer via Adapter Re-Composing Wei Dong Dawei Yan Zhijun Lin Peng Wang 27 21 0 10 Oct 2023
Anchor-Intermediate Detector: Decoupling and Coupling Bounding Boxes for Accurate Object Detection Yilong Lv Min Li Yujie He Shaopeng Li Zhuzhen He Aitao Yang 26 1 0 09 Oct 2023
TransCC: Transformer Network for Coronary Artery CCTA Segmentation Chenchu Xu Meng Li Xue Wu ViT MedIm 33 1 0 07 Oct 2023
GET: Group Event Transformer for Event-Based Vision Yansong Peng Yueyi Zhang Zhiwei Xiong Xiaoyan Sun Feng Wu 40 39 0 04 Oct 2023
Navigating Cultural Chasms: Exploring and Unlocking the Cultural POV of Text-To-Image Models Mor Ventura Eyal Ben-David Anna Korhonen Roi Reichart 21 11 0 03 Oct 2023
PPT: Token Pruning and Pooling for Efficient Vision Transformers Xinjian Wu Fanhu Zeng Xiudong Wang Xinghao Chen ViT 32 22 0 03 Oct 2023
You Do Not Need Additional Priors in Camouflage Object Detection Yuchen Dong Heng Zhou Chengyang Li Junjie Xie Yongqiang Xie Zhongbo Li 49 1 0 01 Oct 2023
PixArt- $α$ : Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis Junsong Chen Jincheng Yu Chongjian Ge Lewei Yao Enze Xie ... Zhongdao Wang James T. Kwok Ping Luo Huchuan Lu Zhenguo Li DiffM 39 391 0 30 Sep 2023
Deep Model Fusion: A Survey Weishi Li Yong Peng Miao Zhang Liang Ding Han Hu Li Shen FedML MoMe 39 52 0 27 Sep 2023
MLPST: MLP is All You Need for Spatio-Temporal Prediction Zijian Zhang Ze Huang Zhiwei Hu Xiangyu Zhao Wanyu Wang Zitao Liu Junbo Zhang S. Qin Hongwei Zhao AI4TS 22 27 0 23 Sep 2023
On Separate Normalization in Self-supervised Transformers Xiaohui Chen Yinkai Wang Yuanqi Du S. Hassoun Liping Liu ViT 24 1 0 22 Sep 2023
DualToken-ViT: Position-aware Efficient Vision Transformer with Dual Token Fusion Zhenzhen Chu Jiayu Chen Cen Chen Chengyu Wang Ziheng Wu Jun Huang Weining Qian ViT 13 2 0 21 Sep 2023
Gold-YOLO: Efficient Object Detector via Gather-and-Distribute Mechanism Chengcheng Wang Wei He Ying Nie Jianyuan Guo Chuanjian Liu Kai Han Yunhe Wang ObjD 29 207 0 20 Sep 2023
RMT: Retentive Networks Meet Vision Transformers Qihang Fan Huaibo Huang Mingrui Chen Hongmin Liu Ran He ViT 43 75 0 20 Sep 2023
SCT: A Simple Baseline for Parameter-Efficient Fine-Tuning via Salient Channels Henry Hengyuan Zhao Pichao Wang Yuyang Zhao Hao Luo F. Wang Mike Zheng Shou ViT 37 14 0 15 Sep 2023
Dynamic Spectrum Mixer for Visual Recognition Zhiqiang Hu Tao Yu 30 3 0 13 Sep 2023
Interdisciplinary Fairness in Imbalanced Research Proposal Topic Inference: A Hierarchical Transformer-based Method with Selective Interpolation Meng Xiao Min-Ying Wu Ziyue Qiao Yanjie Fu Zhiyuan Ning Yi Du Yuanchun Zhou 38 8 0 04 Sep 2023
DAT++: Spatially Dynamic Vision Transformer with Deformable Attention Zhuofan Xia Xuran Pan Shiji Song Li Erran Li Gao Huang ViT 29 24 0 04 Sep 2023