TinyViT: Fast Pretraining Distillation for Small Vision Transformers

21 July 2022

Lu Yuan

Papers citing "TinyViT: Fast Pretraining Distillation for Small Vision Transformers"

33 / 33 papers shown

Title
The Application of Deep Learning for Lymph Node Segmentation: A Systematic Review Jingguo Qu Xinyang Han Man-Lik Chui Yao Pu Simon Takadiyi Gunda ... Jing Qin Ann Dorothy King Winnie Chiu-Wing Chu J. Cai Michael Tin-Cheung Ying 31 0 0 09 May 2025
False Promises in Medical Imaging AI? Assessing Validity of Outperformance Claims Evangelia Christodoulou Annika Reinke Pascaline Andrè Patrick Godau P. Kalinowski ... Amber L. Simpson A. Kopp-Schneider Gaël Varoquaux O. Colliot Lena Maier-Hein 40 0 0 07 May 2025
Image Recognition with Online Lightweight Vision Transformer: A Survey Zherui Zhang Rongtao Xu Jie Zhou Changwei Wang Xingtian Pei ... Jiguang Zhang Li Guo Longxiang Gao Wenyuan Xu Shibiao Xu ViT 166 0 0 06 May 2025
Cues3D: Unleashing the Power of Sole NeRF for Consistent and Unique Instances in Open-Vocabulary 3D Panoptic Segmentation Feng Xue Wenzhuang Xu Guofeng Zhong Anlong Minga N. Sebe 65 0 0 01 May 2025
Differentially Private 2D Human Pose Estimation Kaushik Bhargav Sivangi Idris Zakariyya Paul Henderson F. Deligianni 163 0 0 14 Apr 2025
HGFormer: Topology-Aware Vision Transformer with HyperGraph Learning Hao Wang Shuo Zhang Biao Leng ViT 82 0 0 03 Apr 2025
IncSAR: A Dual Fusion Incremental Learning Framework for SAR Target Recognition George Karantaidis Athanasios Pantsios Y. Kompatsiaris Symeon Papadopoulos CLL 62 0 0 20 Jan 2025
Rethinking Knowledge in Distillation: An In-context Sample Retrieval Perspective Jinjing Zhu Songze Li Lin Wang 47 0 0 13 Jan 2025
Semantics Prompting Data-Free Quantization for Low-Bit Vision Transformers Mingliang Xu Yuyao Zhou Yuxin Zhang Shen Li Yong Li Rongrong Ji Zhanpeng Zeng Rongrong Ji MQ 94 0 0 31 Dec 2024
Forensics Adapter: Adapting CLIP for Generalizable Face Forgery Detection Xinjie Cui Yuan Li Ao Luo Jiaran Zhou Junyu Dong 131 4 0 29 Nov 2024
SAMEdge: An Edge-cloud Video Analytics Architecture for the Segment Anything Model Rui Lu Siping Shi Yanting Liu Dan Wang VLM 32 2 0 23 Sep 2024
PitVis-2023 Challenge: Workflow Recognition in videos of Endoscopic Pituitary Surgery Adrito Das Danyal Z. Khan Dimitrios Psychogyios Yitong Zhang John G. Hanrahan ... Santiago Rodriguez Pablo Arbelaez Danail Stoyanov Hani J. Marcus Sophia Bano 44 5 0 02 Sep 2024
DeMansia: Mamba Never Forgets Any Tokens Ricky Fang Mamba 24 0 0 04 Aug 2024
HDKD: Hybrid Data-Efficient Knowledge Distillation Network for Medical Image Classification Omar S. El-Assiouti Ghada Hamed Dina Khattab H. M. Ebied 45 1 0 10 Jul 2024
ReDistill: Residual Encoded Distillation for Peak Memory Reduction of CNNs Fang Chen Gourav Datta Mujahid Al Rafi Hyeran Jeon Meng Tang 93 1 0 06 Jun 2024
AdaKD: Dynamic Knowledge Distillation of ASR models using Adaptive Loss Weighting Shreyan Ganguly Roshan Nayak Rakshith Rao Ujan Deb AP Prathosh 32 1 0 11 May 2024
Semi-Supervised Unconstrained Head Pose Estimation in the Wild Huayi Zhou Fei Jiang Hongtao Lu Yong Rui Hongtao Lu Kui Jia 58 0 0 03 Apr 2024
TinySAM: Pushing the Envelope for Efficient Segment Anything Model Han Shu Wenshuo Li Yehui Tang Yiman Zhang Yihao Chen Houqiang Li Yunhe Wang Xinghao Chen VLM 44 19 0 21 Dec 2023
EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything Yunyang Xiong Bala Varadarajan Lemeng Wu Xiaoyu Xiang Fanyi Xiao ... Dilin Wang Fei Sun Forrest N. Iandola Raghuraman Krishnamoorthi Vikas Chandra VLM 42 139 0 01 Dec 2023
Block-local learning with probabilistic latent representations David Kappel Khaleelulla Khan Nazeer Cabrel Teguemne Fokam Christian Mayr Anand Subramoney 24 4 0 24 May 2023
RIFormer: Keep Your Vision Backbone Effective While Removing Token Mixer Jiahao Wang Songyang Zhang Yong Liu Taiqiang Wu Yujiu Yang Xihui Liu Kai-xiang Chen Ping Luo Dahua Lin 36 20 0 12 Apr 2023
SVT: Supertoken Video Transformer for Efficient Video Understanding Chen-Ming Pan Rui Hou Hanchao Yu Qifan Wang Senem Velipasalar Madian Khabsa ViT 26 0 0 01 Apr 2023
RGBT Tracking via Progressive Fusion Transformer with Dynamically Guided Learning Yabin Zhu Chenglong Li Tianlin Li Jin Tang Zhixiang Huang 40 7 0 26 Mar 2023
Generic-to-Specific Distillation of Masked Autoencoders Wei Huang Zhiliang Peng Li Dong Furu Wei Jianbin Jiao QiXiang Ye 32 22 0 28 Feb 2023
Pixel-Wise Contrastive Distillation Junqiang Huang Zichao Guo 42 4 0 01 Nov 2022
PSAQ-ViT V2: Towards Accurate and General Data-Free Quantization for Vision Transformers Zhikai Li Mengjuan Chen Junrui Xiao Qingyi Gu ViT MQ 43 33 0 13 Sep 2022
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer Sachin Mehta Mohammad Rastegari ViT 218 1,213 0 05 Oct 2021
Mobile-Former: Bridging MobileNet and Transformer Yinpeng Chen Xiyang Dai Dongdong Chen Mengchen Liu Xiaoyi Dong Lu Yuan Zicheng Liu ViT 183 476 0 12 Aug 2021
ImageNet-21K Pretraining for the Masses T. Ridnik Emanuel Ben-Baruch Asaf Noy Lihi Zelnik-Manor SSeg VLM CLIP 181 689 0 22 Apr 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 304 3,625 0 24 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 322 3,708 0 11 Feb 2021
Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labels Sangdoo Yun Seong Joon Oh Byeongho Heo Dongyoon Han Junsuk Choe Sanghyuk Chun 414 142 0 13 Jan 2021
Talking-Heads Attention Noam M. Shazeer Zhenzhong Lan Youlong Cheng Nan Ding L. Hou 101 80 0 05 Mar 2020