Scaling White-Box Transformers for Vision

Scaling White-Box Transformers for Vision

30 May 2024

Papers citing "Scaling White-Box Transformers for Vision"

9 / 9 papers shown

Title
Revisiting Transformers through the Lens of Low Entropy and Dynamic Sparsity Ruifeng Ren Yong Liu 105 0 0 26 Apr 2025
Simplifying DINO via Coding Rate Regularization Ziyang Wu Jingyuan Zhang Druv Pai X. Wang Chandan Singh Jianwei Yang Jianfeng Gao Yi-An Ma 153 1 0 17 Feb 2025
GEXIA: Granularity Expansion and Iterative Approximation for Scalable Multi-grained Video-language Learning Y. Wang Zhikang Zhang Jue Wang D. Fan Zhenlin Xu Linda Liu Xiang Hao Vimal Bhat Xinyu Li VLM 77 1 0 10 Dec 2024
Rethinking Decoders for Transformer-based Semantic Segmentation: A Compression Perspective Qishuai Wen Chun-Guang Li ViT 32 0 0 05 Nov 2024
EVA-CLIP-18B: Scaling CLIP to 18 Billion Parameters Quan-Sen Sun Jinsheng Wang Qiying Yu Yufeng Cui Fan Zhang Xiaosong Zhang Xinlong Wang VLM CLIP MLLM 83 40 0 06 Feb 2024
The effectiveness of MAE pre-pretraining for billion-scale pretraining Mannat Singh Quentin Duval Kalyan Vasudev Alwala Haoqi Fan Vaibhav Aggarwal ... Piotr Dollár Christoph Feichtenhofer Ross B. Girshick Rohit Girdhar Ishan Misra LRM 107 63 0 23 Mar 2023
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 296 7,434 0 11 Nov 2021
MLP-Mixer: An all-MLP Architecture for Vision Ilya O. Tolstikhin N. Houlsby Alexander Kolesnikov Lucas Beyer Xiaohua Zhai ... Andreas Steiner Daniel Keysers Jakob Uszkoreit Mario Lucic Alexey Dosovitskiy 244 2,600 0 04 May 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 226 4,460 0 23 Jan 2020