White-Box Transformers via Sparse Rate Reduction

White-Box Transformers via Sparse Rate Reduction

1 June 2023

Shengbang Tong

Papers citing "White-Box Transformers via Sparse Rate Reduction"

18 / 18 papers shown

Title
Revisiting Transformers through the Lens of Low Entropy and Dynamic Sparsity Ruifeng Ren Yong Liu 97 0 0 26 Apr 2025
Simplifying DINO via Coding Rate Regularization Ziyang Wu Jingyuan Zhang Druv Pai X. Wang Chandan Singh Jianwei Yang Jianfeng Gao Yi-An Ma 143 1 0 17 Feb 2025
Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignment Harrish Thasarathan Julian Forsyth Thomas Fel M. Kowal Konstantinos G. Derpanis 100 7 0 06 Feb 2025
Context-Scaling versus Task-Scaling in In-Context Learning Amirhesam Abedsoltan Adityanarayanan Radhakrishnan Jingfeng Wu M. Belkin ReLM LRM 32 3 0 16 Oct 2024
CVVLSNet: Vehicle Location and Speed Estimation Using Partial Connected Vehicle Trajectory Data Jiachen Ye Dingyu Wang Shaocheng Jia Xin Pei Zi Yang Yi Zhang S. Wong 28 0 0 30 Sep 2024
CCDepth: A Lightweight Self-supervised Depth Estimation Network with Enhanced Interpretability Xi Zhang Yaru Xue Shaocheng Jia Xin Pei 28 0 0 30 Sep 2024
KAN See In the Dark Aoxiang Ning Minglong Xue Jinhong He Chengyun Song 31 2 0 05 Sep 2024
Scaling White-Box Transformers for Vision Jinrui Yang Xianhang Li Druv Pai Yuyin Zhou Yi-An Ma Yaodong Yu Cihang Xie ViT 34 9 0 30 May 2024
Towards One Model for Classical Dimensionality Reduction: A Probabilistic Perspective on UMAP and t-SNE Aditya Ravuri Neil D. Lawrence 16 1 0 27 May 2024
Emergence of a High-Dimensional Abstraction Phase in Language Transformers Emily Cheng Diego Doimo Corentin Kervadec Iuri Macocco Jade Yu A. Laio Marco Baroni 104 11 0 24 May 2024
GIFT: Unlocking Full Potential of Labels in Distilled Dataset at Near-zero Cost Xinyi Shang Peng Sun Tao Lin 45 2 0 23 May 2024
U-Nets as Belief Propagation: Efficient Classification, Denoising, and Diffusion in Generative Hierarchical Models Song Mei 3DV AI4CE DiffM 34 11 0 29 Apr 2024
SpiralMLP: A Lightweight Vision MLP Architecture Haojie Mu Burhan Ul Tayyab Nicholas Chua 35 0 0 31 Mar 2024
Understanding the Covariance Structure of Convolutional Filters Asher Trockman Devin Willmott J. Zico Kolter 44 11 0 07 Oct 2022
On the Principles of Parsimony and Self-Consistency for the Emergence of Intelligence Y. Ma Doris Y. Tsao H. Shum 59 75 0 11 Jul 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 278 7,434 0 11 Nov 2021
MLP-Mixer: An all-MLP Architecture for Vision Ilya O. Tolstikhin N. Houlsby Alexander Kolesnikov Lucas Beyer Xiaohua Zhai ... Andreas Steiner Daniel Keysers Jakob Uszkoreit Mario Lucic Alexey Dosovitskiy 239 2,600 0 04 May 2021
A Style-Based Generator Architecture for Generative Adversarial Networks Tero Karras S. Laine Timo Aila 262 10,344 0 12 Dec 2018