FLatten Transformer: Vision Transformer using Focused Linear Attention

1 August 2023

Gao Huang

Papers citing "FLatten Transformer: Vision Transformer using Focused Linear Attention"

50 / 83 papers shown

Title
DiCo: Revitalizing ConvNets for Scalable and Efficient Diffusion Modeling Yuang Ai Qihang Fan Xuefeng Hu Zhenheng Yang Ran He Huaibo Huang DiffM 37 0 0 16 May 2025
OLinear: A Linear Model for Time Series Forecasting in Orthogonally Transformed Domain Wenzhen Yue Yang Liu Haoxuan Li Hao Wang Xianghua Ying Ruohao Guo Bowei Xing Ji Shi AI4TS OOD 39 0 0 12 May 2025
T-T: Table Transformer for Tagging-based Aspect Sentiment Triplet Extraction Kun Peng Chaodong Tong Cong Cao Hao Peng Yue Liu Guanlin Wu Lei Jiang Yanbing Liu Philip S. Yu LMTD 58 0 0 08 May 2025
ORBIT-2: Scaling Exascale Vision Foundation Models for Weather and Climate Downscaling Xiao Wang Jong Youl Choi Takuya Kurihaya Isaac Lyngaas Hong-Jun Yoon ... Dali Wang Peter Thornton Prasanna Balaprakash M. Ashfaq Dan Lu 38 0 0 07 May 2025
Image Recognition with Online Lightweight Vision Transformer: A Survey Zherui Zhang Rongtao Xu Jie Zhou Changwei Wang Xingtian Pei ... Jiguang Zhang Li Guo Longxiang Gao Wenyuan Xu Shibiao Xu ViT 250 0 0 06 May 2025
HRMedSeg: Unlocking High-resolution Medical Image segmentation via Memory-efficient Attention Modeling Qing Xu Zhenye Lou Chenxin Li Xiangjian He Rong Qu Tesema Fiseha Berhanu Yi Wang Wenting Duan Zhen Chen MedIm 41 0 0 08 Apr 2025
PSA-MIL: A Probabilistic Spatial Attention-Based Multiple Instance Learning for Whole Slide Image Classification Sharon Peled Y. Maruvka Moti Freiman 56 0 0 20 Mar 2025
DVHGNN: Multi-Scale Dilated Vision HGNN for Efficient Vision Recognition Caoshuo Li Tanzhe Li Xiaobin Hu Donghao Luo Taisong Jin 66 0 0 19 Mar 2025
Improving Speech Enhancement by Cross- and Sub-band Processing with State Space Model Jizhen Li Weiping Tu Yuhong Yang Xinmeng Xu Yiqun Zhang Yanzhen Ren Mamba 40 0 0 22 Feb 2025
HyLiFormer: Hyperbolic Linear Attention for Skeleton-based Human Action Recognition Yue Li Haoxuan Qu Mengyuan Liu Jiaheng Liu Y. Cai 56 1 0 09 Feb 2025
Not Every Patch is Needed: Towards a More Efficient and Effective Backbone for Video-based Person Re-identification Lanyun Zhu Tianrun Chen Deyi Ji Jieping Ye Jing Liu 52 2 0 28 Jan 2025
PolaFormer: Polarity-aware Linear Attention for Vision Transformers Weikang Meng Yadan Luo Xin Li D. Jiang Zheng Zhang 272 2 0 25 Jan 2025
Parallel Sequence Modeling via Generalized Spatial Propagation Network Hongjun Wang Wonmin Byeon Jiarui Xu Liang Feng Ka Chun Cheung Xiaolong Wang Kai Han Jan Kautz Sifei Liu 219 0 0 21 Jan 2025
A Separable Self-attention Inspired by the State Space Model for Computer Vision Juntao Zhang Shaogeng Liu Kun Bian You Zhou Pei Zhang Jianning Liu Jun Zhou Bingyan Liu Mamba 62 0 0 03 Jan 2025
Content-aware Balanced Spectrum Encoding in Masked Modeling for Time Series Classification Yudong Han Haocong Wang Yupeng Hu Yongshun Gong Xuemeng Song Weili Guan AI4TS 89 0 0 17 Dec 2024
Bridging the Divide: Reconsidering Softmax and Linear Attention Dongchen Han Yifan Pu Zhuofan Xia Yizeng Han Xuran Pan Xiu Li Jiwen Lu Shiji Song Gao Huang 83 8 0 09 Dec 2024
CARE Transformer: Mobile-Friendly Linear Visual Transformer via Decoupled Dual Interaction Yuan Zhou Qingshan Xu Jiequan Cui Junbao Zhou Jing Zhang Richang Hong Han Zhang ViT 85 0 0 25 Nov 2024
SASE: A Searching Architecture for Squeeze and Excitation Operations Hanming Wang Yunlong Li Zijun Wu Huifen Wang Yuan Zhang 3DPC 29 0 0 13 Nov 2024
Breaking the Low-Rank Dilemma of Linear Attention Qihang Fan Huaibo Huang Ran He 55 1 0 12 Nov 2024
ENAT: Rethinking Spatial-temporal Interactions in Token-based Image Synthesis Zanlin Ni Yulin Wang Renping Zhou Yizeng Han Jiayi Guo Zhiyuan Liu Yuan Yao Gao Huang 68 4 0 11 Nov 2024
DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Execution Yang Yue Yulin Wang Bingyi Kang Yizeng Han Shenzhi Wang Shiji Song Jiashi Feng Gao Huang VLM 52 16 0 04 Nov 2024
MLLA-UNet: Mamba-like Linear Attention in an Efficient U-Shape Model for Medical Image Segmentation Yufeng Jiang Zongxi Li Xiangyan Chen Haoran Xie Jing Cai Mamba 50 1 0 31 Oct 2024
WaveRoRA: Wavelet Rotary Route Attention for Multivariate Time Series Forecasting Aobo Liang Yan Sun Nadra Guizani AI4TS 52 1 0 30 Oct 2024
Taipan: Efficient and Expressive State Space Language Models with Selective Attention Chien Van Nguyen Huy Huu Nguyen Thang M. Pham Ruiyi Zhang Hanieh Deilamsalehy ... Ryan A. Rossi Trung Bui Viet Dac Lai Franck Dernoncourt Thien Huu Nguyen Mamba RALM 37 1 0 24 Oct 2024
Rethinking Transformer for Long Contextual Histopathology Whole Slide Image Analysis Honglin Li Yunlong Zhang Pingyi Chen Zhongyi Shui Chenglu Zhu Lin Yang MedIm 62 4 0 18 Oct 2024
M $^2$ -ViT: Accelerating Hybrid Vision Transformers with Two-Level Mixed Quantization Yanbiao Liang Huihong Shi Zhongfeng Wang MQ 28 0 0 10 Oct 2024
Efficient Semantic Segmentation via Lightweight Multiple-Information Interaction Network Yangyang Qiu Guoan Xu Guangwei Gao Zhenhua Guo Yi Yu Chia-Wen Lin 35 0 0 03 Oct 2024
Agent Aggregator with Mask Denoise Mechanism for Histopathology Whole Slide Image Analysis Xitong Ling Minxi Ouyang Yizhi Wang Xinrui Chen Renao Yan ... Junru Cheng Tian Guan Sufang Tian Xiaoping Liu Yonghong He 35 5 0 18 Sep 2024
Sorbet: A Neuromorphic Hardware-Compatible Transformer-Based Spiking Language Model Kaiwen Tang Zhanglu Yan Weng-Fai Wong 17 0 0 04 Sep 2024
Efficient Diffusion Transformer with Step-wise Dynamic Attention Mediators Yifan Pu Zhuofan Xia Jiayi Guo Dongchen Han Qixiu Li ... Ji Li Yizeng Han Shiji Song Gao Huang Xiu Li 69 12 0 11 Aug 2024
CAS-ViT: Convolutional Additive Self-attention Vision Transformers for Efficient Mobile Applications Tianfang Zhang Lei Li Yang Zhou Wentao Liu Chen Qian Xiangyang Ji ViT 40 12 0 07 Aug 2024
Image Super-Resolution with Taylor Expansion Approximation and Large Field Reception Jia Feng Yuan Wang Dianbo Sui Fengchuang Xing SupR 49 0 0 01 Aug 2024
Embedding-Free Transformer with Inference Spatial Reduction for Efficient Semantic Segmentation Hyunwoo Yu Yubin Cho Beoungwoo Kang Seunghun Moon Kyeongbo Kong Suk-Ju Kang 39 3 0 24 Jul 2024
PADRe: A Unifying Polynomial Attention Drop-in Replacement for Efficient Vision Transformer Pierre-David Létourneau Manish Kumar Singh Hsin-Pai Cheng Shizhong Han Yunxiao Shi Dalton Jones M. H. Langston Hong Cai Fatih Porikli 44 0 0 16 Jul 2024
Long-Term Prediction Accuracy Improvement of Data-Driven Medium-Range Global Weather Forecast Yifan Hu Fukang Yin Weimin Zhang Kaijun Ren Junqiang Song Kefeng Deng Di Zhang AI4Cl 47 0 0 26 Jun 2024
RepNeXt: A Fast Multi-Scale CNN using Structural Reparameterization Mingshu Zhao Yi Luo Yong Ouyang 45 2 0 23 Jun 2024
SWCF-Net: Similarity-weighted Convolution and Local-global Fusion for Efficient Large-scale Point Cloud Semantic Segmentation Zhenchao Lin Li He Hongqiang Yang Xiaoqun Sun Cuojin Zhang Weinan Chen Yisheng Guan Hong Zhang 3DPC 42 0 0 17 Jun 2024
PyramidMamba: Rethinking Pyramid Feature Fusion with Selective Space State Model for Semantic Segmentation of Remote Sensing Imagery Libo Wang Dongxu Li Sijun Dong Xiaoliang Meng Xiaokang Zhang Danfeng Hong 39 5 0 16 Jun 2024
Hybrid Spatial-spectral Neural Network for Hyperspectral Image Denoising Hao Liang Chengjie Kun Li Xin Tian 36 1 0 13 Jun 2024
Prototypical Transformer as Unified Motion Learners Cheng Han Yawen Lu Guohao Sun James Liang Zhiwen Cao ... S. Dianat Raghuveer M. Rao Tong Geng Zhiqiang Tao Dongfang Liu ViT 46 2 0 03 Jun 2024
Automatic Channel Pruning for Multi-Head Attention Eunho Lee Youngbae Hwang ViT 47 1 0 31 May 2024
P $^2$ -ViT: Power-of-Two Post-Training Quantization and Acceleration for Fully Quantized Vision Transformer Huihong Shi Xin Cheng Wendong Mao Zhongfeng Wang MQ 55 3 0 30 May 2024
FocSAM: Delving Deeply into Focused Objects in Segmenting Anything You Huang Zongyu Lan Liujuan Cao Xianming Lin Shengchuan Zhang Guannan Jiang Rongrong Ji VLM 34 2 0 29 May 2024
Demystify Mamba in Vision: A Linear Attention Perspective Dongchen Han Ziyi Wang Zhuofan Xia Yizeng Han Yifan Pu Chunjiang Ge Jun Song Shiji Song Bo Zheng Gao Huang Mamba 39 51 0 26 May 2024
Dinomaly: The Less Is More Philosophy in Multi-Class Unsupervised Anomaly Detection Jia Guo Shuai Lu Weihang Zhang Huiqi Li Huiqi Li Hongen Liao ViT 69 8 0 23 May 2024
SLAB: Efficient Transformers with Simplified Linear Attention and Progressive Re-parameterized Batch Normalization Jialong Guo Xinghao Chen Yehui Tang Yunhe Wang ViT 51 9 0 19 May 2024
Filling Missing Values Matters for Range Image-Based Point Cloud Segmentation Bike Chen Chen Gong Juha Röning 3DPC 65 3 0 16 May 2024
LeMeViT: Efficient Vision Transformer with Learnable Meta Tokens for Remote Sensing Image Interpretation Wentao Jiang Jing Zhang Di Wang Qiming Zhang Zengmao Wang Bo Du 42 5 0 16 May 2024
HybridHash: Hybrid Convolutional and Self-Attention Deep Hashing for Image Retrieval Chao He Hongxi Wei 38 6 0 13 May 2024
Trio-ViT: Post-Training Quantization and Acceleration for Softmax-Free Efficient Vision Transformer Huihong Shi Haikuo Shao Wendong Mao Zhongfeng Wang ViT MQ 44 3 0 06 May 2024