Nyströmformer: A Nyström-Based Algorithm for Approximating Self-Attention

7 February 2021

Mingxing Tan

Yin Li

Papers citing "Nyströmformer: A Nyström-Based Algorithm for Approximating Self-Attention"

50 / 116 papers shown

Title
OLinear: A Linear Model for Time Series Forecasting in Orthogonally Transformed Domain Wenzhen Yue Yong-Jin Liu Haoxuan Li Hao Wang Xianghua Ying Ruohao Guo Bowei Xing Ji Shi AI4TS OOD 34 0 0 12 May 2025
Graph Laplacian Wavelet Transformer via Learnable Spectral Decomposition Andrew Kiruluta Eric Lundy Priscilla Burity 29 0 0 09 May 2025
HMPE:HeatMap Embedding for Efficient Transformer-Based Small Object Detection YangChen Zeng ViT 33 0 0 18 Apr 2025
Robust Multimodal Survival Prediction with the Latent Differentiation Conditional Variational AutoEncoder Junjie Zhou Jiao Tang Yingli Zuo Peng Wan Daoqiang Zhang Wei Shao 60 1 0 12 Mar 2025
MIRAM: Masked Image Reconstruction Across Multiple Scales for Breast Lesion Risk Prediction H. Q. Vo Pengyu Yuan Zheng Yin Kelvin K. Wong Chika F. Ezeana S. Ly Stephen T. C. Wong H. Nguyen 46 0 0 10 Mar 2025
Attention Condensation via Sparsity Induced Regularized Training Eli Sason Darya Frolova Boris Nazarov Felix Goldberd 210 0 0 03 Mar 2025
Enhancing Video Understanding: Deep Neural Networks for Spatiotemporal Analysis Amir Hosein Fadaei M. Dehaqani 45 0 0 11 Feb 2025
SparseFormer: Detecting Objects in HRW Shots via Sparse Vision Transformer Wenxi Li Yuchen Guo Jilai Zheng Haozhe Lin Chao Ma Lu Fang Xiaokang Yang ViT 62 1 0 11 Feb 2025
ZETA: Leveraging Z-order Curves for Efficient Top-k Attention Qiuhao Zeng Jerry Huang Peng Lu Gezheng Xu Boxing Chen Charles Ling Boyu Wang 54 1 0 24 Jan 2025
EfficientViM: Efficient Vision Mamba with Hidden State Mixer based State Space Duality Sanghyeok Lee Joonmyung Choi Hyunwoo J. Kim 118 3 0 22 Nov 2024
Breaking the Low-Rank Dilemma of Linear Attention Qihang Fan Huaibo Huang Ran He 50 1 0 12 Nov 2024
Rethinking Transformer for Long Contextual Histopathology Whole Slide Image Analysis Honglin Li Yunlong Zhang Pingyi Chen Zhongyi Shui Chenglu Zhu Lin Yang MedIm 44 4 0 18 Oct 2024
Training Ultra Long Context Language Model with Fully Pipelined Distributed Transformer Jinghan Yao Sam Ade Jacobs Masahiro Tanaka Olatunji Ruwase Hari Subramoni D. Panda 33 2 0 30 Aug 2024
Snuffy: Efficient Whole Slide Image Classifier Hossein Jafarinia Alireza Alipanah Danial Hamdi Saeed Razavi Nahal Mirzaie M. Rohban 3DH 50 1 0 15 Aug 2024
SCMIL: Sparse Context-aware Multiple Instance Learning for Predicting Cancer Survival Probability Distribution in Whole Slide Images Zekang Yang Hong Liu Xiangdong Wang 24 1 0 30 Jun 2024
Fredformer: Frequency Debiased Transformer for Time Series Forecasting Xihao Piao Zheng Chen Taichi Murayama Yasuko Matsubara Yasushi Sakurai AI4TS 50 18 0 13 Jun 2024
Learning in Feature Spaces via Coupled Covariances: Asymmetric Kernel SVD and Nyström method Qinghua Tao F. Tonin Alex Lambert Yingyi Chen Panagiotis Patrinos Johan A. K. Suykens 43 1 0 13 Jun 2024
Short-Long Convolutions Help Hardware-Efficient Linear Attention to Focus on Long Sequences Zicheng Liu Siyuan Li Li Wang Zedong Wang Yunfan Liu Stan Z. Li 35 7 0 12 Jun 2024
iMD4GC: Incomplete Multimodal Data Integration to Advance Precise Treatment Response Prediction and Survival Analysis for Gastric Cancer Fengtao Zhou Ying Xu Yanfen Cui Shenyang Zhang Yun Zhu ... Louis Ho Shing Lau Chu Han Dafu Zhang Zhenhui Li Hao Chen 30 1 0 01 Apr 2024
Integrative Graph-Transformer Framework for Histopathology Whole Slide Image Representation and Classification Zhan Shi Jingwei Zhang Jun Kong Fusheng Wang MedIm 42 3 0 26 Mar 2024
PathM3: A Multimodal Multi-Task Multiple Instance Learning Framework for Whole Slide Image Classification and Captioning Qifeng Zhou Wenliang Zhong Yuzhi Guo Michael Xiao Hehuan Ma Junzhou Huang 49 10 0 13 Mar 2024
MamMIL: Multiple Instance Learning for Whole Slide Images with State Space Models Zijie Fang Yifeng Wang Zhi Wang Jian Zhang Xiangyang Ji Yongbing Zhang Mamba 39 6 0 08 Mar 2024
DeepLag: Discovering Deep Lagrangian Dynamics for Intuitive Fluid Prediction Qilong Ma Haixu Wu Lanxiang Xing Jianmin Wang Mingsheng Long AI4CE 29 0 0 04 Feb 2024
CascadedGaze: Efficiency in Global Context Extraction for Image Restoration Amirhosein Ghasemabadi Muhammad Kamran Janjua Mohammad Salameh Chunhua Zhou Fengyu Sun Di Niu 35 11 0 26 Jan 2024
OnDev-LCT: On-Device Lightweight Convolutional Transformers towards federated learning Chu Myaet Thwal Minh N. H. Nguyen Ye Lin Tun Seongjin Kim My T. Thai Choong Seon Hong 64 5 0 22 Jan 2024
MIMONets: Multiple-Input-Multiple-Output Neural Networks Exploiting Computation in Superposition Nicolas Menet Michael Hersche G. Karunaratne Luca Benini Abu Sebastian Abbas Rahimi 36 13 0 05 Dec 2023
Long-MIL: Scaling Long Contextual Multiple Instance Learning for Histopathology Whole Slide Image Analysis Honglin Li Yunlong Zhang Chenglu Zhu Jiatong Cai Sunyi Zheng Lin Yang VLM 37 4 0 21 Nov 2023
DPP-TTS: Diversifying prosodic features of speech via determinantal point processes Seongho Joo Hyukhun Koh Kyomin Jung DiffM 47 0 0 23 Oct 2023
RoFormer for Position Aware Multiple Instance Learning in Whole Slide Image Classification Etienne Pochet Rami Maroun Roger Trullo MedIm 23 2 0 03 Oct 2023
Transformer-VQ: Linear-Time Transformers via Vector Quantization Albert Mohwald 34 15 0 28 Sep 2023
Which Transformer to Favor: A Comparative Analysis of Efficiency in Vision Transformers Tobias Christian Nauen Sebastián M. Palacio Federico Raue Andreas Dengel 42 3 0 18 Aug 2023
Video-FocalNets: Spatio-Temporal Focal Modulation for Video Action Recognition Syed Talal Wasim Muhammad Uzair Khattak Muzammal Naseer Salman Khan M. Shah Fahad Shahbaz Khan ViT 54 19 0 13 Jul 2023
When to Use Efficient Self Attention? Profiling Text, Speech and Image Transformer Variants Anuj Diwan Eunsol Choi David Harwath 43 0 0 14 Jun 2023
NLU on Data Diets: Dynamic Data Subset Selection for NLP Classification Tasks Jean-Michel Attendu Jean-Philippe Corbeil 38 15 0 05 Jun 2023
Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator Ziwei He Meng-Da Yang Minwei Feng Jingcheng Yin Xinbing Wang Jingwen Leng Zhouhan Lin ViT 37 13 0 24 May 2023
Personality Understanding of Fictional Characters during Book Reading Mo Yu JiangNan Li Shunyu Yao Wenjie Pang Xiaochen Zhou Zhou Xiao Fandong Meng Jie Zhou 32 16 0 17 May 2023
SoundStorm: Efficient Parallel Audio Generation Zalan Borsos Matthew Sharifi Damien Vincent Eugene Kharitonov Neil Zeghidour Marco Tagliasacchi 28 98 0 16 May 2023
Ray-Patch: An Efficient Querying for Light Field Transformers T. B. Martins Javier Civera ViT 39 0 0 16 May 2023
SKI to go Faster: Accelerating Toeplitz Neural Networks via Asymmetric Kernels Alexander Moreno Jonathan Mei Luke Walters 23 0 0 15 May 2023
Vcc: Scaling Transformers to 128K Tokens or More by Prioritizing Important Tokens Zhanpeng Zeng Cole Hawkins Min-Fong Hong Aston Zhang Nikolaos Pappas Vikas Singh Shuai Zheng 21 6 0 07 May 2023
Improving Autoregressive NLP Tasks via Modular Linearized Attention Victor Agostinelli Lizhong Chen 27 1 0 17 Apr 2023
On Efficient Training of Large-Scale Deep Learning Models: A Literature Review Li Shen Yan Sun Zhiyuan Yu Liang Ding Xinmei Tian Dacheng Tao VLM 30 41 0 07 Apr 2023
DIR-AS: Decoupling Individual Identification and Temporal Reasoning for Action Segmentation Peiyao Wang Haibin Ling 15 2 0 04 Apr 2023
Dialogue-Contextualized Re-ranking for Medical History-Taking Jian Zhu Ilya Valmianski Anitha Kannan 19 1 0 04 Apr 2023
The Multiscale Surface Vision Transformer Simon Dahan Logan Z. J. Williams Daniel Rueckert E. C. Robinson MedIm ViT 10 2 0 21 Mar 2023
Task-specific Fine-tuning via Variational Information Bottleneck for Weakly-supervised Pathology Whole Slide Image Classification Honglin Li Chenglu Zhu Yunlong Zhang Yuxuan Sun Zhongyi Shui Wenwei Kuang S. Zheng L. Yang 69 57 0 15 Mar 2023
Domain-Specific Pre-training Improves Confidence in Whole Slide Image Classification S. Chitnis Sidong Liu T. Dash T. Verlekar A. Di Ieva S. Berkovsky L. Vig A. Srinivasan 19 4 0 20 Feb 2023
Efficiency 360: Efficient Vision Transformers Badri N. Patro Vijay Srinivas Agneeswaran 26 6 0 16 Feb 2023
The Framework Tax: Disparities Between Inference Efficiency in NLP Research and Deployment Jared Fernandez Jacob Kahn Clara Na Yonatan Bisk Emma Strubell FedML 33 10 0 13 Feb 2023
Efficient Attention via Control Variates Lin Zheng Jianbo Yuan Chong-Jun Wang Lingpeng Kong 34 18 0 09 Feb 2023