ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2006.04768
  4. Cited By
Linformer: Self-Attention with Linear Complexity

Linformer: Self-Attention with Linear Complexity

8 June 2020
Sinong Wang
Belinda Z. Li
Madian Khabsa
Han Fang
Hao Ma
ArXivPDFHTML

Papers citing "Linformer: Self-Attention with Linear Complexity"

50 / 1,050 papers shown
Title
Single-Channel EEG Tokenization Through Time-Frequency Modeling
Single-Channel EEG Tokenization Through Time-Frequency Modeling
Jathurshan Pradeepkumar
Xihao Piao
Zheng Chen
Jimeng Sun
45
1
0
22 Feb 2025
Compression Barriers for Autoregressive Transformers
Compression Barriers for Autoregressive Transformers
Themistoklis Haris
Krzysztof Onak
42
1
0
21 Feb 2025
RhythmFormer: Extracting Patterned rPPG Signals based on Periodic Sparse Attention
RhythmFormer: Extracting Patterned rPPG Signals based on Periodic Sparse Attention
Bochao Zou
Zizheng Guo
Jiansheng Chen
Junbao Zhuo
Weiran Huang
Huimin Ma
ViT
AI4TS
115
0
0
21 Feb 2025
Surface Vision Mamba: Leveraging Bidirectional State Space Model for Efficient Spherical Manifold Representation
Surface Vision Mamba: Leveraging Bidirectional State Space Model for Efficient Spherical Manifold Representation
Rongzhao He
Weihao Zheng
Leilei Zhao
Ying Wang
Dalin Zhu
Dan Wu
Bin Hu
Mamba
95
0
0
21 Feb 2025
MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation
MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation
Sihyun Yu
Meera Hahn
Dan Kondratyuk
Jinwoo Shin
Agrim Gupta
José Lezama
Irfan Essa
David A. Ross
Jonathan Huang
DiffM
VGen
80
0
0
18 Feb 2025
Enhancing Video Understanding: Deep Neural Networks for Spatiotemporal Analysis
Enhancing Video Understanding: Deep Neural Networks for Spatiotemporal Analysis
Amir Hosein Fadaei
M. Dehaqani
45
0
0
11 Feb 2025
ZETA: Leveraging Z-order Curves for Efficient Top-k Attention
ZETA: Leveraging Z-order Curves for Efficient Top-k Attention
Qiuhao Zeng
Jerry Huang
Peng Lu
Gezheng Xu
Boxing Chen
Charles Ling
Boyu Wang
57
1
0
24 Jan 2025
5G LDPC Linear Transformer for Channel Decoding
5G LDPC Linear Transformer for Channel Decoding
Mario Hernandez
Fernando Pinero
41
0
0
23 Jan 2025
Parallel Sequence Modeling via Generalized Spatial Propagation Network
Parallel Sequence Modeling via Generalized Spatial Propagation Network
Hongjun Wang
Wonmin Byeon
Jiarui Xu
Liang Feng
Ka Chun Cheung
Xiaolong Wang
Kai Han
Jan Kautz
Sifei Liu
199
0
0
21 Jan 2025
Episodic Memories Generation and Evaluation Benchmark for Large Language Models
Episodic Memories Generation and Evaluation Benchmark for Large Language Models
Alexis Huet
Zied Ben-Houidi
Dario Rossi
LLMAG
62
0
0
21 Jan 2025
ELITR-Bench: A Meeting Assistant Benchmark for Long-Context Language Models
ELITR-Bench: A Meeting Assistant Benchmark for Long-Context Language Models
Thibaut Thonet
Jos Rozen
Laurent Besacier
RALM
145
2
0
20 Jan 2025
MAMo: Leveraging Memory and Attention for Monocular Video Depth Estimation
MAMo: Leveraging Memory and Attention for Monocular Video Depth Estimation
R. Yasarla
H. Cai
Jisoo Jeong
Y. Shi
Risheek Garrepalli
Fatih Porikli
MDE
73
16
0
17 Jan 2025
FutureDepth: Learning to Predict the Future Improves Video Depth Estimation
FutureDepth: Learning to Predict the Future Improves Video Depth Estimation
R. Yasarla
Manish Kumar Singh
Hong Cai
Yunxiao Shi
Jisoo Jeong
Yinhao Zhu
Shizhong Han
Risheek Garrepalli
Fatih Porikli
MDE
98
6
0
17 Jan 2025
Hadamard Attention Recurrent Transformer: A Strong Baseline for Stereo Matching Transformer
Hadamard Attention Recurrent Transformer: A Strong Baseline for Stereo Matching Transformer
Ziyang Chen
Yongjun Zhang
Wenting Li
Bingshu Wang
Yabo Wu
Yong Zhao
C. L. P. Chen
54
0
0
02 Jan 2025
VMamba: Visual State Space Model
VMamba: Visual State Space Model
Yue Liu
Yunjie Tian
Yuzhong Zhao
Hongtian Yu
Lingxi Xie
Yaowei Wang
Qixiang Ye
Jianbin Jiao
Yunfan Liu
Mamba
154
624
0
31 Dec 2024
A Simple Recipe for Contrastively Pre-training Video-First Encoders Beyond 16 Frames
A Simple Recipe for Contrastively Pre-training Video-First Encoders Beyond 16 Frames
Pinelopi Papalampidi
Skanda Koppula
Shreya Pathak
Justin T Chiu
Joseph Heyward
Viorica Patraucean
Jiajun Shen
Antoine Miech
Andrew Zisserman
Aida Nematzdeh
VLM
72
24
0
31 Dec 2024
Learning an Adaptive and View-Invariant Vision Transformer for Real-Time UAV Tracking
Learning an Adaptive and View-Invariant Vision Transformer for Real-Time UAV Tracking
You Wu
Yongxin Li
Mengyuan Liu
Xucheng Wang
Xiangyang Yang
Hengzhou Ye
Dan Zeng
Qijun Zhao
Shuiwang Li
197
0
0
28 Dec 2024
V"Mean"ba: Visual State Space Models only need 1 hidden dimension
V"Mean"ba: Visual State Space Models only need 1 hidden dimension
Tien-Yu Chi
Hung-Yueh Chiang
Chi-Chih Chang
N. Huang
Kai-Chiang Wu
93
0
0
21 Dec 2024
ImagePiece: Content-aware Re-tokenization for Efficient Image
  Recognition
ImagePiece: Content-aware Re-tokenization for Efficient Image Recognition
Seungdong Yoa
Seungjun Lee
Hyeseung Cho
Bumsoo Kim
Woohyung Lim
ViT
75
0
0
21 Dec 2024
BabyHGRN: Exploring RNNs for Sample-Efficient Training of Language
  Models
BabyHGRN: Exploring RNNs for Sample-Efficient Training of Language Models
Patrick Haller
Jonas Golde
Alan Akbik
84
0
0
20 Dec 2024
LIFT: Improving Long Context Understanding Through Long Input
  Fine-Tuning
LIFT: Improving Long Context Understanding Through Long Input Fine-Tuning
Yansheng Mao
Jiaqi Li
Fanxu Meng
Jing Xiong
Zilong Zheng
Muhan Zhang
LLMAG
RALM
104
1
0
18 Dec 2024
Expansion Span: Combining Fading Memory and Retrieval in Hybrid State
  Space Models
Expansion Span: Combining Fading Memory and Retrieval in Hybrid State Space Models
Elvis Nunez
L. Zancato
Benjamin Bowman
Aditya Golatkar
W. Xia
Stefano Soatto
91
2
0
17 Dec 2024
Advances in Transformers for Robotic Applications: A Review
Advances in Transformers for Robotic Applications: A Review
Nikunj Sanghai
Nik Bear Brown
AI4CE
86
0
0
13 Dec 2024
A Decade of Deep Learning: A Survey on The Magnificent Seven
A Decade of Deep Learning: A Survey on The Magnificent Seven
Dilshod Azizov
Muhammad Arslan Manzoor
Velibor Bojkovic
Yingxu Wang
Zhilin Wang
...
Liang Li
Siwei Liu
Yu Zhong
Wei Liu
Shangsong Liang
OOD
AI4TS
MedIm
129
0
0
13 Dec 2024
CubeFormer: A Simple yet Effective Baseline for Lightweight Image
  Super-Resolution
CubeFormer: A Simple yet Effective Baseline for Lightweight Image Super-Resolution
Jikai Wang
Huan Zheng
Jianbing Shen
SupR
86
0
0
03 Dec 2024
Phaseformer: Phase-based Attention Mechanism for Underwater Image
  Restoration and Beyond
Phaseformer: Phase-based Attention Mechanism for Underwater Image Restoration and Beyond
MD Raqib Khan
Anshul Negi
Ashutosh Kulkarni
Shruti S. Phutke
Santosh Kumar Vipparthi
Subrahmanyam Murala
75
1
0
02 Dec 2024
State-Space Large Audio Language Models
State-Space Large Audio Language Models
Saurabhchand Bhati
Yuan Gong
Leonid Karlinsky
Hilde Kuehne
Rogerio Feris
James Glass
101
0
0
24 Nov 2024
Best of Both Worlds: Advantages of Hybrid Graph Sequence Models
Best of Both Worlds: Advantages of Hybrid Graph Sequence Models
Ali Behrouz
Ali Parviz
Mahdi Karami
Clayton Sanford
Bryan Perozzi
Vahab Mirrokni
84
2
0
23 Nov 2024
Nd-BiMamba2: A Unified Bidirectional Architecture for Multi-Dimensional
  Data Processing
Nd-BiMamba2: A Unified Bidirectional Architecture for Multi-Dimensional Data Processing
Hao Liu
Mamba
AI4CE
82
1
0
22 Nov 2024
Financial Risk Assessment via Long-term Payment Behavior Sequence
  Folding
Financial Risk Assessment via Long-term Payment Behavior Sequence Folding
Yiran Qiao
Yateng Tang
Xiang Ao
Qi Yuan
Ziming Liu
Chen Shen
Xuehao Zheng
77
0
0
22 Nov 2024
EfficientViM: Efficient Vision Mamba with Hidden State Mixer based State Space Duality
EfficientViM: Efficient Vision Mamba with Hidden State Mixer based State Space Duality
Sanghyeok Lee
Joonmyung Choi
Hyunwoo J. Kim
118
3
0
22 Nov 2024
A Theory for Compressibility of Graph Transformers for Transductive
  Learning
A Theory for Compressibility of Graph Transformers for Transductive Learning
Hamed Shirzad
Honghao Lin
A. Velingker
B. Venkatachalam
David P. Woodruff
Danica J. Sutherland
85
2
0
20 Nov 2024
MemoryFormer: Minimize Transformer Computation by Removing
  Fully-Connected Layers
MemoryFormer: Minimize Transformer Computation by Removing Fully-Connected Layers
Ning Ding
Yehui Tang
Haochen Qin
Zhenli Zhou
Chao Xu
Lin Li
Kai Han
Heng Liao
Yunhe Wang
67
0
0
20 Nov 2024
ASER: Activation Smoothing and Error Reconstruction for Large Language
  Model Quantization
ASER: Activation Smoothing and Error Reconstruction for Large Language Model Quantization
Weibo Zhao
Yubin Shi
Xinyu Lyu
Wanchen Sui
Shen Li
Yong Li
MQ
52
1
0
12 Nov 2024
AsCAN: Asymmetric Convolution-Attention Networks for Efficient
  Recognition and Generation
AsCAN: Asymmetric Convolution-Attention Networks for Efficient Recognition and Generation
Anil Kag
Huseyin Coskun
Jierun Chen
Junli Cao
Willi Menapace
Aliaksandr Siarohin
Sergey Tulyakov
Jian Ren
53
3
0
07 Nov 2024
$k$NN Attention Demystified: A Theoretical Exploration for Scalable
  Transformers
kkkNN Attention Demystified: A Theoretical Exploration for Scalable Transformers
Themistoklis Haris
44
0
0
06 Nov 2024
A Mamba Foundation Model for Time Series Forecasting
A Mamba Foundation Model for Time Series Forecasting
Haoyu Ma
Yushu Chen
Wenlai Zhao
Jinzhe Yang
Yingsheng Ji
Xinghua Xu
Xiaozhu Liu
Hao Jing
Shengzhuo Liu
Guangwen Yang
AI4TS
Mamba
55
2
0
05 Nov 2024
The Evolution of RWKV: Advancements in Efficient Language Modeling
The Evolution of RWKV: Advancements in Efficient Language Modeling
Akul Datta
VLM
50
1
0
05 Nov 2024
TransUNext: towards a more advanced U-shaped framework for automatic
  vessel segmentation in the fundus image
TransUNext: towards a more advanced U-shaped framework for automatic vessel segmentation in the fundus image
Xiang Li
Mingsi Liu
Lixin Duan
ViT
MedIm
34
0
0
05 Nov 2024
Context Parallelism for Scalable Million-Token Inference
Context Parallelism for Scalable Million-Token Inference
Amy Yang
Jingyi Yang
Aya Ibrahim
Xinfeng Xie
Bangsheng Tang
Grigory Sizov
Jeremy Reizenstein
Jongsoo Park
Jianyu Huang
MoE
LRM
72
5
0
04 Nov 2024
NIMBA: Towards Robust and Principled Processing of Point Clouds With
  SSMs
NIMBA: Towards Robust and Principled Processing of Point Clouds With SSMs
Nursena Köprücü
Destiny Okpekpe
Antonio Orvieto
Mamba
44
1
0
31 Oct 2024
RAM: Replace Attention with MLP for Efficient Multivariate Time Series Forecasting
RAM: Replace Attention with MLP for Efficient Multivariate Time Series Forecasting
Suhan Guo
Jiahong Deng
Yi Wei
Hui Dou
Furao Shen
Jian Zhao
AI4TS
224
0
0
31 Oct 2024
FilterViT and DropoutViT
FilterViT and DropoutViT
Bohang Sun
39
0
0
30 Oct 2024
Long Sequence Modeling with Attention Tensorization: From Sequence to
  Tensor Learning
Long Sequence Modeling with Attention Tensorization: From Sequence to Tensor Learning
Aosong Feng
Rex Ying
Leandros Tassiulas
32
2
0
28 Oct 2024
PODTILE: Facilitating Podcast Episode Browsing with Auto-generated
  Chapters
PODTILE: Facilitating Podcast Episode Browsing with Auto-generated Chapters
Azin Ghazimatin
Ekaterina Garmash
Gustavo Penha
Kristen Sheets
Martin Achenbach
...
Ben Carterette
Ann Clifton
Paul N. Bennett
C. Hauff
M. Lalmas
31
2
0
21 Oct 2024
Taming Mambas for Voxel Level 3D Medical Image Segmentation
Taming Mambas for Voxel Level 3D Medical Image Segmentation
Luca Lumetti
Vittorio Pipoli
Kevin Marchesini
Elisa Ficarra
C. Grana
Federico Bolelli
MedIm
Mamba
29
0
0
20 Oct 2024
Making Every Frame Matter: Continuous Activity Recognition in Streaming Video via Adaptive Video Context Modeling
Making Every Frame Matter: Continuous Activity Recognition in Streaming Video via Adaptive Video Context Modeling
Hao Wu
Donglin Bai
Shiqi Jiang
Qianxi Zhang
Yue Yang
Ting Cao
Fengyuan Xu
Yunxin Liu
Fengyuan Xu
184
0
0
19 Oct 2024
Rethinking Transformer for Long Contextual Histopathology Whole Slide
  Image Analysis
Rethinking Transformer for Long Contextual Histopathology Whole Slide Image Analysis
Honglin Li
Yunlong Zhang
Pingyi Chen
Zhongyi Shui
Chenglu Zhu
Lin Yang
MedIm
57
4
0
18 Oct 2024
An Evolved Universal Transformer Memory
An Evolved Universal Transformer Memory
Edoardo Cetin
Qi Sun
Tianyu Zhao
Yujin Tang
224
0
0
17 Oct 2024
ChuLo: Chunk-Level Key Information Representation for Long Document
  Processing
ChuLo: Chunk-Level Key Information Representation for Long Document Processing
Yan Li
Soyeon Caren Han
Yue Dai
Feiqi Cao
33
0
0
14 Oct 2024
Previous
12345...192021
Next