Is Attention Better Than Matrix Decomposition?

9 September 2021

Papers citing "Is Attention Better Than Matrix Decomposition?"

36 / 36 papers shown

Title
You Only Look One Step: Accelerating Backpropagation in Diffusion Sampling with Gradient Shortcuts Hongkun Dou Zeyu Li Xingyu Jiang H. Li Lijun Yang Wen Yao Yue Deng DiffM 38 0 0 12 May 2025
Efficient and Robust Multidimensional Attention in Remote Physiological Sensing through Target Signal Constrained Factorization Jitesh Joshi Youngjun Cho 26 0 0 11 May 2025
SCSegamba: Lightweight Structure-Aware Vision Mamba for Crack Segmentation in Structures Hui Liu Chen Jia Fan Shi Xu Cheng Shengyong Chen Mamba 47 0 0 03 Mar 2025
SegMAN: Omni-scale Context Modeling with State Space Models and Local Attention for Semantic Segmentation Yunxiang Fu Meng Lou Yizhou Yu 115 1 0 16 Dec 2024
Heuristical Comparison of Vision Transformers Against Convolutional Neural Networks for Semantic Segmentation on Remote Sensing Imagery Ashim Dahal Saydul Akbar Murad Nick Rahimi ViT 34 1 0 14 Nov 2024
FactorizePhys: Matrix Factorization for Multidimensional Attention in Remote Physiological Sensing Jitesh Joshi Sos S. Agaian Youngjun Cho AI4TS 41 1 0 03 Nov 2024
Closed-Loop Long-Horizon Robotic Planning via Equilibrium Sequence Modeling Jinghan Li Zhicheng Sun Fei Li 102 1 0 02 Oct 2024
Staircase Cascaded Fusion of Lightweight Local Pattern Recognition and Long-Range Dependencies for Structural Crack Segmentation Hui Liu Chen Jia Fan Shi Xu Cheng Mianzhao Wang Shengyong Chen 43 1 0 23 Aug 2024
HSVLT: Hierarchical Scale-Aware Vision-Language Transformer for Multi-Label Image Classification Shuyi Ouyang Hongyi Wang Ziwei Niu Zhenjia Bai Shiao Xie Yingying Xu Ruofeng Tong Yen-Wei Chen Lanfen Lin VLM 33 1 0 23 Jul 2024
MeMSVD: Long-Range Temporal Structure Capturing Using Incremental SVD Ioanna Ntinou Enrique Sanchez Georgios Tzimiropoulos 42 0 0 11 Jun 2024
MCANet: Medical Image Segmentation with Multi-Scale Cross-Axis Attention Hao Shao Quansheng Zeng Qibin Hou Jufeng Yang 51 13 0 14 Dec 2023
TorchDEQ: A Library for Deep Equilibrium Models Zhengyang Geng J. Zico Kolter VLM 54 12 0 28 Oct 2023
Real-time Automatic M-mode Echocardiography Measurement with Panel Attention from Local-to-Global Pixels Ching-Hsun Tseng S. Chien Po-Shen Wang Shin-Jye Lee Wei-Huan Hu Bin Pu Xiaojun Zeng 24 1 0 15 Aug 2023
MTS-Mixers: Multivariate Time Series Forecasting via Factorized Temporal and Channel Mixing Zhe Li Zhongwen Rao Lujia Pan Zenglin Xu AI4TS 30 60 0 09 Feb 2023
Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers K. Choromanski Shanda Li Valerii Likhosherstov Kumar Avinava Dubey Shengjie Luo Di He Yiming Yang Tamás Sarlós Thomas Weingarten Adrian Weller 34 8 0 03 Feb 2023
Orthogonal SVD Covariance Conditioning and Latent Disentanglement Yue Song N. Sebe Wei Wang 26 6 0 11 Dec 2022
Computational Imaging for Machine Perception: Transferring Semantic Segmentation beyond Aberrations Qi Jiang Haowen Shi Shaohua Gao Jiaming Zhang Kailun Yang Lei Sun Huajian Ni Kaiwei Wang 26 5 0 21 Nov 2022
Deep Equilibrium Approaches to Diffusion Models Ashwini Pokle Zhengyang Geng Zico Kolter DiffM 30 39 0 23 Oct 2022
Eliminating Gradient Conflict in Reference-based Line-Art Colorization Zekun Li Zhengyang Geng Zhao Kang Wenyu Chen Yibo Yang 21 35 0 13 Jul 2022
Improving Covariance Conditioning of the SVD Meta-layer by Orthogonality Yue Song N. Sebe Wei Wang 16 8 0 05 Jul 2022
Deep Equilibrium Optical Flow Estimation Shaojie Bai Zhengyang Geng Yash Savani J. Zico Kolter 27 67 0 18 Apr 2022
Parameter-Efficient Tuning by Manipulating Hidden States of Pretrained Language Models For Classification Tasks Haoran Yang Piji Li Wai Lam 23 2 0 10 Apr 2022
Learning Self-Supervised Low-Rank Network for Single-Stage Weakly and Semi-Supervised Semantic Segmentation Junwen Pan Pengfei Zhu Kaihua Zhang Bing Cao Yu Wang Dingwen Zhang Junwei Han Q. Hu 18 60 0 19 Mar 2022
Factorizer: A Scalable Interpretable Approach to Context Modeling for Medical Image Segmentation Pooya Ashtari Diana Sima L. De Lathauwer D. Sappey-Marinier F. Maes Sabine Van Huffel ViT MedIm 26 35 0 24 Feb 2022
Visual Attention Network Meng-Hao Guo Chengrou Lu Zheng-Ning Liu Ming-Ming Cheng Shiyong Hu ViT VLM 24 637 0 20 Feb 2022
On Training Implicit Models Zhengyang Geng Xinyu Zhang Shaojie Bai Yisen Wang Zhouchen Lin 59 69 0 09 Nov 2021
VQ-GNN: A Universal Framework to Scale up Graph Neural Networks using Vector Quantization Mucong Ding Kezhi Kong Jingling Li Chen Zhu John P. Dickerson Furong Huang Tom Goldstein GNN MQ 33 47 0 27 Oct 2021
SOFT: Softmax-free Transformer with Linear Complexity Jiachen Lu Jinghan Yao Junge Zhang Martin Danelljan Hang Xu Weiguo Gao Chunjing Xu Thomas B. Schon Li Zhang 18 161 0 22 Oct 2021
Beyond Self-attention: External Attention using Two Linear Layers for Visual Tasks Meng-Hao Guo Zheng-Ning Liu Tai-Jiang Mu Shimin Hu 20 473 0 05 May 2021
JFB: Jacobian-Free Backpropagation for Implicit Networks Samy Wu Fung Howard Heaton Qiuwei Li Daniel McKenzie Stanley Osher W. Yin FedML 35 84 0 23 Mar 2021
Graph Neural Networks Inspired by Classical Iterative Algorithms Yongyi Yang T. Liu Yangkun Wang Jinjing Zhou Quan Gan Zhewei Wei Zheng-Wei Zhang Zengfeng Huang David Wipf 33 83 0 10 Mar 2021
Video Transformer Network Daniel Neimark Omri Bar Maya Zohar Dotan Asselmann ViT 198 422 0 01 Feb 2021
Spatial Pyramid Based Graph Reasoning for Semantic Segmentation Xia Li Yibo Yang Qijie Zhao Tiancheng Shen Zhouchen Lin Hong Liu SSeg GNN 109 161 0 23 Mar 2020
Graph-Based Global Reasoning Networks Yunpeng Chen Marcus Rohrbach Zhicheng Yan Shuicheng Yan Jiashi Feng Yannis Kalantidis GNN NAI 268 457 0 30 Nov 2018
A Decomposable Attention Model for Natural Language Inference Ankur P. Parikh Oscar Täckström Dipanjan Das Jakob Uszkoreit 201 1,367 0 06 Jun 2016
Effective Approaches to Attention-based Neural Machine Translation Thang Luong Hieu H. Pham Christopher D. Manning 218 7,926 0 17 Aug 2015