How to Capture Higher-order Correlations? Generalizing Matrix Softmax Attention to Kronecker Computation

6 October 2023

Papers citing "How to Capture Higher-order Correlations? Generalizing Matrix Softmax Attention to Kronecker Computation"

12 / 12 papers shown

Title
Looped ReLU MLPs May Be All You Need as Practical Programmable Computers Yingyu Liang Zhizhou Sha Zhenmei Shi Zhao-quan Song Yufa Zhou 96 18 0 21 Feb 2025
Fast Gradient Computation for RoPE Attention in Almost Linear Time Yifang Chen Jiayan Huo Xiaoyu Li Yingyu Liang Zhenmei Shi Zhao-quan Song 61 11 0 03 Jan 2025
HSR-Enhanced Sparse Attention Acceleration Bo Chen Yingyu Liang Zhizhou Sha Zhenmei Shi Zhao-quan Song 93 18 0 14 Oct 2024
Differentially Private Kernel Density Estimation Erzhi Liu Jerry Yao-Chieh Hu Alex Reneau Zhao Song Han Liu 66 3 0 03 Sep 2024
When big data actually are low-rank, or entrywise approximation of certain function-generated matrices Stanislav Budzinskiy 62 2 0 03 Jul 2024
Outlier-Efficient Hopfield Layers for Large Transformer-Based Models Jerry Yao-Chieh Hu Pei-Hsuan Chang Haozheng Luo Hong-Yu Chen Weijian Li Wei-Po Wang Han Liu 39 26 0 04 Apr 2024
Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Models Dennis Wu Jerry Yao-Chieh Hu Teng-Yun Hsiao Han Liu 40 28 0 04 Apr 2024
Fast Heavy Inner Product Identification Between Weights and Inputs in Neural Network Training Lianke Qin Saayan Mitra Zhao-quan Song Yuanyuan Yang Dinesh Manocha 27 0 0 19 Nov 2023
The Expressibility of Polynomial based Attention Scheme Zhao-quan Song Guangyi Xu Junze Yin 32 5 0 30 Oct 2023
Differentially Private Attention Computation Yeqi Gao Zhao-quan Song Xin Yang 47 19 0 08 May 2023
Bypass Exponential Time Preprocessing: Fast Neural Network Training via Weight-Data Correlation Preprocessing Josh Alman Jiehao Liang Zhao-quan Song Ruizhe Zhang Danyang Zhuo 71 31 0 25 Nov 2022
On The Computational Complexity of Self-Attention Feyza Duman Keles Pruthuvi Maheshakya Wijewardena C. Hegde 68 108 0 11 Sep 2022