Exploring Alternatives to Softmax Function

23 November 2020

Papers citing "Exploring Alternatives to Softmax Function"

4 / 4 papers shown

Title
Fast RoPE Attention: Combining the Polynomial Method and Fast Fourier Transform Josh Alman Zhao Song 22 0 0 17 May 2025
Classified as unknown: A novel Bayesian neural network Tianbo Yang Tianshuo Yang BDL UQCV 10 0 0 31 Jan 2023
SimA: Simple Softmax-free Attention for Vision Transformers Soroush Abbasi Koohpayegani Hamed Pirsiavash 24 25 0 17 Jun 2022
Escaping the Gradient Vanishing: Periodic Alternatives of Softmax in Attention Mechanism Shulun Wang Bin Liu Feng Liu 25 16 0 16 Aug 2021