v1v2 (latest)

Attention-free Spikformer: Mixing Spike Sequences with Simple Linear Transforms

2 August 2023

Papers citing "Attention-free Spikformer: Mixing Spike Sequences with Simple Linear Transforms"

28 / 28 papers shown

Title
Complex Dynamic Neurons Improved Spiking Transformer Network for Efficient Automatic Speech Recognition Minglun Han Qingyu Wang Tielin Zhang Yi Wang Duzhen Zhang Bo Xu 61 29 0 02 Feb 2023
Spikeformer: A Novel Architecture for Training High-Performance Low-Latency Spiking Neural Network Yudong Li Yunlin Lei Xu Yang 83 27 0 19 Nov 2022
Neuromorphic Data Augmentation for Training Spiking Neural Networks Yuhang Li Youngeun Kim Hyoungseob Park Tamar Geller Priyadarshini Panda 81 78 0 11 Mar 2022
Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting Shi-Wee Deng Yuhang Li Shanghang Zhang Shi Gu 203 256 0 24 Feb 2022
cosFormer: Rethinking Softmax in Attention Zhen Qin Weixuan Sun Huicai Deng Dongxu Li Yunshen Wei Baohong Lv Junjie Yan Lingpeng Kong Yiran Zhong 76 222 0 17 Feb 2022
Early Convolutions Help Transformers See Better Tete Xiao Mannat Singh Eric Mintun Trevor Darrell Piotr Dollár Ross B. Girshick 55 771 0 28 Jun 2021
VOLO: Vision Outlooker for Visual Recognition Li-xin Yuan Qibin Hou Zihang Jiang Jiashi Feng Shuicheng Yan ViT 116 327 0 24 Jun 2021
Twins: Revisiting the Design of Spatial Attention in Vision Transformers Xiangxiang Chu Zhi Tian Yuqing Wang Bo Zhang Haibing Ren Xiaolin K. Wei Huaxia Xia Chunhua Shen ViT 84 1,026 0 28 Apr 2021
Escaping the Big Data Paradigm with Compact Transformers Ali Hassani Steven Walton Nikhil Shah Abulikemu Abuduweili Jiachen Li Humphrey Shi 120 463 0 12 Apr 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 533 3,734 0 24 Feb 2021
Deep Residual Learning in Spiking Neural Networks Wei Fang Zhaofei Yu Yanqing Chen Tiejun Huang T. Masquelier Yonghong Tian 167 500 0 08 Feb 2021
Pre-Trained Image Processing Transformer Hanting Chen Yunhe Wang Tianyu Guo Chang Xu Yiping Deng Zhenhua Liu Siwei Ma Chunjing Xu Chao Xu Wen Gao VLM ViT 143 1,677 0 01 Dec 2020
LIAF-Net: Leaky Integrate and Analog Fire Network for Lightweight and Efficient Spatiotemporal Information Processing Zhenzhi Wu Hehui Zhang Yihan Lin Guoqi Li Meng Wang Ye Tang AI4TS 52 104 0 12 Nov 2020
Going Deeper With Directly-Trained Larger Spiking Neural Networks Hanle Zheng Yujie Wu Lei Deng Yifan Hu Guoqi Li 69 527 0 29 Oct 2020
Deformable DETR: Deformable Transformers for End-to-End Object Detection Xizhou Zhu Weijie Su Lewei Lu Bin Li Xiaogang Wang Jifeng Dai ViT 240 5,098 0 08 Oct 2020
DIET-SNN: Direct Input Encoding With Leakage and Threshold Optimization in Deep Spiking Neural Networks Nitin Rathi Kaushik Roy 81 135 0 09 Aug 2020
Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks Wei Fang Zhaofei Yu Yanqing Chen T. Masquelier Tiejun Huang Yonghong Tian 155 530 0 11 Jul 2020
Long Short-Term Memory Spiking Networks and Their Applications Ali Lotfi-Rezaabad S. Vishwanath 74 62 0 09 Jul 2020
Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention Angelos Katharopoulos Apoorv Vyas Nikolaos Pappas Franccois Fleuret 201 1,786 0 29 Jun 2020
End-to-End Object Detection with Transformers Nicolas Carion Francisco Massa Gabriel Synnaeve Nicolas Usunier Alexander Kirillov Sergey Zagoruyko ViT 3DV PINN 434 13,108 0 26 May 2020
Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagation Nitin Rathi G. Srinivasan Priyadarshini Panda Kaushik Roy 166 303 0 04 May 2020
RMP-SNN: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Network Bing Han G. Srinivasan Kaushik Roy 72 317 0 25 Feb 2020
Temporal Spike Sequence Learning via Backpropagation for Deep Spiking Neural Networks Wenrui Zhang Peng Li 71 221 0 24 Feb 2020
Surrogate Gradient Learning in Spiking Neural Networks Emre Neftci Hesham Mostafa Friedemann Zenke 99 1,240 0 28 Jan 2019
Direct Training for Spiking Neural Networks: Faster, Larger, Better Yujie Wu Lei Deng Guoqi Li Jun Zhu Luping Shi 76 660 0 16 Sep 2018
Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks Yujie Wu Lei Deng Guoqi Li Jun Zhu Luping Shi 64 1,027 0 08 Jun 2017
Training Deep Spiking Neural Networks using Backpropagation Junhaeng Lee T. Delbruck Michael Pfeiffer 92 946 0 31 Aug 2016
Spiking Deep Networks with LIF Neurons Eric Hunsberger C. Eliasmith 61 277 0 29 Oct 2015