Weight Sparsity Complements Activity Sparsity in Neuromorphic Language
Models

Weight Sparsity Complements Activity Sparsity in Neuromorphic Language Models

1 May 2024

Rishav Mukherji

Khaleelulla Khan Nazeer

Anand Subramoney

Papers citing "Weight Sparsity Complements Activity Sparsity in Neuromorphic Language Models"

18 / 18 papers shown

Title
Sparse Spiking Neural Network: Exploiting Heterogeneity in Timescales for Pruning Recurrent SNN Biswadeep Chakraborty Beomseok Kang H. Kumar Saibal Mukhopadhyay 99 11 0 06 Mar 2024
RWKV: Reinventing RNNs for the Transformer Era Bo Peng Eric Alcaide Quentin G. Anthony Alon Albalak Samuel Arcadinho ... Qihang Zhao P. Zhou Qinghua Zhou Jian Zhu Rui-Jie Zhu 200 593 0 22 May 2023
NeuroBench: A Framework for Benchmarking Neuromorphic Computing Algorithms and Systems Jason Yik Korneel Van den Berghe Douwe den Blanken Younes Bouhadjar Maxime Fabre ... Fatima Tuz Zohora Charlotte Frenkel Vijay Janapa Reddi Charlotte Frenkel Vijay Janapa Reddi 82 21 0 10 Apr 2023
Exploring Lottery Ticket Hypothesis in Spiking Neural Networks Youngeun Kim Yuhang Li Hyoungseob Park Yeshwanth Venkatesha Ruokai Yin Priyadarshini Panda 100 48 0 04 Jul 2022
Two Sparsities Are Better Than One: Unlocking the Performance Benefits of Sparse-Sparse Networks Kevin Lee Hunter Lawrence Spracklen Subutai Ahmad 60 20 0 27 Dec 2021
Spartus: A 9.4 TOp/s FPGA-based LSTM Accelerator Exploiting Spatio-Temporal Sparsity Chang Gao T. Delbruck Shih-Chii Liu 56 45 0 04 Aug 2021
Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks Torsten Hoefler Dan Alistarh Tal Ben-Nun Nikoli Dryden Alexandra Peste MQ 309 717 0 31 Jan 2021
Connection Pruning for Deep Spiking Neural Networks with On-Chip Learning Thao N. N. Nguyen B. Veeravalli Xuanyao Fong 48 14 0 09 Oct 2020
Long Short-Term Memory Spiking Networks and Their Applications Ali Lotfi-Rezaabad S. Vishwanath 47 62 0 09 Jul 2020
STDP Based Pruning of Connections and Weight Quantization in Spiking Neural Networks for Energy Efficient Recognition Nitin Rathi Priyadarshini Panda Kaushik Roy 47 113 0 12 Oct 2017
To prune, or not to prune: exploring the efficacy of pruning for model compression Michael Zhu Suyog Gupta 191 1,275 0 05 Oct 2017
Exploring Sparsity in Recurrent Neural Networks Sharan Narang Erich Elsen G. Diamos Shubho Sengupta 38 312 0 17 Apr 2017
Delta Networks for Optimized Recurrent Network Computation Daniel Neil Junhaeng Lee T. Delbruck Shih-Chii Liu 83 66 0 16 Dec 2016
Improving Neural Language Models with a Continuous Cache Edouard Grave Armand Joulin Nicolas Usunier KELM 46 300 0 13 Dec 2016
Sigma Delta Quantized Networks Peter O'Connor Max Welling 52 48 0 07 Nov 2016
Pointer Sentinel Mixture Models Stephen Merity Caiming Xiong James Bradbury R. Socher RALM 295 2,854 0 26 Sep 2016
Learning both Weights and Connections for Efficient Neural Networks Song Han Jeff Pool J. Tran W. Dally CVBM 310 6,669 0 08 Jun 2015
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation Kyunghyun Cho B. V. Merrienboer Çağlar Gülçehre Dzmitry Bahdanau Fethi Bougares Holger Schwenk Yoshua Bengio AIMat 1.0K 23,338 0 03 Jun 2014

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.