SpikingBERT: Distilling BERT to Train Spiking Language Models Using Implicit Differentiation

21 August 2023

Papers citing "SpikingBERT: Distilling BERT to Train Spiking Language Models Using Implicit Differentiation"

6 / 6 papers shown

Title
SpikeLLM: Scaling up Spiking Neural Network to Large Language Models via Saliency-based Spiking Xingrun Xing Boyan Gao Zheng Zhang David A. Clifton Shitao Xiao LI DU Guoqi Li Jiajun Zhang 50 5 0 05 Jul 2024
Scaling SNNs Trained Using Equilibrium Propagation to Convolutional Architectures Jiaqi Lin Malyaban Bal Abhronil Sengupta 31 2 0 04 May 2024
Spikformer: When Spiking Neural Network Meets Transformer Zhaokun Zhou Yuesheng Zhu Chao He Yaowei Wang Shuicheng Yan Yonghong Tian Liuliang Yuan 145 239 0 29 Sep 2022
Training Feedback Spiking Neural Networks by Implicit Differentiation on the Equilibrium State Mingqing Xiao Qingyan Meng Zongpeng Zhang Yisen Wang Zhouchen Lin 103 60 0 29 Sep 2021
I-BERT: Integer-only BERT Quantization Sehoon Kim A. Gholami Z. Yao Michael W. Mahoney Kurt Keutzer MQ 96 341 0 05 Jan 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,956 0 20 Apr 2018