Knowledge Distillation for Reservoir-based Classifier: Human Activity Recognition

This paper aims to develop an energy-efficient classifier for time-series data by introducing PatchEchoClassifier, a novel model that leverages a reservoir-based mechanism known as the Echo State Network (ESN). The model is designed for human activity recognition (HAR) using one-dimensional sensor signals and incorporates a tokenizer to extract patch-level representations. To train the model efficiently, we propose a knowledge distillation framework that transfers knowledge from a high-capacity MLP-Mixer teacher to the lightweight reservoir-based student model. Experimental evaluations on multiple HAR datasets demonstrate that our model achieves over 80 percent accuracy while significantly reducing computational cost. Notably, PatchEchoClassifier requires only about one-sixth of the floating point operations (FLOPS) compared to DeepConvLSTM, a widely used convolutional baseline. These results suggest that PatchEchoClassifier is a promising solution for real-time and energy-efficient human activity recognition in edge computing environments.

@article{kagiyama2025_2505.22985,
  title={ Knowledge Distillation for Reservoir-based Classifier: Human Activity Recognition },
  author={ Masaharu Kagiyama and Tsuyoshi Okita },
  journal={arXiv preprint arXiv:2505.22985},
  year={ 2025 }
}