Recurrent Neural Networks: An Embedded Computing Perspective

Recurrent Neural Networks (RNNs) are a class of machine learning algorithms used for applications with time-series and sequential data. Recently, a strong interest has emerged to execute RNNs on embedded devices. However, RNN requirements of high computational capability and large memory space is difficult to be met. In this paper, we review the existing implementations of RNN models on embedded platforms and discuss the methods adopted to overcome the limitations of embedded systems. We define the objectives of mapping RNN algorithms on embedded platforms and the challenges facing their realization. Then, we explain the components of RNNs models from an implementation perspective. Furthermore, we discuss the optimizations applied on RNNs to run efficiently on embedded platforms. Additionally, we compare the defined objectives with the implementations and highlight some open research questions and aspects currently not addressed for embedded RNNs. Overall, applying algorithmic optimizations on RNN models and decreasing the memory access overhead is vital to reach high efficiency. To further increase the achievable efficiency, the article points up the more promising optimizations to be applied in future research. Additionally, this article observes that high performance has been targeted by many implementations while flexibility was still less attempted. Thus, the article provides some guidelines for RNNs hardware designers to support flexibility in a better manner.