Accurate Estimation of the CNN Inference Cost for TinyML Devices

Abstract

Our society will be deeply impacted by neural network inference on embedded devices. Many of them are based on the use of microcontroller units (MCUs) which are extremely resource-scarce. The best modality to solve most of computer vision problems are artificial intelligence algorithms such as Convolutional Neural Networks (CNNs). Although, a CNN’s accuracy implies significant inference costs within the targeted hardware: an important energy consumption, a high latency, and a significant memory footprint. This is the Tiny Machine Learning (TinyML) paradigm that aims at bringing machine learning inference to ultra-low-power devices. Therefore, estimating this inference cost within a given microcontroller unit is a design key. To this end, we introduce an estimation method for the energy consumption, the latency, and the memory space required by a CNN within MCUs. The CNNs investigated are LeNet5 and ResNet8 within several TinyML devices. One inference cost for both neural networks is estimated in a range of 14 frequencies. The proposed method shows an average estimation error of 6.91% on energy consumption, 8.79% on latency, and 3.93% on the required memory space.