Exploring Cell-Based Neural Architectures for Embedded Systems

Neural Architectures Search (NAS) methodologies, which automatically discover state-of-the-art neural networks, have seen a growing interest in recent years. One particular group of NAS methodologies searches for small sub-networks called cells, which are then linearly connected to form the complete neural network. The composition of the final neural network, established through the width of the cells and the depth of the connections, is manually designed while being influenced by the available GPU memory. Typically, the hardware architectures targeted in NAS research are powerful, high-end GPUs. Hence, the attention is on creation of a large neural network that will still fit in the GPU, in turn leading to a very high accuracy for the given task. In direct contrast, we exploit the inherent flexibility of cells to create smaller neural networks, with the intention to study their behaviour on resource-constrained embedded systems. We use the cells discovered from Stochastic Neural Architecture Search (SNAS), to explore the effect that the composition of the cell has on various metrics, namely, the number of parameters, accuracy, latency and power usage. The last two metrics are measured on NVIDIA Jetson Nano, an embedded AI computing platform with a small GPU with mere 4GB on-chip memory. When comparing results of our exploration to the original SNAS architecture’s with 97.02% accuracy for the CIFAR-10 dataset, one particular architecture, with only a tenth of original parameters, achieved an accuracy of 96.14%, notably with 15% lower power consumption and ≈3x faster inference time. Furthermore, this model outperforms other architectures, which are designed for edge devices, specifically to reduce the model size. Thus demonstrating that cell-based architectures, with adequate composition, provide efficient models to be deployed on resource-constrained edge devices.