Machine Learning in Context of IoT/Edge Devices and LoLiPoP-IoT Project

Machine learning models are traditionally deployed in the cloud or on
centralized servers to leverage their computing resources. However,
such a deployment may reduce privacy, introduce extra latency, consume
more power, etc., and subsequently negatively impact properties of an
application that typically runs on a battery-operated device used to
communicate via a wireless network. To minimize the negative impact, it
is necessary to deploy a model directly to such a device to minimize
data transfer energy and run the model closer to the data source and,
application and its environment. However, this kind of deployment is a
challenging task due to the very limited resources available in such
devices and applications. Many people and companies have tackled this
challenging problem and proposed different ways and means to solve it.
Having defined the problem and our area of interest, the paper provides
an overview of representative applications, methods and means, including
libraries, frameworks, datasets, devices etc. It then presents a
typical deployment process workflow in the context of
resource-constrained devices. Finally, it sums representative results
for popular resource-constrained devices (e.g., Arduino, ARM Cortex-M,
ESP32, nRF5x, Nvidia Jetson, Raspberry Pi) to demonstrate how various
phenomena (e.g., model type, setting, quantization) affect model
performance (e.g., accuracy, loss), metrics (e.g., ROC AUC, F1 scores)
and device performance (e.g., feature and inference processing time,
memory usage).