An Open-Source Tool for Classification Models in Resource-Constrained Hardware 3

IEEE SENSORS JOURNAL, VOL. XX, NO. XX, XXXX 2021 1 An Open-Source Tool for Classification Models in Resource-Constrained Hardware Lucas Tsutsui da Silva, Vinicius M. A. Souza, and Gustavo E. A. P. A. Batista Abstract— Applications that need to sense, measure, and gather real-time information from the environment frequently face three main restrictions: power consumption, cost, and lack of infrastructure. Most of the challenges imposed by these limitations can be better addressed by embedding Machine Learning (ML) classifiers in the hardware that senses the environment, creating smart sensors able to interpret the low-level data stream. However, for this approach to be cost-effective, we need highly efficient classifiers suitable to execute in unresourceful hardware, such as low-power microcontrollers. In this paper, we present an open-source tool named EmbML – Embedded Machine Learning that implements a pipeline to develop classifiers for resource-constrained hardware. We describe its implementation details and provide a comprehensive analysis of its classifiers considering accuracy, classification time, and memory usage. Moreover, we compare the performance of its classifiers with classifiers produced by related tools to demonstrate that our tool provides a diverse set of classification algorithms that are both compact and accurate. Finally, we validate EmbML classifiers in a practical application of a smart sensor and trap for disease vector mosquitoes. Index Terms— Classification, edge computing, machine learning, smart sensors I. INTRODUCTION the cloud. Therefore, these smart sensors are more power- PPLICATIONS that need to sense, measure, and gather efficient since they eliminate the need for communicating all real-time information from the environment frequently the raw data. Instead, they can only periodically report events A e.g. face three main restrictions [1]: power consumption, cost, of interest – , a dry soil crop area that needs watering or and lack of infrastructure. For example, sensors often have the capture of a disease-vector mosquito. a battery as their main power source, so efficient use of power However, for this approach to be cost-effective, we need allows them to run for extended periods. Price is a significant highly efficient classifiers suitable to execute in sensors’ factor that hinders scaling in several areas, such as agricul- unresourceful hardware, such as low-power microcontrollers. ture. Infrastructure assumptions, including the availability of This scenario conflicts with the state-of-practice of ML, in reliable internet connection or power supply, frequently do not which developers frequently implement classifiers in high- hold in a remote location or low-income countries. level interpreted languages such as Java or Python, make unrestricted use of floating-point operations and assume plenty Most of these challenges can be better addressed by embed- of resources, including memory, processing, and energy. ding Machine Learning (ML) classifiers in the hardware that To overcome these problems, we present an open-source senses the environment, creating smart sensors able to interpret tool named Embedded Machine Learning (EmbML) [3] that arXiv:2105.05983v1 [cs.LG] 12 May 2021 the low-level input. These smart sensors are low-powered implements a pipeline to develop classifiers for resource- systems that usually include one or more sensors, a processing constrained hardware. It starts with learning a classifier in a unit, memory, a power supply, and a radio [2]. Since sensor desktop or server computer using popular software packages devices have restricted memory capacity and can be deployed or libraries such as Waikato Environment for Knowledge in difficult-to-access areas, they often use radio for wireless Analysis (WEKA) [4] and scikit-learn [5]. Next, EmbML communication to transfer the data to a base station or to converts the trained classifier into a carefully crafted C++ code An earlier version of this paper was presented at the 2019 IEEE with support for unresourceful hardware, such as the avoidance ICTAI Conference and was published in its proceedings: https:// of unnecessary use of main memory and implementation of ieeexplore.ieee.org/document/8995408. This work was supported by the United States Agency for Inter- fixed-point operations for real numbers. national Development (USAID), grant AID-OAA-F-16-00072, and the EmbML does not support the learning step in the embedded Brazilian National Council for Scientific and Technological Development hardware. We advocate that, for most ML algorithms, the (CNPq), grant 166919/2017-9. Lucas Tsutsui da Silva is with Universidade de Sao˜ Paulo, Brazil (e- search for model parameters is too expensive to be performed mail: [email protected]). on a microcontroller. However, most ML algorithms output Vinicius M. A. Souza is with the Pontif´ıcia Universidade Catolica´ do classifiers that are highly efficient, including the ones sup- Parana,´ Brazil (e-mail: [email protected]). Gustavo E. A. P. A. Batista is with the University of New South Wales, ported in our tool: Logistic Regression, Decision Trees, Multi- Australia (e-mail: [email protected]). layer Perceptron (MLP), and Support Vector Machine (SVM). 2 IEEE SENSORS JOURNAL, VOL. XX, NO. XX, XXXX 2021 Our main contributions and findings are summarized below: Although the author indicates that this tool can be used to • We provide an open-source tool to convert ML models implement embedded classifiers, the lack of support for other learned on a desktop/server using popular frameworks to classification algorithms restricts its applicability. their use on microcontrollers with constrained resources; There are a considerable number of tools that transform de- • We demonstrate the efficiency of EmbML in a pub- cision tree models into C++ source code. The reason for such lic health case study and carry out a comprehensive prevalence is the direct mapping of the models into if-then- experimental evaluation on six real-world benchmark else statements. Some examples are: J48toCPP3 that supports datasets, four classes of ML algorithms, and six resource- J48 classifiers from WEKA; C4.5 decision tree generator4 that constrained microcontrollers with varied characteristics; converts C4.5 models from WEKA; and DecisionTreeToCpp5 • We empirically demonstrate that the use of fixed-point that converts DecisionTreeClassifier models from scikit-learn. representation improves the classification time on micro- SVM also has various conversion tools to C++. Two exam- 6 controllers without Floating-Point Unit (FPU) and can ples developed for microcontrollers are: mSVM that includes 7 reduce the memory consumption of the classifiers; support to fixed-point arithmetic; and uLIBSVM that provides • For neural network models, we show that the use of a simplified version of the SVM prediction function; however, approximations for the sigmoid function can reduce its without support for fixed-point representation. 8 execution time when compared with the original function; M2cgen converts ML models trained with scikit-learn into • For tree-based models, we show that employing an if- native code in Python, Java, C, JavaScript, PHP, R, Go, and then-else statement structure reduces the model execution others. It supports various classification and regression models, time and does not impact memory consumption. including Logistic Regression, SVM, Decision Tree, RF, and The remaining of this paper is organized as follows: XGBoost. Similarly to sklearn-porter, it does not provide any source code adaptation for microcontrollers. Section II discusses related work; Section III presents the 9 implementation details of EmbML; Section IV describes the Emlearn generates code in C from decision tree, NB, MLP, experimental setup; in Section V we assess the performance and RF models built with scikit-learn or Keras. It includes of EmbML; in Section VI we evaluate the modifications features to support embedded devices, such as avoiding the provided by EmbML; in Section VII we compare the EmbML usage of dynamic memory allocation and standard C library, performance; in Section VIII we present a case study that as well as fixed-point representation for NB classifiers. It has employs our solution; Section IX discusses the limitations of little diversity of models, not supporting popular algorithms our work and, finally, we present our conclusions in Section X. on embedded systems such as SVM. Also, the NB classifier is the only one currently able to use fixed-point arithmetic. TensorFlow Lite for Microcontrollers10 is a library designed II. RELATED WORK to execute TensorFlow Neural Network (NN) models on EmbML has three main objectives: i) having its source 32-bit hardware such as microcontrollers. To decrease the code available to the ML community for free usage and model size and memory usage, it allows applying post-training improvement; ii) generating microcontroller-tailored classifier quantization, reducing the precision of numbers in the model. code with specific modifications to optimize its execution; and Some limitations identified by the authors include: support iii) providing a variety of supported classification models. for a limited subset of TensorFlow operations, support for a Various tools can convert ML models into source code for limited set of devices, low-level C++ API requiring manual unresourceful hardware [6]. However, most of them are from memory management, and lack of support for training models. independent or industry developers, which leads to a scarcity

Load more