Part 1: Introduction
The eIQ Neutron Neural Processing Unit (NPU) is a highly scalable accelerator core architecture that provides machine learning (ML) acceleration. Compared to traditional MCUs like the Kinetis series and LPC series, the MCX N series marks the first integration of NXP's eIQ® Neutron NPU for ML acceleration. The eIQ Neutron NPU offers up to 42 times faster machine learning inference performance compared to a standalone CPU core. Specifically, the MCX N94 can execute 4.8G (150MHz * 4 * 4 * 2) INT8 operations per second.
The eIQ Portal, developed in exclusive partnership with Au-Zone Technologies, is an intuitive graphical user interface (GUI) that simplifies vision based ML solutions development. Developers can create, optimize, debug and export ML models, as well as import datasets and models, rapidly train and deploy neural network models and ML workloads for vision applications.
Hardware Environment:
Development Board: FRDM-MCXN947
Display: 3.5" TFT LCD (Part Number: PAR-LCD-S035)
Camera: OV7670
Software Environment:
eIQ Portal: eIQ® ML Software Development Environment | NXP Semiconductors
Application Code Hub Demo：Label CIFAR10 image
Part 2: Basic Model Classification Training and Deployment
The main content is divided into three steps: model training, model converting, and model deployment.
1. Dataset Preparation
The dataset is prepared for a simple demonstration of binary classification between apples and bananas. The training set and test set are split in an 8:2 ratio. This follows the guidelines mentioned in section 3.3.2 Structured Folders Dataset of the eIQ_Toolkit_UG.pdf:
The folder structure is as follows:
Note: The dataset needs to be organized according to the above folder structure.
2. Create Project and Import Dataset into eIQ
a. Open the eIQ Portal tool, click on "CREATE PROJECT" -> "Import dataset".
b. Import using "Structured folders" as follows:
c. After clicking "IMPORT", select the project save path and click "Save".
3. Select Base Model for Training
a. After the dataset is imported, click on "SELECT MODEL" and choose a base model. Modify the "Input Size" to 128, 128, 3.
b. Click on "Start Training". Note: Other parameters can be set according to your needs, and here the "learning rate", "batch size", and "epoch" are set to their default values. This is a demonstration and trains the model for one epoch. Users can train the model as needed to meet application requirements. Upon completion of training, it will look like this:
If the accuracy does not meet the required standard, you can modify the training parameters or update the training data, and then click "CONTINUE TRAINING" to continue the training process.
4. Model Evaluation "VALIDATE"
a. Click on "VALIDATE" to enter the model evaluation stage. Set the parameters including "Softmax Threshold", "Input Data Type", and "Output Data Type". Currently, the MCXN series Neutron NPU only supports the int8 data type. It should look like this:
b.After setting the parameters, click on "VALIDATE" and wait for the confusion matrix to be generated. The confusion matrix provides a clear view of how different categories are classified. In the diagram, the x-axis represents the predicted labels, while the y-axis represents the actual labels. You can see the correspondence between the predicted and actual labels for each image, as shown below:
5. Model Export to TensorFlow Lite
a. Click on "DEPLOY", set the "Export File Type", "Input Data Type", and "Output Data Type". Turn on "Export Quantized Model", and then click on "EXPORT MODEL", as shown below:
b. Set the location to save the model and click "Save".
6. Convert to TensorFlow Lite for Neutron (.tflite)
a. After saving, click on "OPEN MODEL" to view the model structure, as shown below:
b. Click on "Convert" and select "TensorFlow Lite for Neutron (.tflite)" as shown below:
c. Select the "Neutron target", click on "Convert", and set the save path. It should look like this:
7. Deploy the Model to the Label CIFAR10 Image Project
This example is based on a machine learning algorithm supported by the MCXN947, which can label images captured from a camera and display the type of object at the bottom of the LCD. The model is trained on the CIFAR10 dataset, which supports 10 categories of images:
"Airplane", "Automobile", "Bird", "Cat", "Deer", "Dog", "Frog", "Horse", "Ship", "Truck".
a. Open MCUXpresso IDE and import the Label CIFAR10 Image project from the Application Code Hub, as follows:
b. Select the project, click on "GitHub link" -> "Next", as shown below:
c. Set the save path, click "Next" -> "Next" -> "Finish", as shown below:
d. After successful import, click on the "source" folder -> "model" folder, open "model_data.s", and copy the model file converted through eIQ into the "model" folder. Modify the name of the imported model (the name of the converted model) in "model_data.s", as shown below:
Note: The model imported into the project is the one obtained after multiple training sessions.
e. Click on the "source" folder -> "model" folder -> open the "labels.h" file. Modify the "labels[]" array to match the order of labels displayed in the dataset in eIQ, as shown below:
f. Compile the project and download it to the development board.
Part 3: Experimental Results
Part 4: Summary
By efficiently utilizing the powerful performance of the eIQ Neutron NPU and the convenient tools of the eIQ Portal, developers can significantly streamline the entire process from model training to deployment. This not only accelerates the development cycle of machine learning applications but also enhances their performance and reliability. Therefore, for developers looking to implement efficient machine learning applications on MCX N-series edge devices, mastering these technologies and tools is crucial.
We can also refer to the video for detailed steps.
