Using a Camera with FRDM-MCXN947 for Vision AI/ML

[#1]

The MCX N microcontroller family includes an eIQ Neutron N1-16 NPU for accelerating neural network models. The FRDM-MCXN947 development board can be combined with a camera and LCD screen to showcase running TinyML vision models on a microcontroller. 

 MCX N Camera Hardware Setup:

The following hardware is used:

• MCX N FRDM Development Board - FRDM-MCXN947
• OV7670 camera (with optional wide-angle lens)
• NXP LCD-PAR-S035

 There are three small modifications needed for the FRDM-MCXN947 board for camera support. Without this modification the camera colors will be incorrect and tinted red.  

• Change SJ16, SJ26, and SJ27 found on the back of the Rev B board to connect pin 3 (the dashed side) so that it looks like the following:

        

Then connect the camera and LCD to the FRDM-MCXN947:

1. Plug in the OV7670 camera into J11. It should line up with the orange box.

                 

       

2. Connect the LCD-PAR-S035 LCD into J12. It should be flush with the bottom so that the top 2 rows of pins are left hanging off the edge. Also note that on some LCD-PAR-S035 boards those top two rows of pins are not installed.

        

3. It should look like the following when complete

         

Also as the camera and Ethernet pins are shared, if you need to use the Ethernet+Camera at the same time please see this NXP Community post.

MCX N Vision ML Examples:

The NXP Application Code Hub contains several vision AI/ML examples:

• Face Detect
• Face Detect with Zephyr
• Multiple Person Detection 
• CIFAR10
• Fashion MNIST

There are also Multimedia Processing Pipeline (MPP) examples inside the MCX N MCUXpresso SDK that demonstrate more examples of using vision AI/ML on MCX N. These examples are only available for VSCode/GCC in the Repository-Layout SDK package.

Note: It is recommended to use MCUXpresso SDK 25.09 for these examples. The MPP issues in the 25.12 and 26.03 MCUXpresso SDK releases should be fixed in the upcoming MCUXpresso SDK 26.06. 

MCX N ML Vision Lab:

The attached eIQ Neutron NPU for MCX N Lab Guide - Face Detect.pdf lab document walks through the steps to download an example Face Detect ML project from the NXP Application Code Hub and use the eIQ Neutron Converter tool to convert a model. It also describes how to update the eIQ and Neutron software libraries in an older MCUXpresso SDK project to work with the latest eIQ Neutron SDK libraries. It is recommended to go through the general MCX N NPU Lab Guide first and then do the attached Face Detect lab second. 

The lab is also included below:

1  Lab Overview

This document will demonstrate the acceleration provided by the eIQ Neutron NPU using the Multiple Face Detection demo for the FRDM-MCXN947 found on the NXP App Code Hub. The demo will run with the non-NPU optimized model and then the performance can be compared to the NPU optimized version of that same model. It also demonstrates how the NPU optimized version of the face detect model was generated. This lab is written for MCUXPresso IDE but the same basic steps can be used for VSCode or GCC.

This lab will also cover how to update the Neutron NPU libraries in the project, as the original Face Detect example uses an older Neutron library version.

It is highly recommended to complete the eIQ Neutron NPU for MCX N Lab Guide before starting this lab.

2  Software and Hardware Installation

This section will cover the hardware and software needed for this lab.

2.1 Hardware

The following hardware is required for this lab:

• MCX N FRDM Development Board - FRDM-MCXN947
• OV7670 camera (with optional wide-angle lens)
• NXP LCD-PAR-S035

2.2 NXP Software Installation

1.          Install MCUXpresso IDE v25.6 or later.
2. Download the latest eIQ Neutron SDK
3. Download and unzip the latest MCUXpresso SDK for FRDM-MCXN947 using MCUXpresso SDK builder
   1. Search for the FRDM-MCXN947 board

1. Then click on Others
2. 
3. On the SDK builder page, make sure to select the “eIQ” middleware and that the MCUXpresso IDE toolchain is selected. Then click on Build SDK.  
4. 
5. Then click on the Download button and accept the license agreement to download the zip file.
6. 

3   Face Detection Example

3.1 Download Face Detect Demo from App Code Hub

The code for this lab can be found on the NXP Application Code Hub hosted on Github, and we can use MCUXpresso IDE to directly import the Face Detection example from App Code Hub.

1. Drag-and-drop the FRDM-MCXN947 SDK zip file into the Installed SDKs window, located on a tab at the bottom of the screen named “Installed SDKs”. You will get the following pop-up, so hit OK.
2. 
3. Once imported, the Installed SDK tab will look something like this: 
4. 
5. In the Quickstart Panel found in the lower left corner, click on Import from Application Code Hub..

4. In the dialog box that pops up there are many filters available to filter for different devices and types of demos. But since the name of the demo we are interested in is already known, the search box will be faster. Select the AI/ML category and then type in “face detection” and then click on the “Multiple face detection on mcxn947” demo. Make sure you don’t accidently click on the “Multiple Person Detection” demo.
5. 
6. On the popup that comes up, click on GitHub link at the top. At that point the Next button at the bottom will become clickable so click on that.

6. The next screen displays the possible branches. In this case there is only main so just click on the Next button at the bottom to go with the default.
7. 
8. The next dialog box determines the location on your computer where the code will be downloaded to. You can leave it at the default location if desired or click on Browse to pick your own location. Then click on Next.
9. 
10. The next screen will download the code and ask about importing the project. Click on Next to go with the default Import existing Eclipse projects option.
11. 
12. Then finally on the last screen click on Finish to import the project into your MCUXPresso IDE workspace.
13. 
14. You may get the following warning due to the project being made on an older

version of the SDK. Then hit OK to accept the using the newest version.

 15. It should look like the following when done:

3.2 Convert Model

The demo is already using a model that was converted to take advantage of the eIQ Neutron NPU. This purpose of this section of the lab is to teach new NXP users how that model was converted.

1. Unzip the eIQ Neutron SDK package in a directory of your choosing.  
2. Optionally add <unzip_location>\eIQ_NeutronSDK_<version>\bin to your executable path so that the neutron-converter utility can be directly called from the command line.
3. Back in MCUXpresso IDE, find the location of the original non-converted model used for this demo by right clicking on the face_detect.tflite file in source/model/ and going to Utilities->Open directory browser here.
4. 
5. Copy the directory location as it will be used in the next step
6. 
7. Open a Windows Command prompt and navigate to the directory where the model was at from the previous step

6. Use the Neutron Converter to convert the Face Detection model:

neutron-converter --input face_detect.tflite --output face_npu.tflite --target mcxn94x

3.3 Update eIQ Neutron Libraries

The Face Detect ACH example uses an older version of the eIQ Neutron libraries, and so it needs to be updated to match the Neutron libraries in newest eIQ Neutron SDK since the model was converted with that version of the Neutron Converter tool.

1. In the frdmmcxn947_multi_face_detection project, right click on the eiq folder and go to Utilities->Open directory browser here
2. 
3. Overwrite the Neutron files from the eIQ Neutron SDK folder into your project to update the Neutron libraries to the latest version:

File Name	Source Directory in eIQ Neutron SDK	Target Directory in MCUXpresso SDK
libNeutronDriver.a	target\mcxn94x\board\	eiq\neutron\mcxn\cm33
libNeutronFirmware.a	target\mcxn94x\board\	eiq\neutron\mcxn\cm33
NeutronDriver.h	target\mcxn94x\driver\include\	eiq\neutron\driver\include
NeutronErrors.h	target\mcxn94x\common\include\	eiq\neutron\common\include

3. After the new Neutron libraries are copied over, clean the project to ensure the new libraries will be used

3.4 Board modifcations

There are some hardware modifications to the MCX FRDM board required for this demo since the camera pins are muxed with the Ethernet pins and the Ethernet functionality is the default.

The board version can be determined by scanning the QR code on the back of the MCX FRDM board with your phone. Most people will have Rev B boards.

• Rev A: Remove the R157, R158, and R159 resistors from the back of the Rev A board so that it looks like the following:
• 
• Rev B: Change SJ16, SJ26, and SJ27 found on the back of the Rev B board to connect pin 3 (the dashed side) so that it looks like the following:

3.5 Connect the camera and LCD

1. Plug in the OV7670 camera into J11. It should line up with the orange box.

2. Connect the LCD-PAR-S035 LCD into J12.
   
   Note that some older LCD-PAR-S035 LCDs may have an extra set of pins soldered on, and in that case the extra 2 rows of pins should be hanging off the edge like in the photo below.

3. It should look like the following when complete

3.6 Run Models

1. Now open up model_data.s by double clicking on it, and then modify line 43 to point to the original (non NPU converted) model file named face_detect.tflite. This particular project uses the .tflite file directly.
2. 
3. Build the project by clicking on the Build icon in the Quickstart Panel
4. 
5. Then download and run the project by clicking on the Debug icon in the Quickstart Panel

4. You should see the demo working with an inference time of 817ms printed on the LCD display.
   • Note: The default camera on the OV7670 is not very wide angle so you have to hold it fairly far back. There are wide-angle lenses that can be purchased to make it easier to demonstrate.
   • Note: After POR there may be some glitching on the camera due to the fact the camera is expecting 2.8V but the board is at 3.3V and the initial HSYNC signal was missed. Press the reset button (SW1) and it should fix any camera issue. 
5. Now let’s use the Neutron optimized model by opening model_data.s again and this time selecting the NPU converted model face_npu.tflite
6. 
7. Recompile and reprogram the board.
8. You’ll see it is significantly faster with a 22ms inference time, a 37x improvement!

4  Conclusion

This lab demonstrated how the eIQ Neutron NPU on MCX N devices can significantly decrease inference time on quantized models and the steps to generate a NPU optimized model using the command line tools. Also explore the other App Code Hub ML examples available online.