2390939_en-US

[#1]

How to evaluate NPU tflite model on large dataset on MIMRT700 board?

My question is as my tittle.

I have tflite model, I converted to NPU tflite model. I checked that predictions from tflite model and NPU tflite model are different in some cases (not much). So I want to run evaluation NPU tflite model on the large dataset.

Currently, I follow the sample tflm_cifar10_cm33_core0 to run inference NPU tflite model on  MIMRT700 and it works. But in this sample, we fixed image_data.h for static image (I do not use camera). I want to modify this sample for my new case "run evaluation NPU tflite model on the large dataset."

• I intend to use SD cards to save image and load it when inference as image_data.h. But I do not know where I can start

I saw that MIMRT700 has 3 micro USB port: debug, eUSB and USB-OTG

If you have any recommedation and suggestions, please share with me. If I manually run each image (build + flash), it will take much time.

Re: How to evaluate NPU tflite model on large dataset on MIMRT700 board?

@EdwinHz 

Thank you for supporting me.

The method with SD card is good. I am not good at hardware, so as my teamate's suggestion.

How about this method that I mentioned above?

Currently I follow step by steps to run inference with the sample tflm_cifar10_cm33_core0:

- Copy image_data.h to sample

- Click Build button

- Click debug button, and click continue button to run inference.

I discussed with my teammates and if I can run above via command line (build, and run inference) by using commands, so it is very good. Because if it works, I can create script to change image_data.h each time, build and run inference and get inference results back to PC. Do you have any comment on that?

I mean that if NXP have guideline to build, flash and run sample tflm_cifar10_cm33_core0, I can customize easier. I am not good at hardware, so that I like this method (If NXP support this methods, I can create Python script to creata image_data.h each time, build, flash, run and get result back to PC, save to .csv file, e.t.c). I see other many vendors which support this method (build, flash and run, get result back to PC via command line). I think that NXP also supports this method.

Re: How to evaluate NPU tflite model on large dataset on MIMRT700 board?

Hi @nnxxpp,

The example uses the static-header approach, as you mentioned. However, establishing a pipeline that leverages mounting images via an SD card would be much more suitable for large datasets:

Have the images pre-loaded on the SD card, then mount the SD card, open the image list, and for each image:

- Read into an input buffer
- Run NPU inference
- Write result on a "results.csv" file

We currently do not have a sample code that exemplifies this, but you can refer to both the tflm_cifar10_cm33_core0 that you are using, as well as sdcard_fatfs example from the SDK, which already handles the initialization and card mounting, and has all the available APIs for SD card usage.

I suggest you run and understand the sdcard example, and test reading a binary image file. Then, add the sd card components to the tflm example and import the SD card initialization and FatFs code, and finally replace the static image_data.h input with a buffer containing image info read using f_read() from the SD card.

Having the images already stored in the tensor format would ease the process and prevent the use of JPEG/PNG decoding.

BR,
Edwin.

Re: How to evaluate NPU tflite model on large dataset on MIMRT700 board?

I want to add more information.

Currently I follow step by steps to run inference with the sample tflm_cifar10_cm33_core0:

- Copy image_data.h to sample

- Click Build button

- Click debug button, and click continue button to run inference.

I discussed with my teammates and if I can run above via command line (build, and run inference) by using commands, so it is very good. Because if it works, I can create script to change image_data.h each time, build and run inference and get inference results back to PC. Do you have any comment on that? Thank you so much.

Re: How to evaluate NPU tflite model on large dataset on MIMRT700 board?

I also saw that MIMRT700 has SD card, if I can easily locate images on SDcard on load image to tflm_cifar10_cm33_core0 and save inference results on SD card, so it is very good.

But to be honest, I do know how to start. I am not good at hardware.

Re: How to evaluate NPU tflite model on large dataset on MIMRT700 board?

Hi @nnxxpp,

I understand.

You can run MCUXpresso SDK projects from command line and automate the process using a script. Our SDK command-line flow uses west build, and flashing can be done with west flash -r linkserver.

AN14700 specifically uses CLI to compile the project as described on part 8 of section "7.3 Run the converted model".

So, a general application of a script to automate this process could:

1. copy new image_data.h

2.  build using west build

3. program using west flash

4. capture UART log

Although not the "conventionally recommended method", this would definitely work as well for your application.

BR,
Edwin.