How to evaluate NPU tflite model on large dataset on MIMRT700 board? My question is as my tittle. I have tflite model, I converted to NPU tflite model. I checked that predictions from tflite model and NPU tflite model are different in some cases (not much). So I want to run evaluation NPU tflite model on the large dataset. Currently, I follow the sample tflm_cifar10_cm33_core0 to run inference NPU tflite model on MIMRT700 and it works. But in this sample, we fixed image_data.h for static image (I do not use camera). I want to modify this sample for my new case "run evaluation NPU tflite model on the large dataset." I intend to use SD cards to save image and load it when inference as image_data.h. But I do not know where I can start I saw that MIMRT700 has 3 micro USB port: debug, eUSB and USB-OTG If you have any recommedation and suggestions, please share with me. If I manually run each image (build + flash), it will take much time. Re: How to evaluate NPU tflite model on large dataset on MIMRT700 board? @EdwinHz Thank you for supporting me. The method with SD card is good. I am not good at hardware, so as my teamate's suggestion. How about this method that I mentioned above? Currently I follow step by steps to run inference with the sample tflm_cifar10_cm33_core0: - Copy image_data.h to sample - Click Build button - Click debug button, and click continue button to run inference. I discussed with my teammates and if I can run above via command line (build, and run inference) by using commands, so it is very good. Because if it works, I can create script to change image_data.h each time, build and run inference and get inference results back to PC. Do you have any comment on that? I mean that if NXP have guideline to build, flash and run sample tflm_cifar10_cm33_core0, I can customize easier. I am not good at hardware, so that I like this method (If NXP support this methods, I can create Python script to creata image_data.h each time, build, flash, run and get result back to PC, save to .csv file, e.t.c). I see other many vendors which support this method (build, flash and run, get result back to PC via command line). I think that NXP also supports this method. Re: How to evaluate NPU tflite model on large dataset on MIMRT700 board? Hi @nnxxpp,
The example uses the static-header approach, as you mentioned. However, establishing a pipeline that leverages mounting images via an SD card would be much more suitable for large datasets:
Have the images pre-loaded on the SD card, then mount the SD card, open the image list, and for each image:
- Read into an input buffer - Run NPU inference - Write result on a "results.csv" file
We currently do not have a sample code that exemplifies this, but you can refer to both the tflm_cifar10_cm33_core0 that you are using, as well as sdcard_fatfs example from the SDK, which already handles the initialization and card mounting, and has all the available APIs for SD card usage.
I suggest you run and understand the sdcard example, and test reading a binary image file. Then, add the sd card components to the tflm example and import the SD card initialization and FatFs code, and finally replace the static image_data.h input with a buffer containing image info read using f_read() from the SD card.
Having the images already stored in the tensor format would ease the process and prevent the use of JPEG/PNG decoding.
BR, Edwin. Re: How to evaluate NPU tflite model on large dataset on MIMRT700 board? I want to add more information. Currently I follow step by steps to run inference with the sample tflm_cifar10_cm33_core0: - Copy image_data.h to sample - Click Build button - Click debug button, and click continue button to run inference. I discussed with my teammates and if I can run above via command line (build, and run inference) by using commands, so it is very good. Because if it works, I can create script to change image_data.h each time, build and run inference and get inference results back to PC. Do you have any comment on that? Thank you so much. Re: How to evaluate NPU tflite model on large dataset on MIMRT700 board? I also saw that MIMRT700 has SD card, if I can easily locate images on SDcard on load image to tflm_cifar10_cm33_core0 and save inference results on SD card, so it is very good. But to be honest, I do know how to start. I am not good at hardware. Re: How to evaluate NPU tflite model on large dataset on MIMRT700 board? Hi @nnxxpp,
I understand.
You can run MCUXpresso SDK projects from command line and automate the process using a script. Our SDK command-line flow uses west build, and flashing can be done with west flash -r linkserver.
AN14700 specifically uses CLI to compile the project as described on part 8 of section "7.3 Run the converted model".
So, a general application of a script to automate this process could:
1. copy new image_data.h
2. build using west build
3. program using west flash
4. capture UART log
Although not the "conventionally recommended method", this would definitely work as well for your application.
BR, Edwin. Re: How to evaluate NPU tflite model on large dataset on MIMRT700 board? @EdwinHz Yes. Thank you so much. I also investigated and followed this document, and I could flash an run program by using west. From here, I can build script to automate all processes. Before I used this command west build -b mimxrt700evk examples/demo_apps/hello_world -- -Dcore_id=cm33_core0 I saw in the doc of AN14700, there is west build -p always examples/eiq_examples/tflm_label_image --toolchain armgcc --config flash_debug -b mimxrt700evk -Dcore_id=cm33_core0 I have a question about flag -- in the first command. What does it mean? and which flags I should to config when running NPU tflite model on mimxrt700evk?
查看全文
