MCX MicrocontrollersのトピックRe: How can I evaluate NPU tflite model on my PC?

How can I evaluate NPU tflite model on my PC?

nnxxpp — Mon, 02 Dec 2024 07:53:18 GMT

I converted tflite model to tflite model for NPU. With normal tflite model, I can use Tflite lib from Tensorflow. How can I evaluate NPU tflite model on my PC?

Re: How can I evaluate NPU tflite model on my PC?

Harry_Zhang — Tue, 03 Dec 2024 06:17:07 GMT

Hi @nnxxpp

If you want to evaluate the performance of the model, you need to deploy it to the board and run it.

Harry

Re: How can I evaluate NPU tflite model on my PC?

nnxxpp — Tue, 03 Dec 2024 07:37:04 GMT

@Harry_Zhang

Thanks.

It is difficult to evaluate NPU tflite model on board. Because flash memory is very small and usually dataset has thousands of image, we can not push all images to the board and evaluate. NXP supports NPU tflite, it means that it has implementations of custom operator for NeutronGraph. I don't see any guide. Accuracy evaluation is one of the most key we consider when deploying tflite model to the board beside speed.

Re: How can I evaluate NPU tflite model on my PC?

Harry_Zhang — Wed, 04 Dec 2024 06:35:08 GMT

Hi @nnxxpp

If you want to test the accuracy of the model, you can use eiq testing. You can refer to this.

MCXN947: How to Train and Deploy Customer ML model to NPU - NXP Community

Harry

Re: How can I evaluate NPU tflite model on my PC?

nnxxpp — Wed, 04 Dec 2024 07:50:05 GMT

@Harry_Zhang

Thanks. In the link MCXN947: How to Train and Deploy Customer ML model to NPU - NXP Community in the step 4 "4. Model Evaluation "VALIDATE", it allow valiate tflite model with option of choosing data type.

You can see in the step 5, 6, we convert .h5 model to tflite and tflite to NPU tflite. I do not see about validate NPU tflite in eIQ. If I am wrong, please correct me.

Re: How can I evaluate NPU tflite model on my PC?

Harry_Zhang — Fri, 06 Dec 2024 03:38:40 GMT

Hi @nnxxpp

Yes, you are right.

Eiq does not support verifying npu tflite.

Harry

Re: How can I evaluate NPU tflite model on my PC?

nnxxpp — Fri, 06 Dec 2024 03:52:17 GMT

Hi @Harry_Zhang

Thanks. Sure. It means that for MCXN947, we need to deploy NPU tflite model to board.

Re: How can I evaluate NPU tflite model on my PC?

nnxxpp — Thu, 19 Dec 2024 02:02:16 GMT

@Harry_Zhang

Thanks. I expect that you can help me more. I think that you can do it.

Actually I am not sure whether it is worth to evaluate NPU tflite on PC, because NPU is special for NXP board. I know that you custom TFLM to support NPU tflite on board. Could you give me guide to run this custom TFLM on PC. Thank you.

Re: How can I evaluate NPU tflite model on my PC?

Harry_Zhang — Mon, 23 Dec 2024 10:45:42 GMT

Hi @nnxxpp

Running a custom TensorFlow Lite Micro (TFLM) model designed for NPU acceleration on a PC can be challenging, as TFLM is primarily optimized for embedded systems and custom hardware.

Harry