Step-by-Step Help to Deploy Custom ML Model on i.MX95 NPU

[varshilg]

Hi, Varshil here.

We are using the i.MX95 processor in our organization for a Computer Vision project. Our goal is to deploy a custom-trained ML model on the NPU (Neural Processing Unit) of the i.MX95 to take advantage of hardware acceleration.

Could you please guide me step-by-step on:
- Which tools or SDKs are required (e.g., eIQ Toolkit, TFLite runtime)?
- How to convert and deploy models (e.g., from TensorFlow or PyTorch)?
- How to run the model on NPU in an eMMC-based Yocto environment / SD Card based Yocto envionment?
- Any sample references or documentation to get started?

Thank you!

[varshilg]

I am facing one more issue that I have my custom model which is right now in the H5 format. But I used EIQ Portal for converting this model into the tflite format and after using Neutron Converter I want to deploy that model on the IMX95 NPU Board.

But I am facing this error.

In that they mention that h5 format is not supported but in the user guide mentioned that h5 will support as Input.

Here I attached my model file as well. Please look and please guide me where is the issue?

[varshilg]

Hi @Manuel_Salas ,

Any Update on it?

Regards,

Varshil

[varshilg]

Hi @Manuel_Salas ,

Thank you for your response!

I went through the user manual, and it was really helpful. I was able to deploy the pre-trained MobileNet V1 model on the IMX95 using the NPU.

Here’s a summary of my results:

root@imx95-iwg61m:~/varshil/IMX95_Model_Deployment# python3 label_image.py \
-m mobilenet_v1_1.0_224_quant.tflite \
-l labels.txt \
-i test.jpg \
-e /usr/lib/libneutron_delegate.so
Loading external delegate from /usr/lib/libneutron_delegate.so with args: {}
INFO: NeutronDelegate delegate: 29 nodes delegated out of 31 nodes with 1 partitions.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
1.000000: fox squirrel
0.000000: toilet tissue
0.000000: starfish
0.000000: zebra
0.000000: sorrel
time: 5.978ms

However, I’m currently stuck while trying to deploy my custom CNN model for Emotion Detection. I trained it on my local server and converted it to TFLite format, but I’m encountering some issues during deployment.

My main question is: What layers are necessary for my model to be compatible with the Neutron Delegate? The user manual mentions supported operators for the Ethos-U Delegate but not for the Neutron Delegate.

I would greatly appreciate your guidance on how to move past this blocker.

Thank you!

Best regards,
Varshil

[Manuel_Salas]

Hello @varshilg 

I hope you are doing very well.

Please take a look to the i.MX Machine Learning User's Guide.

There is explained all the necessary about deploying ML on i.MX family, including i.MX95.

Best regards,

Salas.