IMX95 - NPU do not works

[IEI-SW3]

As title, I’ve checked many related posts, and it seems that the IMX95 NPU still has quite a few issues. Here’s what I’ve tried so far:

Using the TFLite label_image demo program:

• CPU A1 with kernel 6.6.3 SDK → Even when running label_image with the parameter --external_delegate_path=/usr/lib/libneutron_delegate.so, the NPU doesn’t seem to activate.

• CPU B0 with kernel 6.12.20 SDK → It shows the following error message and then continues to use CPU computation:

  Error in cpuinfo: prctl(PR_SVE_GET_VL) failed INFO: Created TensorFlow Lite XNNPACK delegate for CPU.

My question is: which SDK version actually supports NPU properly? Would upgrading to the latest kernel 6.12.34 possibly help? I’d prefer not to roll back to kernel 6.6 if possible. Thanks.

[Zhiming_Liu]

Hi,

Please download eIQ to convert .tflite with Neutron support.

Zhiming_Liu_0-1761024034289.png

The eIQ1.16 is for L6.12.20 Neutron support.

Zhiming_Liu_1-1761024318029.png

Best Regards,
Zhiming

[IEI-SW3]

Thanks for your response.
Could you proivde me a prebuilt image for IMX95 EVK with 6.6.20 SDK? I can't found the download link from official website, thanks.

[Zhiming_Liu · Accepted solution]

Hi,

Please try the attachment

Best Regards,
Zhiming

[Tongshen]

Hey Zhiming,

I downloaded the converted model and ran the inference under:
Linux version 6.12.20-lts-next-gdfaf2136deb2 (oe-user@oe-host) (aarch64-poky-linux-gcc (GCC) 14.2.0, GNU ld (GNU Binutils) 2.44) #1 SMP PREEMPT Wed Jun 4 10:15:09 UTC 2025
But get the empty result:

root@imx95-19x19-verdin:/usr/bin/tensorflow-lite-2.18.0/examples# python3 label_image.py -e /usr/lib/libneutron_delegate.so
Loading external delegate from /usr/lib/libneutron_delegate.so with args: {}
INFO: NeutronDelegate delegate: 1 nodes delegated out of 4 nodes with 1 partitions.

INFO: Neutron delegate version: v1.0.0-a5d640e6, zerocp enabled.
Error in cpuinfo: prctl(PR_SVE_GET_VL) failed
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Warning: microcode version mismatch! 0xaf140cf5 (expected 0xcebb80a)
Warm-up time: 0.7 ms

Inference time: 0.5 ms

0.000000: toilet tissue
0.000000: starfish
0.000000: zebra
0.000000: sorrel
0.000000: guinea pig


However, when I downloaded the unconverted model and used eIQ v1.16 to convert the model, the inference result shows:

root@imx95-19x19-verdin:/usr/bin/tensorflow-lite-2.18.0/examples# python3 label_image.py -e /usr/lib/libneutron_delegate.so
Loading external delegate from /usr/lib/libneutron_delegate.so with args: {}
INFO: NeutronDelegate delegate: 1 nodes delegated out of 4 nodes with 1 partitions.

INFO: Neutron delegate version: v1.0.0-a5d640e6, zerocp enabled.
Error in cpuinfo: prctl(PR_SVE_GET_VL) failed
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Warm-up time: 2.0 ms

Inference time: 1.9 ms

0.039216: Newfoundland
0.039216: tench
0.039216: Leonberg
0.039216: shower cap
0.039216: thunder snake

Could you give me more information here? Is my kernel version not correct?

[IEI-SW3]

Great, it works using your converted model even if the message still showing delegate for CPU on Kernel 6.12.20 SDK.

root@imx95evk:~# python3.13 ./classification.py -i grace_hopper.bmp -l labels_mobilenet_quant_v1_224.txt -m mobilenet_v1_1.0_224_quant_converted.tflite -e /usr/lib/libneutron_delegate.so
Loading external delegate from /usr/lib/libneutron_delegate.so with args: {}
INFO: NeutronDelegate delegate: 1 nodes delegated out of 4 nodes with 1 partitions.

INFO: Neutron delegate version: v1.0.0-a5d640e6, zerocp enabled.
Error in cpuinfo: prctl(PR_SVE_GET_VL) failed
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Warning: microcode version mismatch! 0xcebb80a (expected 0x36336a10)
0.850980: military uniform
0.058824: Windsor tie
0.011765: bulletproof vest
0.007843: bow tie
0.007843: mortarboard
time: 1.484ms

Where's the converted tool I can get? Because my target is running mobilebert.tflite for tiny NLP model, it seems convert for Neutron NPU is necessary, thanks.

If you have better way to running NLP (no matter LLM or SLM model), welcome to provide me, thanks a lot

[Zhiming_Liu]

Hi,
Please install eIQ from here.

https://www.nxp.com/design/design-center/software/eiq-ai-development-environment/eiq-toolkit-for-end...

For LLM models, let me confirm if we can provide it.


Best Regards,
Zhiming

[IEI-SW3]

Hello, I would like to know if this conversion method will become the standard approach in the future.
It’s a bit inconvenient for customers, since they can’t directly use a generic model and must perform an additional conversion step (similar to Rockchip’s RKNN). If this will be the mainstream method on IMX95, please let me know — I’ll need to communicate this to the customer in advance. Thank you.

[Zhiming_Liu]

Hi,

This is the standard approach. 

Zhiming_Liu_0-1761095740798.png

Best Regards,
Zhiming