i.MX ProcessorsのトピックRe: IMX 95 NPU not activated

IMX 95 NPU not activated

OlivierG — Thu, 25 Sep 2025 08:58:21 GMT

I use a IMX 95 Verdin Toradex Card a1 (not b0)

I would like to evaluate the NPU

if i launch :
/usr/bin/tensorflow-lite-2.18.0/examples# python3 ./label_image.py --ext_delegate /usr/lib/libneutron_delegate.so

i get the following messages

Error in cpuinfo: prctl(PR_SVE_GET_VL) failed
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.

Is there a connection between these two messages?
Should I conclude that the NPU is not enabled?
What should I do to resolve this issue?

Greetings

Olivier

Re: IMX 95 NPU not activated

Zhiming_Liu — Thu, 25 Sep 2025 23:51:38 GMT

Hi,

Is there a connection between these two messages?

-->There is no connection about them.

Should I conclude that the NPU is not enabled?

-->If you can see below log, that means NPU delegate is working.

INFO: NeutronDelegate delegate: .......

What should I do to resolve this issue?

-->Can you share full log?

Best Regards,
Zhiming

Re: IMX 95 NPU not activated

OlivierG — Fri, 26 Sep 2025 07:23:31 GMT

on IMX 95 i get

python3 label_image.py --ext_delegate /usr/lib/libneutron_delegate.so

Loading external delegate from libneutron_delegate.so with args: {}
ext_delegate [<tflite_runtime.interpreter.Delegate object at 0xffffa6d52660>]
INFO: NeutronDelegate delegate: 0 nodes delegated out of 31 nodes with 0 partitions.

Error in cpuinfo: prctl(PR_SVE_GET_VL) failed
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Warm-up time: 122.3 ms

Inference time: 118.2 ms

0.870588: military uniform
0.031373: Windsor tie
0.011765: mortarboard
0.007843: bow tie
0.007843: bulletproof vest

in comparison, on IMX 93 i get

python3 label_image.py --ext_delegate /usr/lib/libethosu_delegate.so
Loading external delegate from /usr/lib/libethosu_delegate.so with args: {}
INFO: Ethosu delegate: device_name set to /dev/ethosu0.
INFO: Ethosu delegate: cache_file_path set to .
INFO: Ethosu delegate: timeout set to 60000000000.
INFO: Ethosu delegate: enable_cycle_counter set to 0.
INFO: Ethosu delegate: enable_profiling set to 0.
INFO: Ethosu delegate: profiling_buffer_size set to 2048.
INFO: Ethosu delegate: pmu_event0 set to 0.
INFO: Ethosu delegate: pmu_event1 set to 0.
INFO: Ethosu delegate: pmu_event2 set to 0.
INFO: Ethosu delegate: pmu_event3 set to 0.
INFO: EthosuDelegate: 31 nodes delegated out of 31 nodes with 1 partitions.
Warm-up time: 4.1 ms

Inference time: 3.9 ms

0.874510: military uniform
0.031373: Windsor tie
0.015686: mortarboard
0.011765: bulletproof vest
0.007843: bow tie

on imx 93 there is no message:

INFO: Created TensorFlow Lite XNNPACK delegate for CPU.

The inference time is much shorter 118.2 ms -> 3.9 ms

Olivier

Re: IMX 95 NPU not activated

Zhiming_Liu — Tue, 30 Sep 2025 01:32:15 GMT

Hi @OlivierG

Please use older verison with tensorflow 2.16.2, like L6.6.X.

root@imx95frdm:/usr/bin/tensorflow-lite-2.16.2/examples# python3 label_image.py --ext_delegate /usr/lib/libneutron_delegate.so Loading external delegate from /usr/lib/libneutron_delegate.so with args: {} INFO: NeutronDelegate delegate: 29 nodes delegated out of 31 nodes with 1 partitions. INFO: Created TensorFlow Lite XNNPACK delegate for CPU. Warm-up time: 3.9 ms Inference time: 1.8 ms 0.850980: military uniform 0.058824: Windsor tie 0.011765: bulletproof vest 0.007843: bow tie

There are some issue about NPU delegate even in latest BSP L6.12.34

Best Regards,
Zhiming

Re: IMX 95 NPU not activated

OlivierG — Tue, 30 Sep 2025 09:39:18 GMT

i try the folowing python code

import tensorflow as tf

modelpath ='model_yolo_11s_int8_fp32.tflite'

with open(modelpath, 'rb') as fid:
tflite_model = fid.read()

interpreter = tf.lite.Interpreter(model_content=tflite_model,experimental_delegates=[tf.lite.experimental.load_delegate('libneutron_delegate.so')])
interpreter.allocate_tensors()

i get the following error message

/usr/lib/python3.13/site-packages/tensorflow/lite/python/interpreter.py:457: UserWarning: Warning: tf.lite.Interpreter is deprecated and is scheduled for deletion in
TF 2.20. Please use the LiteRT interpreter from the ai_edge_litert package.
See the [migration guide](https://ai.google.dev/edge/litert/migration)
for details.

warnings.warn(_INTERPRETER_DELETION_WARNING)
INFO: NeutronDelegate delegate: 0 nodes delegated out of 339 nodes with 0 partitions.

INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Exception ignored in: <function Delegate.__del__ at 0xffff5171bf60>
Traceback (most recent call last):
File "/usr/lib/python3.13/site-packages/tensorflow/lite/python/interpreter.py", line 122, in __del__
AttributeError: 'NoneType' object has no attribute 'c_void_p'

Does the python code match your answer?
i precise that libethosu_delegate.so is not on my Card

Greetings

Olivier

Re: IMX 95 NPU not activated

Zhiming_Liu — Thu, 09 Oct 2025 01:04:57 GMT

Hi,

Yes, the sw team is investigating this issue.

Best Regards,
Zhiming

Re: IMX 95 NPU not activated

Zhiming_Liu — Tue, 21 Oct 2025 05:27:18 GMT

Hi,

Please download eIQ to convert .tflite with Neutron support.

Zhiming_Liu_0-1761024034289.png

The eIQ1.16 is for L6.12.20 Neutron support.

Zhiming_Liu_1-1761024318029.png

For Neutron Converter CLI, please refer eIQ User Guide.

Zhiming_Liu_0-1761024419102.png

Best Regards,
Zhiming

Re: IMX 95 NPU not activated

OlivierG — Tue, 28 Oct 2025 12:48:31 GMT

now, i have a imx95-19x19-lpddr5-evk

The Yocto is
i.MX Linux Yocto Project BSP 6.12.20_2.0.0 Release

When i launch
root@imx95evk:/usr/bin/tensorflow-lite-2.18.0/examples# ./benchmark_model --graph=mobilenet_v1_1.0_224_quant.tflite --external_delegate_path=/usr/lib/libneutron_delegate.so
INFO: STARTING!
INFO: Log parameter values verbosely: [0]
INFO: Graph: [mobilenet_v1_1.0_224_quant.tflite]
INFO: Signature to run: []
INFO: External delegate path: [/usr/lib/libneutron_delegate.so]
INFO: Loaded model mobilenet_v1_1.0_224_quant.tflite
INFO: EXTERNAL delegate created.
INFO: NeutronDelegate delegate: 0 nodes delegated out of 31 nodes with 0 partitions.

INFO: Though EXTERNAL delegate is explicitly applied, the model graph will not be executed by the delegate.
Error in cpuinfo: prctl(PR_SVE_GET_VL) failed
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: The input model file size (MB): 4.27635
INFO: Initialized session in 56.257ms.
INFO: Running benchmark for at least 1 iterations and at least 0.5 seconds but terminate if exceeding 150 seconds.
INFO: count=5 first=122578 curr=118671 min=118400 max=122578 avg=119396 std=1595

INFO: Running benchmark for at least 50 iterations and at least 1 seconds but terminate if exceeding 150 seconds.
INFO: count=50 first=118459 curr=117971 min=117954 max=118782 avg=118259 std=173

INFO: Inference timings in us: Init: 56257, First inference: 122578, Warmup (avg): 119396, Inference (avg): 118259
INFO: Note: as the benchmark tool itself affects memory footprint, the following is only APPROXIMATE to the actual memory footprint of the model at runtime. Take the information at your discretion.
INFO: Memory footprint delta from the start of the tool (MB): init=12.0352 overall=15.1602

when i launch
cat /sys/class/remoteproc/remoteproc0/state
i get : offline

Greetings

Olivier

Re: IMX 95 NPU not activated

Zhiming_Liu — Wed, 29 Oct 2025 00:55:37 GMT

Hi @OlivierG

Please refer previous reply here.

Zhiming_Liu_0-1761699158616.png

To use NPU in i.MX95, customer need to convert common .tflite to neutron .tflite support with eIQ or neutron converter CLI.

Here you can try this model which i converted with eIQ.

Best Regards,
Zhiming

Re: IMX 95 NPU not activated

OlivierG — Wed, 29 Oct 2025 08:22:13 GMT

When i launch the python sample with the converted file, i get

root@imx95evk:/usr/bin/tensorflow-lite-2.18.0/examples# python3 label_image.py -m mobilenet_v1_1.0_224_quant_converted.tflite
Error in cpuinfo: prctl(PR_SVE_GET_VL) failed
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Traceback (most recent call last):
File "/usr/bin/tensorflow-lite-2.18.0/examples/label_image.py", line 92, in <module>
interpreter.allocate_tensors()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/usr/lib/python3.13/site-packages/tflite_runtime/interpreter.py", line 534, in allocate_tensors
return self._interpreter.AllocateTensors()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
RuntimeError: Encountered unresolved custom op: NeutronGraph.
See instructions: https://www.tensorflow.org/lite/guide/ops_custom Node number 1 (NeutronGraph) failed to prepare.Encountered unresolved custom op: NeutronGraph.
See instructions: https://www.tensorflow.org/lite/guide/ops_custom Node number 1 (NeutronGraph) failed to prepare.

With the executable version i get
root@imx95evk:/usr/bin/tensorflow-lite-2.18.0/examples# ./label_image -m mobilenet_v1_1.0_224_quant_converted.tflite
INFO: Loaded model mobilenet_v1_1.0_224_quant_converted.tflite
INFO: resolved reporter
Error in cpuinfo: prctl(PR_SVE_GET_VL) failed
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
ERROR: Encountered unresolved custom op: NeutronGraph.
See instructions: https://www.tensorflow.org/lite/guide/ops_custom
ERROR: Node number 1 (NeutronGraph) failed to prepare.
ERROR: Encountered unresolved custom op: NeutronGraph.
See instructions: https://www.tensorflow.org/lite/guide/ops_custom
ERROR: Node number 1 (NeutronGraph) failed to prepare.
ERROR: Failed to allocate tensors!

Greetings

Olivier

Re: IMX 95 NPU not activated

min-eta — Wed, 29 Oct 2025 11:07:49 GMT

Hi Zhiming, your reply to this message would seem to indicate that in the older Linux version 6.6.x, to use the NeutronDelegate, you DO NOT need to first use the neutron_converter to convert some of the TFLite Ops in the Model to the custom NeutronGraph.

But for the newer Linux 6.12.x, you DO have to first use neutron_converter to convert the Model first?

@min-eta

Re: IMX 95 NPU not activated

Zhiming_Liu — Thu, 30 Oct 2025 00:19:39 GMT

Hi @min-eta

Yes, the software framework has changed since L6.12.y

Best Regards,
Zhiming

Re: IMX 95 NPU not activated

Zhiming_Liu — Thu, 30 Oct 2025 00:20:38 GMT

Hi @OlivierG

Can you try to test with neutron delegate /usr/lib/libneutron_delegate.so ?

Best Regards,
Zhiming

Re: IMX 95 NPU not activated

OlivierG — Thu, 30 Oct 2025 08:36:57 GMT

I build the last version of the yocto kernel imx-6.12.34-2.1.0

when i launch
python3 label_image.py -e /usr/lib/libneutron_delegate.so
i get
Loading external delegate from /usr/lib/libneutron_delegate.so with args: {}
/usr/lib/python3.13/site-packages/tflite_runtime/interpreter.py:457: UserWarning: Warning: tf.lite.Interpreter is deprecated and is scheduled for deletion in
TF 2.20. Please use the LiteRT interpreter from the ai_edge_litert package.
See the [migration guide](https://ai.google.dev/edge/litert/migration)
for details.

warnings.warn(_INTERPRETER_DELETION_WARNING)
INFO: NeutronDelegate delegate: 0 nodes delegated out of 31 nodes with 0 partitions.

INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Warm-up time: 94.5 ms

Inference time: 60.2 ms

0.878431: military uniform
0.027451: Windsor tie
0.011765: mortarboard
0.011765: bulletproof vest
0.007843: sax

i modify the python code to correct the warning

python3 label_image2.py -e /usr/lib/libneutron_delegate.so
i get
Loading external delegate from /usr/lib/libneutron_delegate.so with args: {}
INFO: NeutronDelegate delegate: 0 nodes delegated out of 31 nodes with 0 partitions.

INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Warm-up time: 58.0 ms

Inference time: 55.5 ms

0.878431: military uniform
0.027451: Windsor tie
0.011765: mortarboard
0.011765: bulletproof vest
0.007843: sax

i always get
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.

i must precise that on IMX95
cat /sys/class/remoteproc/remoteproc0/state
return : offline

The same command on the IMX93 board return : running

On IMX95
dmesg | grep remoteproc

return
remoteproc remoteproc0: imx-rproc is available

On IMX93
dmesg | grep remoteproc
return
remoteproc remoteproc0: imx-rproc is available
remoteproc remoteproc0: powering up imx-rproc
remoteproc remoteproc0: Booting fw image ethosu_firmware, size 244744
remoteproc remoteproc0: remote processor imx-rproc is now up

i think that the NPU is not activated

Greetings

Olivier

Re: IMX 95 NPU not activated

OlivierG — Thu, 30 Oct 2025 08:40:06 GMT

On IMX95
dmesg | grep remoteproc

return
remoteproc remoteproc0: neutron-rproc is available
remoteproc remoteproc1: imx-rproc is available
remoteproc remoteproc1: attaching to imx-rproc
remoteproc remoteproc1: remote processor imx-rproc is now attached

remoteproc remoteproc0: powering up neutron-rproc
remoteproc remoteproc0: Booting fw image NeutronFirmware.elf, size 66896
remoteproc remoteproc0: remote processor neutron-rproc is now up
remoteproc remoteproc0: stopped remote processor neutron-rproc

Re: IMX 95 NPU not activated

Zhiming_Liu — Fri, 31 Oct 2025 06:27:55 GMT

Hi @OlivierG

Please try to use this model in attachment, this model is converted from L6.12.34 neutron converter.

root@imx95evk:/usr/bin/tensorflow-lite-2.19.0/examples# ./label_image -m mobilenet_v1_1.0_224_int8_converted.tflite --external_delegate_path=/lib/libneutron_delegate.so -i grace_hopper.bmp -l labels.txt INFO: Loaded model mobilenet_v1_1.0_224_int8_converted.tflite INFO: resolved reporter INFO: EXTERNAL delegate created. INFO: NeutronDelegate delegate: 1 nodes delegated out of 4 nodes with 1 partitions. INFO: Neutron delegate version: v1.0.0-be8bf399, zerocp enabled. INFO: Applied EXTERNAL delegate. INFO: Created TensorFlow Lite XNNPACK delegate for CPU. INFO: invoked INFO: average time: 1.388 ms INFO: 0.756863: 653 military uniform INFO: 0.145098: 907 Windsor tie INFO: 0.0156863: 458 bow tie INFO: 0.0117647: 466 bulletproof vest INFO: 0.00784314: 835 suit

Best Regards,
Zhiming