NPU not working for iMX95

[emb-dev]

Hi All,

I am working on iMX95 EVK and while I try to run basic example for Mobilenetv1 model on NPU 

I am getting NeutronDelegate delegate: 0 nodes delegated out of 31 nodes with 0 partitions.

This means its not able to utilize the NPU even after I give  Neutron delegate.

 ./label_image -m mobilenet_v1_1.0_224_quant.tflite -i grace_hopper.bmp -l labels.txt --external_delegate_path=/usr/lib/libneutron_delegate.so
INFO: Loaded model mobilenet_v1_1.0_224_quant.tflite
INFO: resolved reporter
INFO: EXTERNAL delegate created.
INFO: NeutronDelegate delegate: 0 nodes delegated out of 31 nodes with 0 partitions.

INFO: Applied EXTERNAL delegate.
INFO: invoked
INFO: average time: 16.563 ms
INFO: 0.764706: 653 military uniform
INFO: 0.121569: 907 Windsor tie
INFO: 0.0156863: 458 bow tie
INFO: 0.0117647: 466 bulletproof vest
INFO: 0.00784314: 835 suit

Could anyone share why is that happening?

[Zhiming_Liu · Accepted solution]

Hello,

Please try the L6.6.36 images. I don't test other version but there is no such issue on L6.6.36 images.

root@imx95evk:/usr/bin/tensorflow-lite-2.16.2/examples# ./benchmark_model --graph=mobilenet_v1_1.0_224_quant.tflite  --external_delegate_path=/usr/lib/libneutron_delegate.so
INFO: STARTING!
INFO: Log parameter values verbosely: [0]
INFO: Graph: [mobilenet_v1_1.0_224_quant.tflite]
INFO: External delegate path: [/usr/lib/libneutron_delegate.so]
INFO: Loaded model mobilenet_v1_1.0_224_quant.tflite
INFO: EXTERNAL delegate created.
INFO: NeutronDelegate delegate: 29 nodes delegated out of 31 nodes with 1 partitions.

INFO: Explicitly applied EXTERNAL delegate, and the model graph will be partially executed by the delegate w/ 1 delegate kernels.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: The input model file size (MB): 4.27635
INFO: Initialized session in 65166.9ms.
INFO: Running benchmark for at least 1 iterations and at least 0.5 seconds but terminate if exceeding 150 seconds.
INFO: count=274 first=3027 curr=1831 min=1629 max=3027 avg=1776.68 std=102

INFO: Running benchmark for at least 50 iterations and at least 1 seconds but terminate if exceeding 150 seconds.
INFO: count=552 first=1843 curr=1842 min=1623 max=2180 avg=1762.19 std=69

INFO: Inference timings in us: Init: 65166931, First inference: 3027, Warmup (avg): 1776.68, Inference (avg): 1762.19
INFO: Note: as the benchmark tool itself affects memory footprint, the following is only APPROXIMATE to the actual memory footprint of the model at runtime. Take the information at your discretion.
INFO: Memory footprint delta from the start of the tool (MB): init=77.3242 overall=77.3242

Best Regards,
Zhiming