NPU accelaration on i.MX93

[mohamed_gaseen]

Hello,

I'm currently working on accelerating Neural Processing Unit(NPU) on my i.MX93EVK board Revision 6.6.3 Yocto binary flashed on it.

As I need to run the tasks such as face recognition, speech processing and more AI & Machine learning tasks to be run on the NPU instead of CPU as it consumes lower power & gives better performance than running on the CPU & GPU based solutions.

Specifically I need to run the tasks such as Qt graphics demo, GUI guider graphics, eIQ face recognition demo and VIT wake word detection tasks on the NPU hardware accelerator.

Can you please share me a way to run the above mentioned tasks on NPU on my i.MX93 board.

Thanks in Advance..

[Chavira]

Hi @mohamed_gaseen!
Thank you for contacting NXP Support!

The NPU of the iMX93 only has support for Machine Learning tasks.

The iMX93 has a PXP IP that process the graphics generated by GUI GUIDER.

You can refer to our Machine Learning User's Guide to download and run our examples.

Best Regards!

Chavira

[mohamed_gaseen]

Thank you for the reply @Chavira 

Whatever the example models I'm running, either it is directly passing through the CPU or else it is getting failed.

Can you share me any procedure to run any of the model.

Any example is enough to get the flow of the NPU configuration.

I'm working on face recognition as the requirement.

[Chavira]

HI @mohamed_gaseen!

You can consult our examples in the link below:

https://github.com/nxp-imx/eiq-example/tree/lf-6.6.3_1.0.0/face_recognition

Best Regards!

Chavira

[mohamed_gaseen]

Hey @Chavira 

I've ran the main.py code as you mentioned, But the issue is while running the processor is waiting for NPU initially & taking over the CPU at last.

The command I've ran:

The log:

Thanks for your reply @Chavira 

I've and ran the object detection python code as you mentioned

# The command I ran

python3 main.py -i /dev/video0 -d /usr/lib/libethosu_delegate.so

#The log

root@imx93AMT:/usr/bin/eiq-examples-git/object_detection# python3 main.py -i /dev/video0 -d /usr/lib/libethosu_delegate.so
[ WARN:0@0.190] global cap_gstreamer.cpp:2784 handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module source reported: Could not read from resource.
[ WARN:0@0.191] global cap_gstreamer.cpp:1679 open OpenCV | GStreamer warning: u
nable to start pipeline
[ WARN:0@0.191] global cap_gstreamer.cpp:1164 isPipelinePlaying OpenCV | GStream
er warning: GStreamer: pipeline have not been created
INFO: Ethosu delegate: device_name set to /dev/ethosu0.
INFO: Ethosu delegate: cache_file_path set to .
INFO: Ethosu delegate: timeout set to 60000000000.
INFO: Ethosu delegate: enable_cycle_counter set to 0.
INFO: Ethosu delegate: enable_profiling set to 0.
INFO: Ethosu delegate: profiling_buffer_size set to 2048.
INFO: Ethosu delegate: pmu_event0 set to 0.
INFO: Ethosu delegate: pmu_event1 set to 0.
INFO: Ethosu delegate: pmu_event2 set to 0.
INFO: Ethosu delegate: pmu_event3 set to 0.
INFO: EthosuDelegate: 63 nodes delegated out of 64 nodes with 1 partitions.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
rectangle:(146,241),(223,433) label:person
rectangle:(142,243),(222,436) label:person
rectangle:(88,247),(220,471) label:person


INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
rectangle:(2,357),(119,476) label:person
rectangle:(2,356),(122,475) label:person
rectangle:(2,357),(124,477) label:person
rectangle:(2,367),(102,479) label:person
rectangle:(2,367),(102,479) label:person
rectangle:(2,369),(102,480) label:person
rectangle:(2,369),(101,480) label:person
rectangle:(2,369),(102,480) label:person
rectangle:(2,368),(103,480) label:person
rectangle:(2,368),(104,480) label:person
rectangle:(2,367),(103,479) label:person
^Cobject address : 0xffff885f33a0
object refcount : 2125,474) label:person
object type : 0xffff956a9400l:person
object type name: RuntimeErrorbel:person
object repr : RuntimeError('Failed to invoke ethos_u op.\nNode number 64 (Et
hosuDelegate) failed to invoke.'):person
lost sys.stderr3),(114,468) label:person
root@imx93AMT:/usr/bin/eiq-examples-git/object_detection#
rectangle:(2,368),(104,480) label:person
rectangle:(2,362),(114,469) label:person

Please verify my logs & support me. I'm facing the same issue while running any models through NPU.

[Chavira]

Hi @mohamed_gaseen!

Try using the latest BSP version 6.6.52, the machine learning examples are working better with the latest versions of our BSP, I have ran the example without problems.

[mohamed_gaseen]

Okay @Chavira ,

I'm currently working on the BSP version 6.6.3 with all the functionalities and requirements.

I'll run the examples on version 6.6.52 as u said and let you know further.

Also please let me know if there's any way to run the examples on 6.6.3 Binary without CPU overtaking the NPU after delegation.

Thank you for the reply.