NPU usage for AI models

[Mohan51]

Hi, I incorporated the hardware accelerator for using npu in IMX93 board.but still it is using cpu.could you please correct me.if i am doing any wrong while implementing.I am attaching the logs and code snippet of npu implementation.please assist me.

[Mohan51]

still i am not able to use the npu.i converted my currency tflite model with vela on board and used the same by passing delegate(present in /usr/lib/libethosu_delegate.so). still the program using NPU.can i find any sample tflite models from nxp that uses npu?

[Chavira]

Hi @Mohan51!

Thank you for contacting NXP Support!

You can refer to the next document:

https://www.nxp.com/docs/en/user-guide/IMX-MACHINE-LEARNING-UG.pdf

That document explains how to implement your tensor flow lite models and run them into the NPU.

Best Regards!

Chavira

[Mohan51]

Hi, Currently i am using tflite model that runs on cpu, while referring IMX machine learning guide,i came to know that we need to convert tflite using vela tool for npu usage.but while installing vela i am facing issues.Can i convert the tflite models using IMX EIQ tool for npu usage?

Note: I flashed the latest BSP full image. using this command :

sudo uuu -b emmc_all imx-boot-imx93evk-sd.bin-flash_singleboot imx-image-full-imx93evk.wic

[Chavira]

Hi @Mohan51!

Yes, we have some examples on our BSP.

Please check the chapter "7.2.3 Getting Started" of the Machine Learning User Guide, I can use vela and the NPU.

https://www.nxp.com/docs/en/user-guide/IMX-MACHINE-LEARNING-UG.pdf

Best Regards!

Chavira

[Mohan51]

Hi, I implemented the examples and worked out,but my own tflite model is not utilizing npu, i compiled the tflite model with vela tool. is there any specific parameters to train a model for utilizing npu?

[Chavira]

Hi @Mohan51!

Ethos-U Delegate is an external delegate on i.MX 93 Linux platforms. It enables accelerating the inference on the on-chip hardware accelerator. The Ethos-U Delegate directly uses the hardware accelerator driver (Ethos-U driver stack) to fully utilize the accelerator capabilities.

The Ethos-U Delegate is available as an external delegate. The corresponding library is available in /usr/lib/libethosu_delegate.so.

Ethos-U Delegate is supported in both C++ and Python API. For using Ethos-U Delegate (or any external
delegate), see the external_delegate_provider implementation in C++ and/or label_image.py for Python.

A list of supported operators is available in https://github.com/nxp-imx/ethos-u-vela/blob/lf-6.6.3_1.0.0/SUPPORTED_OPS.md

Best Regards!

Chavira

[Chavira]

Hi @Mohan51!

You can do the process on the board, but please ensure that you are using the full image.

In the full image, you have all the needed packages.

Best Regards!

Chavira