topic Re: eIQ Tensorflow-lite c++ on NPU in i.MX Processors

eIQ Tensorflow-lite c++ on NPU

rbanks — Tue, 17 Nov 2020 17:40:10 GMT

I am developing on an 8Mplus board and have had some issues with TensorFlow-Lite in c++.

I have my own TensorFlow-Lite model and want to utilize it in my c++ application. I have found that the eIQ software installed with Yocto doesn't come with a static library for TensorFlow-lite.

None of the guides describe how to build a Yocto image with a static library for TensorFlow-Lite. I've had to piece parts of several guides in order to even get the TensorFlow-Lite header files.

Is there a known step by step method to build a Yocto image of the imx 8Mplus dev board with a Static Library or some other method of linking the c++ TensorFlow-Lite binaries while allowing TensorFlow-Lite to utilize the NPU?

Re: eIQ Tensorflow-lite c++ on NPU

Bio_TICFSL — Tue, 24 Nov 2020 14:07:32 GMT

Hello rbanks.

I found this info on Yocto User Guide, could you please try adding the following line to your local.conf?

TOOLCHAIN_TARGET_TASK_append += " tensorflow-lite-staticdev tensorflow-lite-dev"

Please, let me know if this works or not for you

Regards

Re: eIQ Tensorflow-lite c++ on NPU

rbanks — Mon, 30 Nov 2020 15:57:34 GMT

Hi Bio_TICFSL,

My local.conf file looks as follow:

MACHINE ??= 'imx8mpevk'
DISTRO ?= 'fsl-imx-xwayland'
PACKAGE_CLASSES ?= 'package_rpm'
EXTRA_IMAGE_FEATURES ?= "dev-pkgs debug-tweaks tools-debug tools-sdk ssh-server-openssh"
IMAGE_INSTALL_append = "opencv git wget cmake"

TOOLCHAIN_TARGET_TASK += "tensorflow-lite-dev tensorflow-lite-staticdev"

USER_CLASSES ?= "buildstats image-mklibs image-prelink"
PATCHRESOLVE = "noop"
BB_DISKMON_DIRS ??= "\
STOPTASKS,${TMPDIR},1G,100K \
STOPTASKS,${DL_DIR},1G,100K \
STOPTASKS,${SSTATE_DIR},1G,100K \
STOPTASKS,/tmp,100M,100K \
ABORT,${TMPDIR},100M,1K \
ABORT,${DL_DIR},100M,1K \
ABORT,${SSTATE_DIR},100M,1K \
ABORT,/tmp,10M,1K"
PACKAGECONFIG_append_pn-qemu-system-native = " sdl"
CONF_VERSION = "1"

DL_DIR ?= "${BSPDIR}/downloads/"
ACCEPT_FSL_EULA = "1"

# Switch to Debian packaging and include package-management in the image
PACKAGE_CLASSES = "package_deb"
EXTRA_IMAGE_FEATURES += "package-management"

The line TOOLCHAIN_TARGET_TASK_append += " tensorflow-lite-staticdev tensorflow-lite-dev" only adds header files. The static library is no where to be found or perhaps not named as expected.

Re: eIQ Tensorflow-lite c++ on NPU

Bio_TICFSL — Tue, 01 Dec 2020 15:04:43 GMT

I understand you have a package that you are building against the tensorflow-lite, right?

Are you using DEPENDS += "tensorflow-lite" inside the recipe of your package?

Regards

Re: eIQ Tensorflow-lite c++ on NPU

rbanks — Tue, 01 Dec 2020 22:08:21 GMT

I have my own c++ code that utilizes the TensorFlow-Lite static library. I need to be able to run make on my code which requires the tensorflow-lite static library. I'm rather new to Yocto. Where would I add DEPENDS += "tensorflow-lite"?

Re: eIQ Tensorflow-lite c++ on NPU

Bio_TICFSL — Wed, 02 Dec 2020 13:59:17 GMT

It seems that the TensorFlow libraries are in the Yocto project but it is not integrated into the rootfs.

Could you please check in your build project at this path?

bld-xwayland/tmp/work/aarch64-poky-linux/tensorflow-lite/2.2.0-r0/git/tensorflow/lite/tools/make/gen/linux_aarch64/lib/libtensorflow-lite.a

Please, let me know!

Re: eIQ Tensorflow-lite c++ on NPU

rbanks — Wed, 02 Dec 2020 15:54:27 GMT

libtensorflow-lite.a does exist in the build directory at that location.

Re: eIQ Tensorflow-lite c++ on NPU

Bio_TICFSL — Wed, 02 Dec 2020 16:09:21 GMT

libtensorflow-lite.a does exist in the build directory at that location.

Re: eIQ Tensorflow-lite c++ on NPU

rbanks — Wed, 02 Dec 2020 20:51:53 GMT

So what does that mean? After I run bitbake on the build directory the resulting image is missing libtensorflow-lite.a. Are you implying that I can copy that static library to the image?

Is that static library compiled to use the NPU?

Re: eIQ Tensorflow-lite c++ on NPU

Bio_TICFSL — Thu, 03 Dec 2020 15:13:19 GMT

That basically means that *.a files are not intended to be installed on a image rootfs, as they are development files only.

The *.a files should be part of the -dev packages, or installed on a SDK image (toolchain)

That's why it's important to understand how you are integrating the package recipe, or if they are working on a native build system, or developing the package only using the SDK (toolchain)

Could you, please confirm which tensorflow-lite recipe are you using on your tests?

Would be this one?

https://source.codeaurora.org/external/imx/meta-imx/tree/meta-ml/recipes-libraries/tensorflow-lite/t...

I want to test locally to make sure the recipe does not have a bug on that installation. Could you, please, confirm that?

Regards

Re: eIQ Tensorflow-lite c++ on NPU

rbanks — Thu, 03 Dec 2020 16:10:28 GMT

I think I understand what you are getting at.

On my native build machine at yocto_build/source/meta-imx/meta-ml/recipes-libraries/tensorflow-lite exists the file tensorflow-lite_2.2.0.bb. I did a file compare to the one you linked and they are identical.

I am also using Ubuntu 18.04 but I would hope that that isn't the problem as I don't want to have to spend the day or so rebuilding my environment in Ubuntu 16.04.

Thanks for your help on this.

Re: eIQ Tensorflow-lite c++ on NPU

rbanks — Wed, 09 Dec 2020 20:15:29 GMT

Were you able to verify that the recipe doesn't have a bug?

Re: eIQ Tensorflow-lite c++ on NPU

Bio_TICFSL — Thu, 10 Dec 2020 14:17:48 GMT

Yes, it don´t have a bug.

Re: eIQ Tensorflow-lite c++ on NPU

rbanks — Thu, 10 Dec 2020 15:25:29 GMT

So when you build the recipe and then flash an SD card, the libtensorflow-lite.a file exists in the image?

If that is true, can you provide the step by step solution that you used to get there? Nothing I try has given me the desired results.

Re: eIQ Tensorflow-lite c++ on NPU

Bio_TICFSL — Mon, 21 Dec 2020 17:00:38 GMT

After some builds and some search, I found this:

https://stackoverflow.com/questions/55125977/how-to-build-tensorflow-lite-as-a-static-library-and-li...

I can also see this:

yocto/imx-5.4.47-2.2.0/bld-xwayland$ find -name libtensorflow*a
./tmp/work/aarch64-poky-linux/tensorflow-lite/2.2.0-r0/git/tensorflow/lite/tools/make/gen/linux_aarch64/lib/libtensorflow-lite.a
./tmp/work/aarch64-poky-linux/tensorflow-lite/2.2.0-r0/sysroot-destdir/usr/lib/libtensorflow-lite.a
./tmp/work/aarch64-poky-linux/tensorflow-lite/2.2.0-r0/image/usr/lib/libtensorflow-lite.a
./tmp/work/aarch64-poky-linux/tensorflow-lite/2.2.0-r0/package/usr/lib/libtensorflow-lite.a
./tmp/work/aarch64-poky-linux/tensorflow-lite/2.2.0-r0/packages-split/tensorflow-lite-staticdev/usr/lib/libtensorflow-lite.a
./tmp/sysroots-components/aarch64/tensorflow-lite/usr/lib/libtensorflow-lite.a

so, the libtensorflow-lite.a is being installed when you install the tensorflow-lite-staticdev lib. However from the original message "I have my own TensorFlow-Lite model"

It might be that some adjustments should be done in the CMake environment so that all the static libs needed are being built/created.

Could you, please, share exactly which is the error log?

Regards

Re: eIQ Tensorflow-lite c++ on NPU

rbanks — Tue, 09 Mar 2021 16:53:22 GMT

Sorry it took so long to get back to this. There seems to be some discrepancy between what you mean and what I mean. I was trying to get the static library on the end device. You are talking about building my app on a host computer. Either way is fine as long as the static library is compiled to take advantage of the end device hardware.