eIQ Tensorflow-lite c++ on NPU

[rbanks]

I am developing on an 8Mplus board and have had some issues with TensorFlow-Lite in c++.

I have my own TensorFlow-Lite model and want to utilize it in my c++ application. I have found that the eIQ software installed with Yocto doesn't come with a static library for TensorFlow-lite.

None of the guides describe how to build a Yocto image with a static library for TensorFlow-Lite. I've had to piece parts of several guides in order to even get the TensorFlow-Lite header files.

Is there a known step by step method to build a Yocto image of the imx 8Mplus dev board with a Static Library or some other method of linking the c++ TensorFlow-Lite binaries while allowing TensorFlow-Lite to utilize the NPU?

[Bio_TICFSL]

Hello rbanks.

I found this info on Yocto User Guide, could you please try adding the following line to your local.conf?

TOOLCHAIN_TARGET_TASK_append += " tensorflow-lite-staticdev tensorflow-lite-dev"

Please, let me know if this works or not for you

Regards

[rbanks]

Hi Bio_TICFSL,

My local.conf file looks as follow:

MACHINE ??= 'imx8mpevk'
DISTRO ?= 'fsl-imx-xwayland'
PACKAGE_CLASSES ?= 'package_rpm'
EXTRA_IMAGE_FEATURES ?= "dev-pkgs debug-tweaks tools-debug tools-sdk ssh-server-openssh"
IMAGE_INSTALL_append = "opencv git wget cmake"

TOOLCHAIN_TARGET_TASK += "tensorflow-lite-dev tensorflow-lite-staticdev"

USER_CLASSES ?= "buildstats image-mklibs image-prelink"
PATCHRESOLVE = "noop"
BB_DISKMON_DIRS ??= "\
STOPTASKS,${TMPDIR},1G,100K \
STOPTASKS,${DL_DIR},1G,100K \
STOPTASKS,${SSTATE_DIR},1G,100K \
STOPTASKS,/tmp,100M,100K \
ABORT,${TMPDIR},100M,1K \
ABORT,${DL_DIR},100M,1K \
ABORT,${SSTATE_DIR},100M,1K \
ABORT,/tmp,10M,1K"
PACKAGECONFIG_append_pn-qemu-system-native = " sdl"
CONF_VERSION = "1"

DL_DIR ?= "${BSPDIR}/downloads/"
ACCEPT_FSL_EULA = "1"

# Switch to Debian packaging and include package-management in the image
PACKAGE_CLASSES = "package_deb"
EXTRA_IMAGE_FEATURES += "package-management"


The line TOOLCHAIN_TARGET_TASK_append += " tensorflow-lite-staticdev tensorflow-lite-dev" only adds header files. The static library is  no where to be found or perhaps not named as expected.

[Bio_TICFSL]

I understand you have a package that you are building against the tensorflow-lite, right?

Are you using DEPENDS += "tensorflow-lite" inside the recipe of your package?

Regards

[Bio_TICFSL]

I understand you have a package that you are building against the tensorflow-lite, right?

Are you using DEPENDS += "tensorflow-lite" inside the recipe of your package?

Regards

[Bio_TICFSL]

It seems that the TensorFlow libraries are in the Yocto project but it is not integrated into the rootfs.

Could you please check in your build project at this path?

bld-xwayland/tmp/work/aarch64-poky-linux/tensorflow-lite/2.2.0-r0/git/tensorflow/lite/tools/make/gen/linux_aarch64/lib/libtensorflow-lite.a

Please, let me know!

[rbanks]

So what does that mean? After I run bitbake on the build directory the resulting image is missing libtensorflow-lite.a. Are you implying that I can copy that static library to the image?

Is that static library compiled to use the NPU?

[Bio_TICFSL]

libtensorflow-lite.a does exist in the build directory at that location.

[rbanks]

libtensorflow-lite.a does exist in the build directory at that location.

[Bio_TICFSL]

That basically means that *.a files are not intended to be installed on a image rootfs, as they are development files only.

The *.a files should be part of the -dev packages, or installed on a SDK image (toolchain)

That's why it's important to understand how you are integrating the package recipe, or if they are working on a native build system, or developing the package only using the SDK (toolchain)

Could you, please confirm which tensorflow-lite recipe are you using on your tests?

Would be this one?

https://source.codeaurora.org/external/imx/meta-imx/tree/meta-ml/recipes-libraries/tensorflow-lite/t...

I want to test locally to make sure the recipe does not have a bug on that installation. Could you, please, confirm that?

Regards

[Bio_TICFSL]

That basically means that *.a files are not intended to be installed on a image rootfs, as they are development files only.

The *.a files should be part of the -dev packages, or installed on a SDK image (toolchain)

That's why it's important to understand how you are integrating the package recipe, or if they are working on a native build system, or developing the package only using the SDK (toolchain)

Could you, please confirm which tensorflow-lite recipe are you using on your tests?

Would be this one?

https://source.codeaurora.org/external/imx/meta-imx/tree/meta-ml/recipes-libraries/tensorflow-lite/t...

I want to test locally to make sure the recipe does not have a bug on that installation. Could you, please, confirm that?

Regards

[rbanks]

Were you able to verify that the recipe doesn't have a bug?

[Bio_TICFSL]

Yes, it don´t have a bug.

[rbanks]

So when you build the recipe and then flash an SD card, the libtensorflow-lite.a file exists in the image?

If that is true, can you provide the step by step solution that you used to get there? Nothing I try has given me the desired results.

[rbanks]

I think I understand what you are getting at.

On my native build machine at yocto_build/source/meta-imx/meta-ml/recipes-libraries/tensorflow-lite exists the file tensorflow-lite_2.2.0.bb. I did a file compare to the one you linked and they are identical.

I am also using Ubuntu 18.04 but I would hope that that isn't the problem as I don't want to have to spend the day or so rebuilding my environment in Ubuntu 16.04.

Thanks for your help on this.

[rbanks]

I have my own c++ code that utilizes the TensorFlow-Lite static library. I need to be able to run make on my code which requires the tensorflow-lite static library. I'm rather new to Yocto. Where would I add DEPENDS += "tensorflow-lite"?

[Bio_TICFSL · Accepted solution]

After some builds and some search, I found this:

https://stackoverflow.com/questions/55125977/how-to-build-tensorflow-lite-as-a-static-library-and-li...

I can also see this:

yocto/imx-5.4.47-2.2.0/bld-xwayland$ find -name libtensorflow*a
./tmp/work/aarch64-poky-linux/tensorflow-lite/2.2.0-r0/git/tensorflow/lite/tools/make/gen/linux_aarch64/lib/libtensorflow-lite.a
./tmp/work/aarch64-poky-linux/tensorflow-lite/2.2.0-r0/sysroot-destdir/usr/lib/libtensorflow-lite.a
./tmp/work/aarch64-poky-linux/tensorflow-lite/2.2.0-r0/image/usr/lib/libtensorflow-lite.a
./tmp/work/aarch64-poky-linux/tensorflow-lite/2.2.0-r0/package/usr/lib/libtensorflow-lite.a
./tmp/work/aarch64-poky-linux/tensorflow-lite/2.2.0-r0/packages-split/tensorflow-lite-staticdev/usr/lib/libtensorflow-lite.a
./tmp/sysroots-components/aarch64/tensorflow-lite/usr/lib/libtensorflow-lite.a

so, the libtensorflow-lite.a is being installed when you install the tensorflow-lite-staticdev lib. However from the original message "I have my own TensorFlow-Lite model"

It might be that some adjustments should be done in the CMake environment so that all the static libs needed are being built/created.

Could you, please, share exactly which is the error log?

Regards

[rbanks]

Sorry it took so long to get back to this. There seems to be some discrepancy between what you mean and what I mean. I was trying to get the static library on the end device. You are talking about building my app on a host computer. Either way is fine as long as the static library is compiled to take advantage of the end device hardware.