Arm NN support for the i.MX 8 GPUs

shmulikh · ‎01-16-2020

Hello,

Since the release of document number UM11226 - "NXP eIQ™ Machine Learning Software Development Environment for i.MX Applications Processors", on 06/2019, has there been any change to the status of the following statement on page 25 - "Arm NN does not currently support the i.MX 8 GPUs due to the Arm NN OpenCL requirements, which are not met by i.MX 8 GPUs." ?

What family of GPUs is that, and why are they not supported by OpenCL?

Perhaps a new version of OpenCL or ArmNN has been released that does support i.MX 8 GPUs ?

Thanks in advance,

Shmulik Hen.

vanessa_maegima · ‎01-16-2020

Hi shmulikh@foresightauto.com,

Since L4.19 releases, eIQ has been integrated into the NXP BSP so it's not an standalone release anymore.

Please refer to the latest L4.19.35_1.1.0 Documentation (Linux User Guide) for updated information on eIQ support. You can see that only TfLite engine supports GPU at this time. ArmNN still doesn't support GPU in the latest release.

Thanks,

Vanessa

damiano_ciprian · ‎01-27-2020

Hi vanessamaegima‌,

I am trying to run FaceRecognition on imx8qmmek, but applications crashes and i need support to understand where is the problem. Here i explain all steps i've made:

i have build last Yocto BSP release for i.MX machines, L4.19.35_1.1.0
i have applied your patch eiq-sample-apps-Add-recipe.patch to build and deploy eiq sample apps

i have modified Makefile and face_recognition.cpp changing

/usr/include/tensorflow/contrib/lite‍‍

with

/usr/include/tensorflow/lite
‍

because in tensorflow 1.13.2, lite have been moved from contrib to main tree (with sumo branch of BSP and release r1.12 of tensorflow-lite all were build successfully)

i start FaceRecognition with following command (my aim is to use GPU acceleration)
```
./FaceRecognition -c 0 -h 0.85 -a 1 -v 1 -p 1‍
```
if data folder is empty, application starts correctly, camera video is displayed but no face is been recognized
if i populate data folder with some face images (i have tried with jpg and bmp formats), application crashes

starting application with gdb, the problem is in facerecognition.cpp row 212 (interpreter->Invoke()), when i execute this row this is the output

Thread 1 "FaceRecognition" received signal SIGSEGV, Segmentation fault.
0x0000ffffbecb313c in __dynamic_cast () from /usr/lib/libstdc++.so.6
‍‍

Could you give us some support?

Thanks in advance,

Damiano Cipriani

vanessa_maegima · ‎02-17-2020

Hi damiano.cipriani@engicam.com‌,

Sorry for the delay in getting back to you. We have just fixed this error.

On your L4.19 Yocto build you need to enable OpenCV + Qt5 windowing support:

PACKAGECONFIG_append_pn-opencv_mx8 = " qt5"

Let me know if this works for you.

Vanessa

damiano_ciprian · ‎02-19-2020

Hi vanessamaegima,

i have just rebuild imx-image-full with configuration you suggested to me, but now application crash in another point:

terminate called after throwing an instance of 'cv::Exception'
  what():  OpenCV(4.0.1) /usr/src/debug/opencv/4.0.1.imx-r0/git/modules/objdetect/src/cascadedetect.cpp:1658: error: (-215:Assertion failed) !empty() in function 'detectMultiScale'
‍‍

This happens in all way i start application, both with data folder empty and containing some images, both with acceleration enabled and disabled.

Let me know if i have to make some other test to help you in solving this problem.

Thanks

Damiano

vanessa_maegima · ‎02-19-2020

Hi damiano.cipriani@engicam.com‌,

That's weird. This CV exception is the exact error we could fix with Qt5 windowing support.

Could you please share your whole local.conf?

This is what mine looks like for opencv packages, but I guess that's not the problem:

PACKAGECONFIG_append_pn-opencv_mx8 = " dnn python3 qt5 jasper test"

Also, we are working to update the eIQ Sample Apps recipe (this did not work for me yesterday in my build), so if you could please try cross compiling the application in your host machine instead of in Yocto just to give it a try. Tutorial to build this is here: README.md\face_recognition\examples-tflite - eiq_sample_apps - Unnamed repository; edit this file 'd...

Thanks,
Vanessa

damiano_ciprian · ‎02-20-2020

Hi vanessamaegima,

i have rebuild image with this last option and compiled application from outside Yocto setting sdk toolchain, but result is the same. Here my local.conf

MACHINE ??= 'imx8qmmek'
DISTRO ?= 'fsl-imx-wayland'
PACKAGE_CLASSES ?= 'package_rpm'
EXTRA_IMAGE_FEATURES ?= "debug-tweaks"
USER_CLASSES ?= "buildstats image-mklibs image-prelink"
PATCHRESOLVE = "noop"
BB_DISKMON_DIRS ??= "\
    STOPTASKS,${TMPDIR},1G,100K \
    STOPTASKS,${DL_DIR},1G,100K \
    STOPTASKS,${SSTATE_DIR},1G,100K \
    STOPTASKS,/tmp,100M,100K \
    ABORT,${TMPDIR},100M,1K \
    ABORT,${DL_DIR},100M,1K \
    ABORT,${SSTATE_DIR},100M,1K \
    ABORT,/tmp,10M,1K"
PACKAGECONFIG_append_pn-qemu-system-native = " sdl"
PACKAGECONFIG_append_pn-nativesdk-qemu = " sdl"
PACKAGECONFIG_append_pn-opencv = " qt5"
PACKAGECONFIG_append_pn-opencv_mx8 = " dnn python3 qt5 jasper test"
CONF_VERSION = "1"

DL_DIR ?= "${BSPDIR}/downloads/"
ACCEPT_FSL_EULA = "1"

BB_NUMBER_THREADS=" 4 "
PARALLEL_MAKE =" -j 8 "
INHERIT += "rm_work"

IMAGE_INSTALL_append = " arm-compute-library tensorflow tensorflow-lite armnn onnxruntime"
IMAGE_INSTALL_append += "eiq-sample-apps"
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Only one question: before enabling OpenCV + Qt5 windowing support, application crashed with SIGSEGV. After this enabling, crash disappeared but application exit with a failed assert. Assert that you told me was the error fixed by Qt5 windowing support. So this support would have fixed both errors?

Thank you again for your support

Damiano

vanessa_maegima · ‎02-20-2020

Hi Damiano,

Your local.conf seems okay. We haven't faced the SIGSEGV error on our side, so I can't comment there.

Is it possible for you to check another demo from eiq_sample_apps just to check if the windowing support is working correctly on your setup?

Could you also share what dtb are you using to boot the board? Is it fsl-imx8qm-mek-ov5640.dtb?

Thanks,
Vanessa

damiano_ciprian · ‎02-24-2020

Hi vanessamaegima,

i have run opencv/camera.py and all works correctly. When i run FaceRecognition instead, this is the behaviour:

first time after machine boot, application try to start, application window appear for some moment ad then opencv exception is throwed causing application exit. Last output from application is

The all 0faces database is done!
run inference 
total time: 67 ms 
run inference 
total time: 35 ms 
‍‍‍‍‍

if now i try again to run it, application stucks as in the image below and i need to reboot machine to replicate exception throwing. Last output from application is

The all 0faces database is done!‍

if now (i.e. situation when FaceRecognition is not able to continue with execution and stucks) i start opencv/camera.py, this continue to works correctly.

So it seems to be a problem of FaceRecognition only: if you can give me instructions, i make also application debug to try to solve this problem.

Thanks for the support

Damiano

vanessa_maegima · ‎02-27-2020

Hi damiano.cipriani@engicam.com‌,

Could you please open a different thread to investigate this FaceRecognition issue? It's unrelated to this current thread.

You can @ me there and I will continue the follow up.

Thanks,
Vanessa

vanessa_maegima · ‎01-27-2020

Hi damiano.cipriani@engicam.com‌,

We are facing similar issues with this demo internally, we are debugging this as well on our side.

I will let you know the workaround when we get this running in the latest release.

Thanks,

Vanessa