GPU usage with Yocto

[fikretalim]

Hello,

We are trying to see the performance of the GPU with a simple example such as segmentation. For this purpose, we have tried to use the gesture recognition example prepared by Mr. Andre Silva given in the following link:

http://imxcv.blogspot.com/2012/03/gesture-recognition-on-imx6.html

But this example is compiled with LTIB, so it didn't work with Yocto image. I have tried to compile it with Yocto, but I had some problems.

First of all, it was not compiled, because it can not find stubs_soft.h. So I changed -mfloat-abi flag from softfp to hard. I have also changed the include and library directories according to my PC. I have attached the modified makefile to this post.

Now, it starts compiling, but I get another error because of fbGetDisplay function. It says that too few arguments to the function. The complete log is given below. I have also tried to change fbGetDisplay function with fbGetDisplayByIndex(4). But it still was not compiled

What can be wrong? If there are other examples that use a webcam and perform some video processing tasks on GPU that I can use with Yocto image, it will also very helpful for us (OpenCL or OpenGL examples).

Thank you

Fikret

fikret@fikret-VirtualBox:/media/918970b5-b414-445e-b8f3-7e4029338bc8/GPUbased/gesture-recognition-master$ make

cd src; \

                make -f Makefile distclean; \

                make -f Makefile install;

make[1]: Entering directory `/media/918970b5-b414-445e-b8f3-7e4029338bc8/GPUbased/gesture-recognition-master/src'

rm -rf main.o glcvutils.o glcvplane.o netplayer_gst.o mlp.o

rm -rf *~ core *.core

rm -rf ../bin/gesture

make[1]: Leaving directory `/media/918970b5-b414-445e-b8f3-7e4029338bc8/GPUbased/gesture-recognition-master/src'

make[1]: Entering directory `/media/918970b5-b414-445e-b8f3-7e4029338bc8/GPUbased/gesture-recognition-master/src'

arm-poky-linux-gnueabi-g++ -Wall -O2 -fsigned-char -march=armv7-a -mfpu=neon -mfloat-abi=hard -I. -I/media/918970b5-b414-445e-b8f3-7e4029338bc8/yocto/fsl-community-bsp/build/tmp/sysroots/imx6qsabrelite/usr/include -I/media/918970b5-b414-445e-b8f3-7e4029338bc8/yocto/fsl-community-bsp/build/tmp/sysroots/imx6qsabrelite/usr/include/gstreamer-0.10 -I/media/918970b5-b414-445e-b8f3-7e4029338bc8/yocto/fsl-community-bsp/build/tmp/sysroots/imx6qsabrelite/usr/include/glib-2.0 -I/media/918970b5-b414-445e-b8f3-7e4029338bc8/yocto/fsl-community-bsp/build/tmp/sysroots/imx6qsabrelite/usr/lib/glib-2.0/include -I/media/918970b5-b414-445e-b8f3-7e4029338bc8/yocto/fsl-community-bsp/build/tmp/sysroots/imx6qsabrelite/usr/include/libxml2  -c -o main.o main.cpp

main.cpp:167:0: warning: "EXIT" redefined [enabled by default]

#define EXIT   5

^

In file included from /media/918970b5-b414-445e-b8f3-7e4029338bc8/yocto/fsl-community-bsp/build/tmp/sysroots/imx6qsabrelite/usr/include/opencv/cv.h:79:0,

                 from main.cpp:39:

/media/918970b5-b414-445e-b8f3-7e4029338bc8/yocto/fsl-community-bsp/build/tmp/sysroots/imx6qsabrelite/usr/include/opencv2/core/internal.hpp:90:0: note: this is the location of the previous definition

#define EXIT __CV_EXIT__

^

main.cpp: In function 'int main(int, char**)':

main.cpp:242:8: warning: unused variable 'key' [-Wunused-variable]

  char  key = 0;

        ^

main.cpp:243:10: warning: variable 'endpoint' set but not used [-Wunused-but-set-variable]

  CvPoint endpoint = cvPoint (10, TEXTURE_H/2/2-10);

          ^

main.cpp:244:7: warning: unused variable 'm' [-Wunused-variable]

  char m[100];

       ^

main.cpp: In function 'void Render()':

main.cpp:389:15: warning: unused variable 'z_angle' [-Wunused-variable]

  static float z_angle = 0;

               ^

main.cpp: In function 'void* UpdateTextureFromCamera(void*)':

main.cpp:479:1: warning: no return statement in function returning non-void [-Wreturn-type]

}

^

main.cpp: In function 'void* GstLoop(void*)':

main.cpp:493:1: warning: no return statement in function returning non-void [-Wreturn-type]

}

^

main.cpp: In function 'void IPInit()':

main.cpp:628:10: warning: unused variable 'contours' [-Wunused-variable]

   CvSeq *contours;

          ^

main.cpp: In function 'void PreProcessing()':

main.cpp:737:13: warning: unused variable 'menu_option' [-Wunused-variable]

  static int menu_option;

             ^

main.cpp: In function 'int PatternRecognition(IplImage*, CvSeq*, CvPoint*)':

main.cpp:886:7: warning: unused variable 'a' [-Wunused-variable]

  char a[100];

       ^

main.cpp: In function 'int StateMachine(int, int, int)':

main.cpp:1081:6: warning: unused variable 'new_state' [-Wunused-variable]

  int new_state;

      ^

main.cpp: In function 'int PatternRecognition(IplImage*, CvSeq*, CvPoint*)':

main.cpp:890:3: warning: 'pos' may be used uninitialized in this function [-Wmaybe-uninitialized]

   switch (pos)

   ^

main.cpp: In function 'int MotionDetect(int, CvPoint, bool)':

main.cpp:985:56: warning: 'vertical' may be used uninitialized in this function [-Wmaybe-uninitialized]

    state = StateMachine (CHANNEL, horizontal, vertical);

                                                        ^

main.cpp:1010:55: warning: 'horizontal' may be used uninitialized in this function [-Wmaybe-uninitialized]

    state = StateMachine (VOLUME, horizontal, vertical);

                                                       ^

arm-poky-linux-gnueabi-g++ -Wall -O2 -fsigned-char -march=armv7-a -mfpu=neon -mfloat-abi=hard -I. -I/media/918970b5-b414-445e-b8f3-7e4029338bc8/yocto/fsl-community-bsp/build/tmp/sysroots/imx6qsabrelite/usr/include -I/media/918970b5-b414-445e-b8f3-7e4029338bc8/yocto/fsl-community-bsp/build/tmp/sysroots/imx6qsabrelite/usr/include/gstreamer-0.10 -I/media/918970b5-b414-445e-b8f3-7e4029338bc8/yocto/fsl-community-bsp/build/tmp/sysroots/imx6qsabrelite/usr/include/glib-2.0 -I/media/918970b5-b414-445e-b8f3-7e4029338bc8/yocto/fsl-community-bsp/build/tmp/sysroots/imx6qsabrelite/usr/lib/glib-2.0/include -I/media/918970b5-b414-445e-b8f3-7e4029338bc8/yocto/fsl-community-bsp/build/tmp/sysroots/imx6qsabrelite/usr/include/libxml2  -c -o glcvutils.o glcvutils.cpp

glcvutils.cpp: In member function 'bool GLCVUtils::GLInit()':

glcvutils.cpp:59:53: error: too few arguments to function '_FBDisplay* fbGetDisplay(void*)'

  EGLNativeDisplayType native_display = fbGetDisplay();

                                                     ^

In file included from /media/918970b5-b414-445e-b8f3-7e4029338bc8/yocto/fsl-community-bsp/build/tmp/sysroots/imx6qsabrelite/usr/include/EGL/eglplatform.h:38:0,

                 from /media/918970b5-b414-445e-b8f3-7e4029338bc8/yocto/fsl-community-bsp/build/tmp/sysroots/imx6qsabrelite/usr/include/EGL/egl.h:34,

                 from glcvutils.h:45,

                 from glcvutils.cpp:21:

/media/918970b5-b414-445e-b8f3-7e4029338bc8/yocto/fsl-community-bsp/build/tmp/sysroots/imx6qsabrelite/usr/include/EGL/eglvivante.h:101:1: note: declared here

fbGetDisplay(

^

make[1]: *** [glcvutils.o] Error 1

make[1]: Leaving directory `/media/918970b5-b414-445e-b8f3-7e4029338bc8/GPUbased/gesture-recognition-master/src'

make: *** [gpucv] Error 2

Original Attachment has been moved to: Makefile.zip

[andre_silva]

Hi Fikret,

this framebuffer error is due the EGL initialization. Since the code was firstly developed to work with the framebuffer, so the EGL init must be changed to work with X11 instead, here in this post you can get the correct init code: https://community.freescale.com/thread/302339

Regards,

Andre

[vladspiridonesc]

Hello Andre,

I have the same error, but I was thinking, isn't it better to output directly to the framebuffer? Do you have an idea of what I need to install in the BSP in order to do that?

I have a Phytec board with iMX6 so the building process is a bit different but I think I can handle.

Regards,

Vlad

[andre_silva]

Hi Vlad,

for the use on framebuffer you can base on the original code on the link provided on the first question (imxcv blog) the EGL is set to work on FB there. Is that your question ?

regards,

Andre

[vladspiridonesc]

Just a part of it. The other part was what to install in order to manage to use that code and write directly to the framebuffer and I think the answer is DirectFB.  I dropped the idea for the moment because I don't need to visualize things for the moment. Thanks anyway!

[fikretalim]

Hi Andre,

Thank you for your prompt answer, but I can not see the post that you gave. I get the following error;

Unauthorized

Access to this place or content is restricted. If you think this is a mistake, please contact your administrator or the person who directed you here.

How can I see it?

Thanks,

Fikret

[andre_silva]

Hi Friket,

sorry, please, find the code attached (use it to modify your code).

regards,

Andre

[fikretalim]

Hi Andre,

Is there a header file missing? Because, some variables can not be found such as g_hShaderProgram, g_hVertexLoc, g_hColorLoc, etc. I tried to find some of them from web, but I still get compile errors.

Thanks,

Fikret

[andre_silva · Accepted solution]

Hi Friket,

You will not use them, use what you have in your code, and just change the init part to this:

     eglNativeDisplayType = fsl_getNativeDisplay();

     egldisplay = eglGetDisplay(eglNativeDisplayType);

     eglInitialize(egldisplay, NULL, NULL);

     assert(eglGetError() == EGL_SUCCESS);

     eglBindAPI(EGL_OPENGL_ES_API);

     eglChooseConfig(egldisplay, s_configAttribs, &eglconfig, 1, &numconfigs);

     assert(eglGetError() == EGL_SUCCESS);

     assert(numconfigs == 1);

     eglNativeWindow = fsl_createwindow(egldisplay, eglNativeDisplayType);

     assert(eglNativeWindow);

     eglsurface = eglCreateWindowSurface(egldisplay, eglconfig, eglNativeWindow, NULL);

     assert(eglGetError() == EGL_SUCCESS);

     EGLint ContextAttribList[] = { EGL_CONTEXT_CLIENT_VERSION, 2, EGL_NONE };

     eglcontext = eglCreateContext( egldisplay, eglconfig, EGL_NO_CONTEXT, ContextAttribList );

     assert(eglGetError() == EGL_SUCCESS);

     eglMakeCurrent(egldisplay, eglsurface, eglsurface, eglcontext);

     assert(eglGetError() == EGL_SUCCESS);

this is the part that tells EGL to use X11 instead FB, if you look at the beginning of the code I sent, the you will find those functions in the code above.

regards,

Andre

[fikretalim]

Dear Andre,

I have applied the changes as recommended, the only difference is I have commented out "//display = EGL_DEFAULT_DISPLAY;" line inside fsl_getNativeDisplay function, because it is not defined anywhere.

After that, I got some other compile errors;

glcvutils.cpp: In member function '_FBDisplay* GLCVUtils::fsl_getNativeDisplay()':

glcvutils.cpp:26:27: error: cannot convert 'Display* {aka _XDisplay*}' to 'EGLNativeDisplayType {aka _FBDisplay*}' in assignment

      eglNativeDisplayType = XOpenDisplay(NULL);

                          ^

glcvutils.cpp: In member function '_FBWindow* GLCVUtils::fsl_createwindow(EGLDisplay, EGLNativeDisplayType)':

glcvutils.cpp:38:134: error: cannot convert 'EGLNativeDisplayType {aka _FBDisplay*}' to 'Display* {aka _XDisplay*}' for argument '1' to 'Window XCreateSimpleWindow(Display*, Window, int, int, unsigned int, unsigned int, unsigned int, long unsigned int, long unsigned int)'

      window = XCreateSimpleWindow(eglNativeDisplayType2, rootwindow, 0, 0, 400, 533, 0, 0, WhitePixel (eglNativeDisplayType2, screen));

                                                                                                                                      ^

glcvutils.cpp:39:46: error: cannot convert 'EGLNativeDisplayType {aka _FBDisplay*}' to 'Display* {aka _XDisplay*}' for argument '1' to 'int XMapWindow(Display*, Window)'

      XMapWindow(eglNativeDisplayType2, window);

                                              ^

glcvutils.cpp:40:20: error: invalid conversion from 'Window {aka long unsigned int}' to 'EGLNativeWindowType {aka _FBWindow*}' [-fpermissive]

      native_window = window;

Then, I have modified eglvivante.h file; I have removed all #ifdefs except X11.

After that, I added -lX11 inside the makefile and finally the file is compiled.

But, unfortunately now I have another problem :smileyhappy:

I tried to run the generated file on the board, but I got the following error:

Using: min:15, max: 40, val: -1

Yellow: ./gesture 15 40 -1

Skin: ./gesture 5 30 20

Loading netweights ...OK

Insufficient buffer memory on /dev/video0 -- decreasing buffers

Insufficient buffer memory on /dev/video0 -- decreasing buffers

Insufficient buffer memory on /dev/video0 -- decreasing buffers

Insufficient buffer memory on /dev/video0 -- decreasing buffers

Initializing OpenCV...OK

gesture: glcvutils.cpp:27: Display* GLCVUtils::fsl_getNativeDisplay(): Assertion 'eglNativeDisplayType != __null' failed.

Initializing OGLES1...Aborted

I have attached gvlcutils.cpp, glvcutils.h and eglvivante.h files that I have modified.

What can be wrong in this state? It looks like the XOpenDisplay function returns NULL, but why?

I would like to thank you for your time and patience.

Thanks&Best Regards,

Fikret

[andre_silva]

Hi Friket,

Unfortunately I didn´ t try this demo on x11 yet, I Believe you should try a simple helloworld application first using x11 just to see if everything is ok and then move again to this demo. We have some OpenGL ES demos using x11 in your GPU Sdk package, which you can found at our website (download tab in mx6 page), also, are you using an USB camera for this demo, right ? I saw that you got some errors in /dev/video0.

regards,

andre

[fikretalim]

Hi Andre,

Yes, I am using a Philips USB webcam, but I have no problem with the camera, because I always use it at our OpenCV applications.

Ok, I will try the applications as you recommended.

Thank you very much for your support.

Best Regards,

Fikret

[andre_silva]

Hi Friket,

no problem, good luck =)

regards,

Andre