OpenGL use without a display on a imx6 processor

[stefan_riesen]

Hello

In our project we want to use the GPU of a Freescale iMX6Q ARM processor to render an image and then write over a gstreamer pipeline to a VNC server.

The sequence is as follows:

1. Initialize OpenGL
2. For each frame of the video stream:
   1. Render Image with OpenGL ES to a texture using a Framebuffer and a Renderbuffer
   2. Read Image pixels using glReadPixels()
   3. Copy the Image data to the video frame
   4. Push frame to the video stream

We use Yocto 1.8 as the build-system, and the source code below of the init sequence works on our nitrogen6X-lite prototype board, but not on our headless production board. On the production board we can't initalize a "display": Either with USE_PBUFFER defined or not defined, we get a 0x3003 EGL_BAD_ALLOC Error. We have the vfb kernel module loaded with vfb_enable=1 and the video=vfb kernel options. The dev file /dev/fb0 is created successfully. In yocto, we tried the fsl-image-multimedia-full, image-core and image-core-directfb images both in default mode, and wtih

DISTRO_FEATURES_remove = " x11 wayland"
DISTRO_FEATURES_append = " directfb"

in our conf/local.conf file.

How should can we use the GPU of the iMX processor in an embedded Linux settings to render offscreen, without having a display? What are the appropriate Kernel settings? What are the appropriate user-space functions to use?

Current user-space implementation of the OpenGL init sequence:

EGLDisplay display;
EGLSurface surface;
#ifdef  USE_PBUFFER
  display = eglGetDisplay(EGL_DEFAULT_DISPLAY);
#else
 int fbnum = 0; // fbnum is an integer for /dev/fb0 fbnum = 0
 EGLNativeDisplayType native_display = fbGetDisplayByIndex(fbnum);
 EGLNativeWindowType native_window = fbCreateWindow(native_display, 0, 0, 0, 0);
 display = eglGetDisplay(native_display);
#endif  
if (display == EGL_NO_DISPLAY) {
 g_print("Unable to open connection to Window system: 0x%x\n",eglGetError());
 return 0; // <-- Point of failure, both with and without USE_PBUFFER set
}
// ...
if (!eglInitialize(display, &majorVersion, &minorVersion)) {
 g_print("Unable to initialize Display: 0x%x\n",eglGetError());
 return 0;
}
// ...
#ifdef USE_PBUFFER
  surface = eglCreatePbufferSurface(display, config, surfaceAttribList); 
#else
  surface = eglCreateWindowSurface(display, config, native_display, surfaceAttribList);
#endif

Extract of the current kernel config, based on 3.14.28

# CONFIG_VGASTATE is not set
# CONFIG_VIDEO_OUTPUT_CONTROL is not set
CONFIG_HDMI=y
CONFIG_FB=y
# CONFIG_FIRMWARE_EDID is not set
# CONFIG_FB_DDC is not set
# CONFIG_FB_BOOT_VESA_SUPPORT is not set
CONFIG_FB_CFB_FILLRECT=y
CONFIG_FB_CFB_COPYAREA=y
CONFIG_FB_CFB_IMAGEBLIT=y
# CONFIG_FB_CFB_REV_PIXELS_IN_BYTE is not set
CONFIG_FB_SYS_FILLRECT=m
CONFIG_FB_SYS_COPYAREA=m
CONFIG_FB_SYS_IMAGEBLIT=m
# CONFIG_FB_FOREIGN_ENDIAN is not set
CONFIG_FB_SYS_FOPS=m
# CONFIG_FB_SVGALIB is not set
# CONFIG_FB_MACMODES is not set
# CONFIG_FB_BACKLIGHT is not set
CONFIG_FB_MODE_HELPERS=y
# CONFIG_FB_TILEBLITTING is not set
#
# Frame buffer hardware drivers
#
# CONFIG_FB_IMX is not set
# CONFIG_FB_UVESA is not set
# CONFIG_FB_OPENCORES is not set
# CONFIG_FB_S1D13XXX is not set
# CONFIG_FB_TMIO is not set
# CONFIG_FB_SMSCUFX is not set
# CONFIG_FB_UDL is not set
# CONFIG_FB_GOLDFISH is not set
CONFIG_FB_VIRTUAL=m
# CONFIG_FB_METRONOME is not set
CONFIG_FB_MX3=y
# CONFIG_FB_BROADSHEET is not set
# CONFIG_FB_AUO_K190X is not set
# CONFIG_FB_MXS is not set
# CONFIG_FB_SIMPLE is not set
# CONFIG_EXYNOS_VIDEO is not set
# CONFIG_BACKLIGHT_LCD_SUPPORT is not set
CONFIG_FB_MXC=y
# CONFIG_FB_MXC_SYNC_PANEL is not set
CONFIG_FB_MXC_EDID=y
# CONFIG_FB_MXC_EINK_PANEL is not set

[Bio_TICFSL]

Hi Stefan,

I have not see any discrepancy on the initialization code or your kernel config I guess, You can write your application to target EGL, and it can be run without the presence of a window manager or even a compositor. To do so, you would call eglGetDisplay, eglInitialize, eglCreateContext and eglMakeCurrent, EGL accepts both X11 displays and Wayland displays, I think it is not possible for EGL to operate without one. For that propose you should install mesa in your yocto packages. http://www.mesa3d.org/egl.html

Hope this helps