Using a Full HD camera stream as OpenGL texture?

[d_s_e]

I'm fetching video data with V4L from a MIPI camera with 1920x1080 at 30 fps.

When I use glTexImage2D or glTexSubImage2D to use that data as an OpenGL texture, one of the cores goes up to 100% and the framerate drops significantly. Shouldn't it be possible to do something like this? What's the best way?

Working with Yocto on Wandboard Quad.

[d_s_e]

Thanks for your reply. I also suspect it's the upload step.

1080p@30fps is quite some data, but not uncommon nowadays, so I thought, the board could handle this without problems.

Is there a possibility to do this without moving the data around too much? Maybe with the GL Pixel Buffer Objects you mentioned? I'll have a look at that ...

The format coming from the camera is UYVY.

I didn't address the conversion step, yet. The plan was doing that inside the shader.

[leoschwab]

d.s.e wrote:

Thanks for your reply. I also suspect it's the upload step.  [ ... ]

The format coming from the camera is UYVY.

I didn't address the conversion step, yet. The plan was doing that inside the shader.

Ooooog, that may be another big part of the problem.  I suspect glTexImage2D() is performing a YUV->RGB conversion during the upload, and this conversion is almost always done on the CPU, and is murderously expensive.  You'll need to work out a way to get the data over to GL memory without conversion (probably using PBOs), then let GL shaders do the conversion.

[leoschwab]

My OpenGL-fu is not strong (ironically enough), but I suspect the upload step is partially responsible for killing performance.  Pixel format conversion may also be killing you.  What format are the camera pixels in?  Have you investigated using GL Pixel Buffer Objects?

That said, 1080p video is still a lot of data to be pushing around at 30 fps...