AnsweredAssumed Answered

i.MX6 demosaic using GPU in Android framework code

Question asked by Matthew Davis on May 3, 2016
Latest reply on Jun 13, 2016 by Matthew Davis
Hello all,

 

 

I'm working on some GPU-based demosaic code for a RAW-only image sensor and I'm running into some trouble getting Android to allow me to access the GPU hardware using OpenGL ES.  Here's what I've done:

 

0)  Wrote a kernel driver for my sensor and modifed the kernel driver framework to pass through grayscale images.

 

1)  Created a custom DeviceAdapter by inheriting from OvCamera in libcamera2.  This class treats the camera sensor as a grayscale camera but tells the Android layer that the images are RGB565.  Verified that this puts images up on the screen at a smooth frame rate.  So far so good. 

 

2)  Wrote some cod to handle the demosiac logic using thee CPU to get the pixel format to match what Android expects.  I've inserted this in the camera thread, so images come off of the camera in raw bayer format and are immediately converted to RGB565.  This works well, but obviously performance is not adequate.

 

3)  Wrote a replacement demosaic module using OpenGL ES and attempted to swap it in.  This fails at runtime (no pixels are read back out after processing).  It looks like the problem starts immediately at eglGetDisplay(), because I see this in the logs:

 

E/libEGL  (  154): call to OpenGL ES API with no current context (logged once per thread)
I interpret the error to mean that something else has already opened up an OpenGL context in another thread and I need to make my calls from that context.  I've seen others mention in the forum that they've done everything on the camera capture thread (making sure to make all OpenGL calls on the capture thread, which I have done), but this error happens immediately as I try to initialize the library.  Since this this is Android framework code, I guess I shouldn't be surprised that I'm not the only one in my process space hoping to use the GPU, but I don't know how others have gotten around this problem.  The rules for writing stand-alone native apps seem like they don't apply to this situation.

 

Does anybody have any suggestions?

 

Thanks,

 

-Matt

Outcomes