YUV to RG color conversion with G2D problem

[ivan_garcia]

Hi!
We are trying to convert YUV frames captured using libuvc (usb cameras) to RGB, but we are not able to get the right color.

The format that libuvc gives the frames in is YUV422 acording with the headers:

enum uvc_frame_format {
....
 UVC_FRAME_FORMAT_COMPRESSED,
/** YUYV/YUV2/YUV422: YUV encoding with one luminance value per pixel and
* one UV (chrominance) pair for every two pixels.
*/

With dimensions 1920x1024, stride 3840 (1920*2)

And the code we are using to convert the color is:

cv::Mat ConvertFrame(uvc_frame_t *frame, int inputFormat=G2D_YUYV){
void *G2D_Handler=NULL;
if (g2d_open(&G2D_Handler)!=0){
puts("G2D_Start: Error opening g2d device");
};

int size = frame->data_bytes;
struct g2d_buf *srcM = g2d_alloc (size,1);
struct g2d_buf *dstM = g2d_alloc (frame->width * frame->height * 3,1);
memcpy(srcM->buf_vaddr, frame->data, size);

struct g2d_surface src,dst;
src.planes[0] = srcM->buf_paddr;
src.planes[1] = 0; //srcM->buf_paddr;
src.planes[2] = 0; //srcM->buf_paddr;
src.left = 0;
src.top = 0;
src.right = frame->width;
src.bottom = frame->height;
src.stride = frame->step;
src.width = frame->width;
src.height = frame->height;
src.rot = G2D_ROTATION_0;
src.format = inputFormat;


cv::Mat out(frame->height, frame->width, CV_8UC3, dstM->buf_vaddr );
dst.planes[0] = dstM->buf_paddr;
dst.left = 0;
dst.top = 0;
dst.right = out.cols;
dst.bottom = out.rows;
dst.stride = out.step1();
dst.width = out.cols;
dst.height = out.rows;
dst.rot = G2D_ROTATION_0;
dst.format = G2D_RGB888 ;

int r = g2d_blit(G2D_Handler, &src, &dst);
if (r!=0) puts("convertFrame :: ERROR g2d_blit");
r=g2d_finish(G2D_Handler);
if (r!=0) puts("convertFrame :: ERROR g2d_finish");

char name[128];
snprintf(name,128,"g2dOutput_%d.png", inputFormat );
cv::imwrite(name,out);

g2d_free (srcM);
return out;
}

The better results we could get are with inputFormat = G2D_YUYV, but as can be seen in the attached image, this is far to be correct.

Can someone give us an idea on how to solve this problem?

Thanks in advance!

[jheaffey]

Am using the G2D on an IMX8M Plus also. Strangely, YUYV to RGB888 works fine on Kirkstone, yet on Scarthgap the g2d library prints an error that the destination format is invalid.

Would it be possible to fix this issue?

[malik_cisse]

How could you convert YUYV to RGB888 on Kirkstone. I could never make this work. Would you share the code? thx
YUYV to RGBX8888 works fine though as I already stated bellow. Maybe that is ok for your application as you can ignore the "X" component as don't care.

[jheaffey]

Something like this:

g2d_surface srcSurface;
g2d_surface dstSurface;

srcSurface.format = G2D_YUYV;
srcSurface.left = 0;
srcSurface.right = 1920;
srcSurface.top = 0;
srcSurface.bottom = 1080;
srcSurface.stride = 1920;
srcSurface.width = 1920;
srcSurface.height = 1080;
srcSurface.blendfunc = G2D_ZERO;
srcSurface.global_alpha = 0;
srcSurface.clrcolor = 0;
srcSurface.rot = G2D_ROTATION_0;
srcSurface->planes[0] = src_g2d_buf->buf_paddr;
srcSurface->planes[1] = 0;
srcSurface->planes[2] = 0;

dstSurface.format = G2D_RGB888;
dstSurface.left = 0;
dstSurface.right = 300;
dstSurface.top = 0;
dstSurface.bottom = 300;
dstSurface.stride = 300;
dstSurface.width = 300;
dstSurface.height = 300;
dstSurface.blendfunc = G2D_ZERO;
dstSurface.global_alpha = 0;
dstSurface.clrcolor = 0;
dstSurface.rot = G2D_ROTATION_0;
srcSurface->planes[0] = dst_g2d_buf->buf_paddr;
srcSurface->planes[1] = 0;
srcSurface->planes[2] = 0;

int result = g2d_blit(_handle, src, dest);

[jheaffey]

I have had a breakthrough.

The i.MX Graphics User Guide states that

"

RGB stride alignment is 16 bytes on i.MX 6 (except i.MX 6Quad Plus), 1 pixel
alignment on i.MX 6Quad Plus, i.MX 7ULP and i.MX 8 family devices, both for source
and destination surface.

"

Even though I am using an i.MX8MP and stride alignment should not be necessary, I changed the stride value to be 16 byte aligned and the g2d operation (YUYV to RGB888) was successful!

Annoyingly my application now needs to handle the extra stride. Again, this stride alignment was not necessary on Kirkstone (specifically this version of the yocto manifest - imx-5.15.52-2.1.0.xml: Add manifest for L5.15.52-2.1.0 release [YOCIM… · nxp-imx/imx-manifest@842ad7...)

[ivan_garcia]

Hi everybody,

Yes, I was able to fix the problem:
It was only a problem of the sizes used.

void *G2D::convert(void *imageBuffer, int width, int height){
		int size = width * height *2;

		struct g2d_surface src,dst;
		src.planes[0] = srcM->buf_paddr;
		src.planes[1] = 0;
		src.planes[2] = 0;
		src.left = 0;
		src.top = 0;
		src.right = width;
		src.bottom = height/2;
		src.stride = width *2;
		src.width = width;
		src.height = height/2;
		src.rot = G2D_ROTATION_0;
		src.format = G2D_YUYV;


		dst.planes[0] = dstM->buf_paddr;
		dst.left = 0;
		dst.top = 0;
		dst.right = width;
		dst.bottom = height;
		dst.stride = width ;
		dst.width = width;
		dst.height = height;
		dst.rot = G2D_ROTATION_0;
		dst.format = G2D_RGB888 ;


		memcpy(srcM->buf_vaddr, imageBuffer, size);

		int r = g2d_blit(G2D_Handler, &src, &dst);
		if (r!=0) puts("convertFrame :: ERROR g2d_blit");
		r=g2d_finish(G2D_Handler);
		if (r!=0) puts("convertFrame :: ERROR g2d_finish");

		return dstM->buf_vaddr;
	}

Hope it is useful for you!

[malik_cisse]

Thank you for the swift feedback Ivan,

I implemented your solution but my image is still broken (see attachment). I also attached the code, maybe you find something obvious.

What CPU are you using.

Not sure why you have 
src.bottom = height/2;
src.height = height/2;
Thx

[malik_cisse]

Hi Ivan,

Could you solve this problem?
I am facing same issue.
Thx

[malik_cisse]

Hello @Bio_TICFSL ,

I have exact same problem:

On NXP imx8mp G2D GPU (GC520L) YUYV to RGBX works fine:
G2D_YUYV to G2D_RGBX8888 works fine

However YUYV to RGB (24bit RGB888) does not work. I have same broken pictures as above.
G2D_YUYV to G2D_RGB888

Any hints how I can solve this problem?
Thank you

[Bio_TICFSL]

Hello ivan_garcia

Currently, libg2d does not support YUYV convert to BGR. I will discuss with R&D team to enable.

Regards

[quercuspau]

Hello @Bio_TICFSL ,

any news related to this topic?

BR