i.MX28 pxp colorspace conversion (and rotation) for gstreamer

[markurup]

I'm trying to optimize the performance of our application, that captures images from a camera, because it runs very slowly.

The application is using gstreamer for interfacing with the camera.

The gst-launch cmd is like this:

gst-launch-0.10 v4l2src device=/dev/video1 ! video/x-raw-yuv,width=640,height=480 ! ffmpegcolorspace ! videoflip ! ffmpegcolorspace ! mfw_v4lsink

The camera only supports output in YUV format and is mounted at 90 degrees, so it also needs to be rotated.

When running it uses ~100% cpu.

It actually does this as soon as I insert the ffmpegcolorspace element in the chain.

Is there any way I can use the CSC in the PXP to convert the YUV input to RGB (i.MX28 Applications Processor Reference Manual, Rev. 1, 2010, ch. 34.2.4 - Color Space Conversion (CSC))?

I've also noted that the PXP has support for rotation (i.MX28 Applications Processor Reference Manual, Rev. 1, 2010, ch. 34.2.9 - Rotation).

I've found several posts on this site stating that rotation is not supported on the i.MX28.

Is this just because it's not implemented in software, or are there hardware issues that make this impossible?

I'm also implementing a barcode scanner, using an appsink, so I can't afford to spend all that cpu-time on rotating and color conversion:

gst-launch-0.10 v4l2src device=/dev/video1 ! video/x-raw-yuv,width=640,height=480 ! ffmpegcolorspace ! video/x-raw-rgb ! appsink max-buffers=1 drop=0

Basically what I'm asking is if there exists a mfw_colorspace (and a mfw_rotate) element for gstreamer (or similar).

Or if it would be possible to make those?

As I see it, that's where all my performance goes to die.

Thanks in advance.

[markurup]

By hacking around in the mxs_pxp driver, I can forcefully set "pxp->rotate = 90;"

Now the image is rotated, but it rotates the entire framebuffer, not just the overlay image.

This means that only part of my application screen is visible and another part of the screen is white (no data).

The performance is good however; using "gst-launch-0.10 -mv v4l2src device=/dev/video1  queue-size=1 ! video/x-raw-yuv,width=640,height=480 ! ffmpegcolorspace ! 'video/x-raw-yuv,format=(fourcc)I420' ! mfw_v4lsink force-aspect-ratio=1 disp-width=480 &" the CPU load is ~54%, and the image is updated much faster/more smooth than before.

Also, I can't find anywhere where I can read back the RGBBUF so I can use this in my QT application (QImage).

From the Reference Manual I see there's an option to write the data back to the original buffer (IN_PLACE), but since this is controlled by gstreamer I don't really want to do that.

Does anyone know if I can make the PxP do this color conversion and rotation, WITHOUT messing up my existing framebuffer, and just dumping the resulting data in a userspace accessible buffer somewhere (I might try to let qt show the image while it also scans it for barcodes)?

[GraceH · Accepted solution]

Hi Mark,

The fsl gst-plugin doesn't support rotation/CSC in MX233/MX28. You can try the patches in the post for MX233.  MX233 and MX28 both use PXP.

https://community.freescale.com/message/336068#336068

Grace