AnsweredAssumed Answered

i.MX53 V4L2: Is De-Interlace in Overlay mode possible?

Question asked by TomE on Aug 25, 2015
Latest reply on Sep 29, 2015 by gusarambula
Branched to a new discussion

i.MX53 based on the Karo TX53 board. ADV7180 capturing composite (and therefore Interlaced) video.

 

This is an Automotive application, so the video is the Reversing Camera. A very common application that should be easily supported by the hardware and software.

 

I would expect there to be Application Notes detailing how to set up and use all of these cases, but the only thing that is available seems to be copies of Freescale's software testing "unit tests". They're made to regression-test the drivers rather than demonstrate their use in an Application.

 

Running Freescale Linux 2.6.35 taken from "git://git.freescale.com/imx/linux-2.6-imx.git" and starting from the imx_2.6.35_maintain branch. As far as I know this is the latest version of Linux to run on the i.MX53. That's the version linked from the i.MX53 Product Pages anyway. The last patch to that, the latest 2.6.35 branch, is dated 30 Oct 2013.

 

I can run "imx-test-11.09.01/test/mxc_v4l2_test/mxc_v4l2_tvin.c" and it runs very well, but with one major problem.

 

It uses V4L2_MEMORY_MMAP buffers on both the Capture and Output sides, and uses memcpy() to copy all data between the interfaces/drivers. The 720x480x2 buffers require copying 30 times a second, resulting in the CPU copying 20 megabytes/second.

 

Because the rescaling is only supported on the output driver, even if I'm only showing a postage-stamp-sized view of the camera it still has to copy the whole full resolution frame.

 

Without video, our Application takes 25% of the CPU doing what it does. With Video enabled, this rises to 88%. With a slightly more "busy" version of our Application which runs at 35% CPU loading, the total is now nudging 100%. That's no good. We can't handle the "reversing camera" taking 60% of the CPU. We need some spare CPU for USB data storage and for handling CAN traffic.

 

60% of the CPU taken copying 20 megabytes/sec? That's 5 mega-32-bit-words/sec read and the same written (total 10 MW/s) on a 32-bit DDR3 memory system running at 400 MHz. That should only be 1/40 of the memory bandwidth and not 60%!

 

Replacing the "memcpy()" with one written using NEON opcodes drops the overhead from 60% down to 40%. That's still way too much.

 

The obvious fix is to rewrite mxc_v4l2_tvin.c to use V4L2_MEMORY_USERPTR buffers, so the user code only has to swap pointers instead of copying data.

 

That's supported by the OUTPUT drivers in "drivers/media/video/mxc/output", but is NOT supported by the CAPTURE drivers in "drivers/media/video/mxc/capture". They only support V4L2_MEMORY_MMAP. I find that the later versions based on 3.14 and supporting subdevices do support USERPTR, but that's no good for 2.6-based drivers.

 

http://git.freescale.com/git/cgit.cgi/imx/linux-2.6-imx.git/tree/drivers/media/platform/mxc/capture/mxc_v4l2_capture.c?h=imx_3.14.38_6ul_engr

 

I can also run "imx-test-11.09.01/test/mxc_v4l2_test/mxc_v4l2_overlay.c", and it runs with ZERO CPU overhead as it is doing everything in hardware. That program doesn't support setting up the hardware for deinterlacing, so the video appears on the screen as two squashed frames, being the odd field and the even field. Inspection of the drivers indicates that only the OUTPUT drivers know how to deinterlace, and the OVERLAY driver is an INPUT one, and doesn't seem to.

 

I know there's a patch that adds deinterlacing to the Capture interface, but it is for "a rather old BSP version" meaning 3.10, and not 2.6.35 and is "old and unsupported" anyway.

 

    De-interlace capture device

    https://community.freescale.com/docs/DOC-93633

 

So does anyone have any suggestions for getting interlaced CVBS video from the camera to the display without killing the CPU?

 

Thanks,

 

Tom

Outcomes