i.MX6 & OV5640 at 720p30 latency issue

[pratikagrawal]

Hello Team,

Setup Details

1. We are using i.MX6D platform with OV5640 camera.
2. For evaluation, we have used boundary nitrogen6_max evaluation board.
3. The customized board is also ready with a different display panel. But here I am discussing latency on evaluation board.

My Purpose:

1. I am trying to get optimized latency number for capture to display.

Current Results:

1. I am getting ~90ms latency for capture display only.
2. Using below GStreamer pipeline for this.

gst-launch-1.0 imxv4l2videosrc device=/dev/video1 imx-capture-mode=4 ! video/x-raw,width=1280,height=720 ! imxipuvideosink

Queries:

1. Is ~90ms is expected as a result?
2. Is there any way we can optimize it to lower than 50ms?
3. If I am capturing at 30 fps, then what can be the frame buffering at each level of image transition?

For example, Frame coming form CMOS ISP to i.MX6 IPU, how frames are buffered for color conversion and pixel conversion.

   4. Latency is almost same if I use 60 FPS at 1080p. Is that mean there is no fix frame numbers buffered at each part of image transition?

[pratikagrawal]

Hello NXP-Team,

The primary requirement is very low latency video capture, display, and streaming for maximum real-time performance. We have derived a solution and the results are good but there are two questions for that we need answers. 

Current Results:

1. Local display latency 40 to 60 ms.
2. Remove display latency is 90 to 130 ms.

Questions:

1. Is this the minimum latency we can achieve? Is there any possibility to further optimize the results? 
2. Why do we see local and remote latency in a variable range? We expect a constant latency if FPS is fixed at 30fps.

[jamesbone]

Hello Pratik,

We do not have theorical data, that I can share with you, since you are using a Nitrogen Board,  so the hardware Amount of memory, speed of the memory and this kind of HW details are required to provide a theorical number.

In the case of the CODECS, it is always necessary to use our Tools  ( NXP Board,  SABRE Board) and our Linux BSP that has been optimize.   If you can try ,  I would be able to escalate your issue so we can review if the latency it is out of the scope of the CODEC.

[pratikagrawal]

Thanks, Mr. Jamesbone for addressing my query.

Here are some details required for you.

1. Actual HW is not nitrogen, We are using i.MX6D processor in our product (customized HW).

2. It is using 2GB of RAM (part #: MT41K512M16HA-125:A) at 528MHz.

3. Yocto Pyro Linux (fsl multimedia) is used as operating system.

4. Using gstreamer and imx gstreamer plugins to leverage HW capabilities of multimedia.

5. WL18x7 is used here for Wi-Fi communication between HW.

Let me know if any more details required.

[pratikagrawal]

Kindly provide theoretical data, as I can not find such details from any documents.