We are using i.MX6 to process video from HD camera to show live preview on the screen, record the short video clips, etc. We use GStreamer to achieve this, but the delay between camera and display is higher than what we expected. What could we do to lower the video latency?
HW used:
SW used:
We use minimal GStreamer pipeline (only mfw_v4lsrc and mfw_isink, nothing more) to get the video signal from camera and show it on LCD.
We did some measurements on whole system with following results:
Below you can see oscilloscope output from our measurement, where we flashed bright light (white LED) directly into the camera, which causes bright flash of white on the LCD display (before the camera auto-gain even reacts), so there is rapid measurable change.
The channels show following:
Summary:
Hello Ivo,
Are you mounting the camera first? or you are just running the gstreamer pipeline, I think the latency came from the ADV7181, but I do not have something to test it.
Hi James,
I am sorry, but do not know what do you mean by "mounting the camera", could you please clarify that?
We believe that ADV7181C does not cause any significant delay, because the chip has no internal buffers that could cause delay of this order (it would be quite a lot of data, remember we are talking about 1080p25). I also thing that we in fact confirmed it when we did the measurements I have posted results from, but I have no data recorded to share with you.
_________________________________________________________
EDIT: I have measured delay caused by ADV7181C, results are attached.
Setup:
Results:
Here you can clearly see when the signal from camera reacts to short pulse of light from LED. You can also see that the ADV7181C output is not significantly delayed.
Zoomed in little bit to show blanking between two frames (the short "gap" in the Y component from camera.
Zoomed in to show separate pulses in both signal and showing some change in signal so you can compare them.
Conclusion:
As you can see, there is no significant delay caused by ADV7181C. There is definitely delay caused by AD conversion, but we are talking about 74.275 MHz. That is pixel clock for 1080p30 (which is used by ADV7181C, although we use only 25p).
We are reviewing your issue internally, as mention before it is difficult for us to reproduced, so this is taking some time to research. I apologize
James, do you have any new information regarding the latency? Do you need my help? I can carry out some tests if you are able to instruct me.
There is a parameter in mfw_v4lsrc which allow you to preview the camera output directly to the display, it use a different IPU path, maybe it is faster.
another advice would be testing to get a better timing using unit-test.
Dear James,
could you please clarify if there is any chance to use preview function in mfw_v4lsrc in recent version of gst-gsl-plugin? If not, is there any alternative providing the same function, or was this functionality simply ditched? Thank you for any answer.
Best regards
Ivo
Unfortunately I cannot test mfw_v4lsrc with preview enabled, because the attribute is not writable. We are using gst-fsl-plugins version 3.0.9 right now.
It seems that the gst-fsl-plugin-2.0.3 introduced some changes (at least that is what I am able to tell from sources on http://repository.timesys.com/buildsources/g/gst-fsl-plugin/, as I cannot find official Freescale repository) that disabled possibility to use preview function along with the following comment:
/*
* FixME: The overlay channel will cause v4l error:
* v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0 error
* disable it
*/
I tried changing the code so the preview parameter is writable, but while I am able to set the parameter to true, the preview function seems to be broken.
Could you tell me more information about the unit test? I am not sure which of them is related to the latency.
Thank you very much for the effort. If you need any more information, please let me know.