Combing and loss of VSYNC with ISL79987 MIPI decoder

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Combing and loss of VSYNC with ISL79987 MIPI decoder

1,648 Views
jpuckett
Contributor III

We have an i.MX6S based system using an ISL79987 video decoder.

The ISL79987 supports up to four simultaneous analog inputs over a MIPI bus, of which we are using two.

We're running the imx_4.14.78_1.0.0_ga kernel and are using a driver we received from Intersil for a previous version of the kernel that we have migrated.

There were also numerous changes made to the IPU drivers by Intersil that allow for associating MIPI virtual channels with CSIs.  I have included the changed files, the current (very messy) version of our driver and the device tree file we're using.

We are using gstreamer for playback with the following command:

gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxipuvideosink force-aspect-ratio=false use-vsync-true

We are seeing two problems:

  1. There are very noticeable combing artifacts when there's motion in front of the camera.
  2. The video sometimes loses VSYNC and offsets vertically.

The first problem is very straightforward to see.  I just have to wave my hand in front of the camera.  See the attached Combing.jpg for an example.

The easiest way to trigger the offset is to remove and reapply power to the camera during playback.  Repeating this will change where the offset is but it will not go away until gstreamer is restarted.  See attached file "Offset Video.jpg".

We have an application that manages multiple video feeds using gstreamer API calls and simply shutting down the offending pipeline when we detect that the signal has been lost is not sufficient to reestablish VSYNC.  I suspect that there's some left over data in a buffer somewhere that isn't getting reset.

I have also seen this occur spontaneously when letting playback continue overnight and also by re-plugging the analog signal cable.

I will be out of the office starting May 17 and returning May 23.

Any help would be greatly appreciated!

Here are some additional comments/questions:

  1. The driver we received from Intersil (isl79987_mipi.c) is calling a number of mipi_csi2_*() functions to initialize and reset the interface.  It doesn't seem like this kind of initialization belongs here.  Is there another method?
  2. So far the only way I've been able to get a stable video feed is to use the decoder's 4 channel, 1 lane configuration which results in a 432MHz MIPI clock.  Any other setting results in very unstable video that constantly loses VSYNC and/or has a green bar on the side of the feed.  It seems like I should be able to use a 2 channel, 2 lane configuration that only requires a 108MHz MIPI clock.
  3. When video playback is active I see this error continually printed on the terminal:
    imx-ipuv3 2400000.ipu: IPU Warning - IPU_INT_STAT_5 = 0x00000001
    If the second analog channel is also active I see this one as well:
    imx-ipuv3 2400000.ipu: IPU Warning - IPU_INT_STAT_5 = 0x00000002
    These are both new-frame-before-end-of-frame errors for the CSI channels.  These show up even when the video output is stable.
  4. Is there some other way I don't know about to associate MIPI virtual channels to CSIs?  I would prefer not to have to use modified versions of the IPU/MIPI/V4L2 drivers.
Labels (4)
0 Kudos
3 Replies

1,285 Views
igorpadykov
NXP Employee
NXP Employee

Hi James

for other virtual channels example one can look on

iMX6DQ ISL79985/79987 MIPI CSI2 CVBS camera surround view solution for Linux BSP 

also can try with nxp i.mx gstreamer plugins imx-gst1.0 described in sect.7 Multimedia

attached Linux Guide (imxv4l2videosrc,imxipuvideosink are parts of gstreamer-imx plugins

not supported by nxp). Try with nxp linux from source.codeaurora.org/external/imx repository

linux-imx - i.MX Linux kernel 

For custom patches one can proceed with extended support of NXP Professional Services | NXP 

Best regards
igor
-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

0 Kudos

1,285 Views
jpuckett
Contributor III

Igor,

Thanks for the reply!

The imx_4.14.78_1.0.0_ga kernel we have been using was retrieved from the source.codeaurora.org git repository.

I looked through the Linux user's guide but did not see a reference to the imx-gst1.0 plugins.

However, I did build them and tried them with our setup.

As distributed, they will not work with our ISL79987 decoder driver.  I discovered in imx-gst1.0-plugin-4.1.4/libs/gstimxv4l2.c that the function gst_imx_v4l2capture_set_function() is hard coded to check if the chip name starts with "ov" or "adv".

If the name doesn't match, the function returns an error resulting in gstreamer failing to play the video.

For expediency I changed the chip name in our driver to "adv7180" and I was able to play the video with this command:

gst-launch-1.0 imxv4l2src  device=/dev/video0 ! imxv4l2sink

At this point I discovered that playing the video at its native resolution of 720x480 (which was a specific request from a prospective customer) still showed the combing with this command:

gst-launch-1.0 imxv4l2src  device=/dev/video0 ! imxv4l2sink overlay-top=0 overlay-left=0 overlay-width=720 overlay-height=480

When I change the playback resolution, something in the pipeline filters the combing out of the video:

gst-launch-1.0 imxv4l2src  device=/dev/video0 ! imxv4l2sink overlay-top=0 overlay-left=0 overlay-width=728 overlay-height=480

Also, when playing the video at its native resolution I only see one IPU_INT_STAT_5 error right when playback starts.

If I remove and reapply power to the camera I still see the shifted video until I restart gstreamer.

Questions:

  1. When I scale the video, what is it that is actually filtering out the combing?  Is there some way to enable it while showing the video at its native resolution?
  2. What would be the correct way to enable the imx-gst1.0 plugins to work with chip names that don't start with "ov" or "adv"?
0 Kudos

1,285 Views
igorpadykov
NXP Employee
NXP Employee

Hi James

>I looked through the Linux user's guide but did not see a reference to the imx-gst1.0 plugins.

it is referenced in Table 5. i.MX Code Aurora Forum Distributed Repos attached Release Notes

from Linux Documentation Linux L4.14.98_2.0.0 Documentation

for other issues suggest to proceed with extended support of NXP Professional Services

NXP Professional Services | NXP 

Best regards
igor

0 Kudos