i.MX6 Problem displaying deinterlaced video at 1080

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

i.MX6 Problem displaying deinterlaced video at 1080

7,115 Views
davidthompson
Contributor II

I'm having a problem where video that was originally interlaced is displayed with some weird artifacts/tearing at the bottom of the image. I'm not sure of the correct technical term. I'll include an image below. It looks like pieces of the picture from the middle of the screen are being weaved into the bottom few rows of the image. I've investigated it quite a bit, but I'm finally stuck. Let me just explain what all I've figured out, and if there is other information needed, just ask.

My original source is a 1080i h.264 video, that I am attempting to decode, deinterlace, and then display on my 1080p HDMI monitor. I am using gstreamer. Both of the imx sinks, mfw_v4lsink and mfw_isink display the problem identically. I have determined that this means the problem is in a piece that is common between the two of them, the IPU cropping/scaling. 1080p and 720p videos have no problems.

At first I thought the problem was caused by the fact that my 1080i video, technically, had a height of 1088. I believe this is related to h.264 needing multiples of 16. The vpudec plugin, seemed to correctly identify all of this though, and even passed on the information that the sink should crop the bottom 8 pixels. Maybe there was an error with that? Well, I ruled that out eventually. I tested a 480i video (480x480) scaled up to fill my 1080 screen, and got the same result. The input source doesn't seem to be a factor, its a matter of output resolution.

Through trial and error, I found that the error only happens if my output height is greater than 1024. Well, that seemed kind of arbitrary, so I did a search to find why 1024 was significant. It turns out that the IPU has to split anything bigger than 1024x1024 into smaller pieces. This is where I'm stuck. I looked through the IPU code that splits up the image, and everything looks ok to me. It seems to divide the 1920x1080 screen into 4 equal pieces, it looks like there is some overlap between them, all the math looks accurate. I don't know where to look next, anyone have any suggestions?

My GStreamer pipeline:
gst-launch filesrc location=/home/root/h264-1080i.ts ! tsdemux name=demux demux. ! queue max-size-buffers=0 max-size-time=0 ! vpudec ! mfw_v4lsink
I've tried several variations on that pipeline, different plugins, etc, all with the same results, so I think I can rule out a gstreamer issue.

Board: imx6qsabresd
OS: Yocto, dora branch
Kernel: linux-imx-3.0.35-4.1.0

This is a picture of the effect. Note the out of place data on the right side just underneath the ticker at the very bottom. Those things bounce back and forth between the left and right bottom side of the screen, and are not always one solid block. I can take more pictures if that would help.

deinterlace-bug.jpg

Thanks in advance for all of the help!
David

0 Kudos
11 Replies

1,365 Views
davidthompson
Contributor II

I've made a little more progress with my own research. Nothing definitive, but I thought the additional info might be helpful for tracking down the root cause of this issue.

For testing purposes, I switched from kernel 3.0.35-4.1.0 to 3.10.9-1.0.0-alpha and this problem appears to be gone. I have some other issues with the alpha kernel, but this issue specifically does not appear to be present. I checked to see what was different between versions, particularly in the IPU drivers, as I suspect that is where the issue begins. Not a lot has been updated in 3.10.9, but I did notice that a few updates to 3.0.35, never made it into 3.5.7 or 3.10.9. One in particular, http://git.freescale.com/git/cgit.cgi/imx/linux-2.6-imx.git/commit/drivers/mxc/ipu3?h=imx_3.0.35_4.1... appears to be making a lot of changes right in areas I have been investigating, where a lot of the splitting and positioning math. I don't actually see any problem with that patch, but I'm not familiar enough with the code to say for sure. I thought it would be a good place to start looking.

I'll keep investigating, and see if I can find a way to get this working in 3.0.35. Is there an estimate on when kernel 3.10.9 will be upgraded from 'alpha' status?

Thanks for the info,

David

0 Kudos

1,365 Views
LeonardoSandova
Specialist I

Hi David,

Thanks for sharing your research on the problem.

I am not sure why that particular commit did not end into the 3.10 branch, but that is a good hint. Regarding your last question, you can actually use Yocto to bake a complete image with the 3.10.9 kernel version. For example, check this thread

When it will be available the i.MX6 - Yocto 1.5 and Linux kernel 3.10?

Are you in a position to use Yocto in your project? If yes, we first need to make sure what is the status of the 3.10 patch migration to master branch which I believe it is almost complete as mentioned by Lauren.

Leo

0 Kudos

1,365 Views
LeonardoSandova
Specialist I

Sorry David, the link I shared is just available internally. But here is the answer for your question:

3.10.9_alpha is almost completely upstreamed into meta-fsl-arm on Yocto 1.5 dora branch.   I believe the kernel and uboot recipes will be upstreamed by next week since patches were sent to the community earlier this week.  Once this happens you just use community version or stick with our version from our release layer meta-fsl-bsp-release.   It takes about 6 weeks from our release to get the changes upstreamed into the community version.

0 Kudos

1,365 Views
david_kosir
Contributor II

Sorry for not being constructive, I'm just declaring interest in your problem and I hope that you will share results with us.

BTW, I'm still waiting my I.MX6 board, could you tell me what is deinterlacing quality? Is it using any familiar algorithm?

0 Kudos

1,365 Views
LeonardoSandova
Specialist I

Hi David,

Have you read the corresponding section on the reference manual ( 9.2.2.2 Video de-interlacer (VDIC) )?. In case you need more info, you may need to contact the closest FAE.

Leo

1,365 Views
david_kosir
Contributor II

Thanks Leo,

     I've found what I was looking for. De-interlacing support looks very good, as its motion adaptive 3-way de-interlacing.

     Although this is hard off topic, you mentioned reference manual and I've checked it (for i.MX6Q). Its stated that VPU supports 1080p30 decoding while it's advertised to support 1080p60. What is the truth there?

Regards

0 Kudos

1,365 Views
LeonardoSandova
Specialist I

David, is there a way you can share the media file? it may be too heavy to attach it here....

Leo

0 Kudos

1,364 Views
davidthompson
Contributor II

Sure, I've attached a sample below.

Actually I've mainly been testing with IPTV streams (udpsrc), rather than local files. But I have, to rule out network issues, tested this issue with a filesrc as well. I made a recording of one of the streams I've been testing. To keep it under the 50 MB limit, I could only get about 30 seconds. If you need a longer recording, I'm sure we can figure something out. I'm seeing this same issue on pretty much all of the interlaced channels I'm testing, so I don't believe it is an issue with a specific broadcaster.

Let me know if there is anything else that would be helpful.

Thanks,

David

0 Kudos

1,364 Views
LeonardoSandova
Specialist I

David,

I was able to reproduce the issue using the pipeline you posted, except that I used aiurdemux as a demuxer.

In the other hand, I am not seeing the artifact using mfw_isink element. Can you try this pipeline:

$ export VSALPHA=1

$ gst-launch -v filesrc location=IPTV-cnbc.ts typefind=true ! aiurdemux name=demux demux. ! queue max-size-buffers=0 max-size-time=0 ! vpudec ! mfw_isink

I played the ts file on my host machine using mplayer, and I see that bar at the bottom, the same bar is seen using mfw_isink, so the video itself has that extra bar at the bottom, meaning that the video sinks are not adding it. Could you run the commands above and show the log

# uname -a

Linux imx6qsabresd 3.0.35-4.1.0+yocto+gbdde708 #1 SMP PREEMPT Tue Oct 29 19:51:21 TFT 2013 armv7l GNU/Linux

# cat /proc/cmdline

console=ttymxc0,115200 root=/dev/mmcblk0p2 rootwait rw video=mxcfb0:dev=hdmi,1920x1080M@60,if=RGB24

BTW, do you see the same artifacts using other monitors?

Leo

0 Kudos

1,364 Views
LeonardoSandova
Specialist I

You are right David, problem is indeed present even with isink. We are looking into the root problem.

0 Kudos

1,364 Views
davidthompson
Contributor II


I have tried mfw_isink before, and I think the reason you aren't seeing the issue, is that mfw_isink does not, by default, deinterlace the video. So while it won't create the artifact at the bottom, it is also not deinterlacing the video. I believe if you add disp-deinterlace=1 to your mfw_isink the same issue should appear there as well?

# uname -a

Linux imx6qsabresd 3.0.35-4.1.0+yocto+gbdde708 #1 SMP PREEMPT Fri Oct 4 01:09:41 EDT 2013 armv7l GNU/Linux

# cat /proc/cmdline

console=ttymxc0,115200 root=/dev/mmcblk1p2 rootwait rw video=mxcfb0:dev=hdmi,1920x1080M@60,if=BGR32,bpp=32 video=mxcfb1:off video=mxcfb2:off video=mxcfb3:off fbmem=28M dmfc=3

As far as the solid grey/white bar at the bottom of the video is concerned, I believe that is simply because the video is technically 1088 pixels tall, rather than 1080, due to h.264 need for multiples of 16. That bottom 8 pixels will usually be cropped, and while that is a little unusual, I don't think its related to the other artifacting. I've tested 480i videos (which don't produce that little bar) but they still have the artifacts when scaled up to 1080.

I have tested other HDMI monitors, yes. All of the 1080 monitors produce this same issue. I haven't yet tested any output other than HDMI.

Let me know if there is other information I can provide. Thank you for all of the assistance.

David Thompson

0 Kudos