mfw_isink low quality, ipu driver

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

mfw_isink low quality, ipu driver

Jump to solution
3,293 Views
janiscoders
Contributor III

Hi, I've been playing around with gstreamer's mfw_isink plugin and noticed a lower quality playback than using mfw_v4lsink.

I am using Yocto dora branch, imx53qsb based board, 2.6.35_maintain kernel.

I've tested and seen this issue when playing video files (h264+mp4/mkv) or even using videotestsrc.

I have a presentation converted to video and when playing with mfw_v4lsink, then everything is fine - text is visible fine (have tested with 960x720 and 1024x768 video resolution and have tested with .mkv and .mp4):

     gst-launch filesrc location=xxx.mp4 ! qtdemux ! mfw_vpudecoder ! mfw_v4lsink

When playing with mfw_isink then the quality drops and text on video almost can't be read.

     gst-launch filesrc location=xxx.mp4 ! qtdemux ! mfw_vpudecoder ! mfw_isink

Note, that I've tested many combinations - using playbin2, inserting mfw_ipucsc, inserting caps with different resolutions and raw video formats,played with disp-width, disp-height variables - result is the same.

I need this because I am playing multiple videos at the same time, so mfw_v4lsink can't be used in the end.

What could be the problem? I have used verbose output and see that negotiated caps of mfw_isink and mfw_v4lsink are the same.

Could this be ipu driver problem? If so, then is imx53 and imx6 IPU hardware the same? If so, then could I try to merge newer ipu/ipu3 driver version from 3.0.35 into 2.6.35_maintain? Or could it be gst-fsl-plugin problem?

Tags (3)
1 Solution
1,586 Views
qiang_li-mpu_se
NXP Employee
NXP Employee

Hi Janis, I think your issue was caused by alpha setting, the v4lsink will set alpha to 0, so fb0 will not affect the video playback in overlay, but for isink, it will not set alpha, and I found the alpha was enabled default in kernel. I can see the kernel logo if I used isink to playback after booted.

I think you can use v4l2sink to play video first, then use isink to play. There is no difference for video quality in this way.

View solution in original post

11 Replies
1,586 Views
LeonardoSandova
Specialist I

Both video sinkers should have the same quality.

I have a imx6q setup ready, can you please share the xxx.mp4 file?

Leo

0 Kudos
1,586 Views
janiscoders
Contributor III

Shared 2 versions of video - mp4 and mkv. Parameters: 960x720, h264, 30fps.

The problem can be seen on second 'slide', when smaller text appears. isink has much worse quality.

0 Kudos
1,586 Views
LeonardoSandova
Specialist I

Janis, are you doing HDMI playback? I got a LVDS screen and I do not see the difference between the two sinkers.

Also, please check this post

https://community.freescale.com/docs/DOC-93788#comment-5973

and provide that info.

Leo

0 Kudos
1,586 Views
joshkurland
Contributor IV

Hi Janis,

I just tested your clips on my imx6Q Wandboard.  I saw no discernible differences between the v4lsink and the isink.  I am using a 1080p HDMI monitor.  Here are the pipelines I used:

  gst-launch playbin2 uri=file:///fill_in_directory/output.mp4 video-sink=mfw_v4lsink

  gst-launch playbin2 uri=file:///fill_in_directory/output.mp4 video-sink=mfw_isink

Its funny, I have a problem with text on one of my test clips when I use mfw_v4lsink.  It gets a little pixelated (nothing like what you are describing), while mfw_isink provides very clean crisp text.

0 Kudos
1,586 Views
janiscoders
Contributor III

Thank you for answers. The long story is like this - we have created our custom board which is based mostly on Sabre (SMD) and a bit of quickboard (QSB).

I am testing this this with original Sabre LVDS screen.

fbset gives me this output:

root@imx53be:/videos# fbset

mode "1024x768-60"

    # D: 64.998 MHz, H: 48.362 kHz, V: 60.002 Hz

    geometry 1024 768 1024 2304 16

    timings 15385 220 40 21 7 60 10

    accel false

    rgba 5/11,6/5,5/0,0/0

endmode

vssconfig has these configurations parameters set:

root@imx53be:/videos# cat /usr/share/vssconfig

# vss device definition

# Singel display

# master display

[master]

type = framebuffer

format = RGBP

fb_num = 2

main_fb_num = 0

Using mfw_v4lsink this command (gst-launch playbin2 uri=file:///videos/output.mp4 video-sink=mfw_v4lsink) log output:

root@imx53be:/videos# gst-launch playbin2 uri=file:///videos/output.mp4 video-sink=mfw_v4lsink

mxc_v4l_close: release resource

MAX resolution 1024x768

mxc_v4l_close: release resource

MFW_GST_V4LSINK_PLUGIN 3.0.9 build on Nov  8 2013 10:17:33.

Setting pipeline to PAUSED ...

Aiur: 3.0.9

Core: MPEG4PARSER_06.07.00  build on Sep 10 2013 01:41:44

  mime: video/quicktime; audio/x-m4a; application/x-3gp

  file: /usr/lib/imx-mm/parser/lib_mp4_parser_arm11_elinux.so.3.2

Content Info:

    URI:

          file:///videos/output.mp4

    Idx File:

          /home/root/.aiur/.videos.output.mp4.aidx

    Seekable  : Yes

    Size(byte): 2094296

Pipeline is PREROLLING ...

Movie Info:

    Seekable  : Yes

    Live      : No

    Duration  : 0:01:00.588000000

    ReadMode  : Track

    Track    : 1

Track 00 [video_000000] Enabled

    Duration: 0:01:00.588000000

    Language: eng

    Mime:

          video/x-h264, parsed=(boolean)true, width=(int)960, height=(int)720, framerate=(

          fraction)1000/33, codec_data=(buffer)00000001674d401fe880780b7420000004200000fa0

          1e30622400000000168ebc3cb20

[INFO]    Product Info: i.MX53

VPU Version: firmware 1.4.50; libvpu: 5.3.2

MFW_GST_VPU_DECODER_PLUGIN 3.0.9 build on Nov  8 2013 10:17:39.

>>V4L_SINK: Actually buffer status:

    hardware buffer : 11

    software buffer : 0

Pipeline is PREROLLED ...

Setting pipeline to PLAYING ...

full screen size:1024x768

[V4L Update Display]: left=0, top=0, width=1024, height=768

set v4l display crop sucessfully

New clock: GstSystemClock

mxc_ipu mxc_ipu: Channel already disabled 10

mxc_ipu mxc_ipu: Channel already uninitialized 10

mxc_ipu mxc_ipu: Channel already disabled 10

mxc_ipu mxc_ipu: Channel already uninitialized 10

VPU blocking: timeout.

hrtimer: interrupt took 15250 ns

^CCaught interrupt -- handling interrupt.

Interrupt: Stopping pipeline ...

Execution ended after 53576547625 ns.

Setting pipeline to PAUSED ...

Running time 0:00:53.582285000 render fps 30.271

Setting pipeline to READY ...

Total rendered:1622

VPU blocking: timeout.

>>VPU_DEC: State: Ready to Null

Setting pipeline to NULL ...

[--->FINALIZE vpu_dec

[--->FINALIZE aiurdemux

Freeing pipeline ...

[--->FINALIZE v4l_sink

IMG_20131126_103257.jpg

Using mfw_isink this command (gst-launch playbin2 uri=file:///videos/output.mp4 video-sink=mfw_isink) log output:

root@imx53be:/videos# gst-launch playbin2 uri=file:///videos/output.mp4 video-sink=mfw_isink

set color key

MFW_GST_ISINK_PLUGIN 3.0.9 build on Nov  8 2013 10:17:25.

Setting pipeline to PAUSED ...

Aiur: 3.0.9

Core: MPEG4PARSER_06.07.00  build on Sep 10 2013 01:41:44

  mime: video/quicktime; audio/x-m4a; application/x-3gp

  file: /usr/lib/imx-mm/parser/lib_mp4_parser_arm11_elinux.so.3.2

Content Info:

    URI:

          file:///videos/output.mp4

    Idx File:

720:960x720) out win(0,0-1024,768:1024x768)

New clock: GstSystemClock

set deinterlace mode 0

hwbuf allocator zone(614400) created

hwbuf allocator zone(614400) destroied.

hwbuf allocator zone(614400) created

hwbuf allocator zone(614400) destroied.

hwbuf allocator zone(614400) created

hwbuf allocator zone(614400) destroied.

^CCaught interrupt -- handling interrupt.

Interrupt: Stopping pipeline ...

Execution ended after 58080931693 ns.

Setting pipeline to PAUSED ...

Setting pipeline to READY ...

VS0 destroyed, force=0!

>>VPU_DEC: State: Ready to Null

Setting pipeline to NULL ...

hwbuf allocator zone(1040384) destroied.

[--->FINALIZE vpu_dec

[--->FINALIZE aiurdemux

Freeing pipeline ...

[--->FINALIZE isink

          /home/root/.aiur/.videos.output.mp4.aidx

    Seekable  : Yes

    Size(byte): 2094296

Pipeline is PREROLLING ...

Movie Info:

    Seekable  : Yes

    Live      : No

    Duration  : 0:01:00.588000000

    ReadMode  : Track

    Track     : 1

Track 00 [video_000000] Enabled

    Duration: 0:01:00.588000000

    Language: eng

    Mime:

          video/x-h264, parsed=(boolean)true, width=(int)960, height=(int)720, framerate=(

          fraction)1000/33, codec_data=(buffer)00000001674d401fe880780b7420000004200000fa0

          1e30622400000000168ebc3cb20

[INFO]    Product Info: i.MX53

VPU Version: firmware 1.4.50; libvpu: 5.3.2

MFW_GST_VPU_DECODER_PLUGIN 3.0.9 build on Nov  8 2013 10:17:39.

hwbuf allocator zone(1040384) created

can not create threadmfw_gst_isink_setcaps:1489

Pipeline is PREROLLEmxc_ipu mxc_ipu: Channel already disabled 10

mxc_ipu mxc_ipu: Channel already uninitialized 10

D ...

Setting pipeline to PLAYING ...

VS0 created. in fmxc_ipu mxc_ipu: Channel already disabled 10

mxc_ipu mxc_ipu: Channel already uninitialized 10

mt[NV12] win(0,0-960,720:960x720) out win(0,0-1024,768:1024x768)

New clock: GstSystemClock

set deinterlace mode 0

hwbuf allocator zone(614400) created

hwbuf allocator zone(614400) destroied.

hwbuf allocator zone(614400) created

hwbuf allocator zone(614400) destroied.

hwbuf allocator zone(614400) created

hwbuf allocator zone(614400) destroied.

^CCaught interrupt -- handling interrupt.

Interrupt: Stopping pipeline ...

Execution ended after 58080931693 ns.

Setting pipeline to PAUSED ...

Setting pipeline to READY ...

VS0 destroyed, force=0!

>>VPU_DEC: State: Ready to Null

Setting pipeline to NULL ...

hwbuf allocator zone(1040384) destroied.

[--->FINALIZE vpu_dec

[--->FINALIZE aiurdemux

Freeing pipeline ...

[--->FINALIZE isink

IMG_20131126_104342.jpg

NOTE, that I am getting the same results if video is converted to resolution 1024x768 !!!

As you can see, there are pixels lost in letters when using isink.

I have tried to build pipeline manually and specifying NV12 and I420 in caps after mfw_vpudecoder - getting the same results. Tried even mfw_ipucsc after vpudecoder with NV12 and I420.

0 Kudos
1,587 Views
qiang_li-mpu_se
NXP Employee
NXP Employee

Hi Janis, I think your issue was caused by alpha setting, the v4lsink will set alpha to 0, so fb0 will not affect the video playback in overlay, but for isink, it will not set alpha, and I found the alpha was enabled default in kernel. I can see the kernel logo if I used isink to playback after booted.

I think you can use v4l2sink to play video first, then use isink to play. There is no difference for video quality in this way.

1,586 Views
Tarek
Senior Contributor I

HIQiang, i have a similar problem. Could you please elaborate which alpha global or local? And what to set it to ?Enable , disable, 0xFF, 0x00?

Thanks

0 Kudos
1,586 Views
qiang_li-mpu_se
NXP Employee
NXP Employee

you can set alpha to 0x00:

struct mxcfb_gbl_alpha alpha;

char fb_device[100] = "/dev/fb0";

if ((fd_fb = open(fb_device, O_RDWR )) < 0) {
  printf("Unable to open frame buffer\n");
  return TFAIL;
}

alpha.alpha = 0;
alpha.enable = 1;
if (ioctl(fd_fb, MXCFB_SET_GBL_ALPHA, &alpha) < 0) {
  printf("Set global alpha failed\n");
  close(fd_fb);
  return TFAIL;
}

close(fd_fb);

0 Kudos
1,586 Views
LeonardoSandova
Specialist I

BTW, this app sets the alpha

https://community.freescale.com/docs/DOC-1518

1,586 Views
LeonardoSandova
Specialist I

Janis,

on the mfw_isink pipeline, can you add disp-deinterlace=1 as property?

The only difference I see is that v4lsink deinterlaces by default and isink does not.

Leo

0 Kudos
1,586 Views
janiscoders
Contributor III

Thank you, but I have tried that and it doesn't help :smileysad:

I can see many changes in ipu drivers from 2.6.35_maintain to newer 3.x.x kernels. Could that be the problem and can these drivers be merged from newer kernel?

0 Kudos