Gstreamer dual display resoultions

[williamtung]

Hi,

We tried dual LVDS output and could play two video simultaneously on a solo board. However, the resolution of 2nd display always follows the first one. Is there anything we're missing?

u-boot parameter:

video=mxcfb0:dev=ldb,LDB-WVGA,if=RGB666 video=mxcfb1:dev=ldb,LDB-WSVGA,if=RGB666 ldb=sep0

check mode:

root@linaro-ubuntu-desktop:~# cat /sys/class/graphics/fb0/mode

D:800x480p-60

root@linaro-ubuntu-desktop:~# cat /sys/class/graphics/fb2/mode

D:1024x600p-60

video1:

gst-launch  filesrc location=/home/demo/albertsons.mp4 typefind=true  ! aiurdemux name=demux demux.  ! queue max-size-buffers=0 max-size-time=0 ! vpudec  ! mfw_v4lsink device=/dev/video16 demux. ! queue max-size-time=0  ! beepdec ! audioconvert ! 'audio/x-raw-int, channels=2' ! alsasink demux.

output:

MFW_GST_V4LSINK_PLUGIN 3.0.5 build on Dec 14 2012 13:58:00.

Setting pipeline to PAUSED ...

[INFO]  Product Info: i.MX6Q/D/S

vpudec versions :smileyhappy:

        plugin: 3.0.4

        wrapper: 1.0.25(VPUWRAPPER_ARM_LINUX Build on Dec  7 2012 20:22:02)

        vpulib: 5.4.8

        firmware: 2.1.8.34588

Pipeline is PREROLLING ...

Aiur: 3.0.4

Core: MPEG4PARSER_06.04.25  build on Nov  9 2012 10:33:54

  mime: video/quicktime; audio/x-m4a; application/x-3gp

  file: /usr/lib/imx-mm/parser/lib_mp4_parser_arm11_elinux.so.3.1

Content Info:

        URI:

              file:///home/demo/albertsons.mp4

        Idx File:

              /root/.aiur/.home.demo.albertsons.mp4.aidx

        Seekable  : Yes

        Size(byte): 36913315

Movie Info:

        Seekable  : Yes

        Live      : No

        Duration  : 0:01:36.595000000

        ReadMode  : File

        Track     : 2

Track 00 [video_000000] Enabled

        Duration: 0:01:36.595000000

        Language: und

        Mime:

              video/x-h264, parsed=(boolean)true, width=(int)1280, height=(int)720, framerate=

              (fraction)30000/1001, codec_data=(buffer)000000016764001fac248805005ba100000303e

              90000ea600f1832a00000000168ee3cb0

Track 01 [audio_000000] Enabled

        Duration: 0:01:36.595000000

        Language: und

        Mime:

              audio/mpeg, mpegversion=(int)4, channels=(int)2, rate=(int)44100, bitrate=(int)1

              46528, framed=(boolean)true, stream-format=(string)raw, codec_data=(buffer)1210

[INFO]  bitstreamMode 1, chromaInterleave 1, mapType 0, tiled2LinearEnable 0

[WARN]  VPU iram is less than needed, some parts don't use iram

>>V4L_SINK: Actually buffer status:

        hardware buffer : 9

        software buffer : 0

Beep: 3.0.4

Core: AAC-Plus decoder Wrapper  build on Nov  8 2012 16:49:52

  mime: audio/mpeg, mpegversion=(int){2,4}

  file: /usr/lib/imx-mm/audio-codec/wrap/lib_aacplusd_wrap_arm12_elinux.so.3

CODEC: BLN_MAD-MMCODECS_AACD_ARM_03.07.00  build on Jul 17 2012 17:47:46.

BLN_MAD-MMCODECS_AACPD_ARM_03.07.00_ARM11  build on Jul 17 2012 17:58:59.

Pipeline is PREROLLED ...

Setting pipeline to PLAYING ...

full screen size:800x480

[V4L Update Display]: left=0, top=0, width=800, height=480

New clock: GstAudioSinkClock

video2:

gst-launch  filesrc location=/home/demo/albertsons.mp4 typefind=true  ! aiurdemux name=demux demux.  ! queue max-size-buffers=0 max-size-time=0 ! vpudec  ! mfw_v4lsink device=/dev/video18  demux.

output:

MFW_GST_V4LSINK_PLUGIN 3.0.5 build on Dec 14 2012 13:58:00.

Setting pipeline to PAUSED ...

[INFO]  Product Info: i.MX6Q/D/S

vpudec versions :smileyhappy:

        plugin: 3.0.4

        wrapper: 1.0.25(VPUWRAPPER_ARM_LINUX Build on Dec  7 2012 20:22:02)

        vpulib: 5.4.8

        firmware: 2.1.8.34588

Pipeline is PREROLLING ...

Aiur: 3.0.4

Core: MPEG4PARSER_06.04.25  build on Nov  9 2012 10:33:54

  mime: video/quicktime; audio/x-m4a; application/x-3gp

  file: /usr/lib/imx-mm/parser/lib_mp4_parser_arm11_elinux.so.3.1

Content Info:

        URI:

              file:///home/demo/albertsons.mp4

        Idx File:

              /root/.aiur/.home.demo.albertsons.mp4.aidx

        Seekable  : Yes

        Size(byte): 36913315

Movie Info:

        Seekable  : Yes

        Live      : No

        Duration  : 0:01:36.595000000

        ReadMode  : File

        Track     : 2

Track 00 [video_000000] Enabled

        Duration: 0:01:36.595000000

        Language: und

        Mime:

              video/x-h264, parsed=(boolean)true, width=(int)1280, height=(int)720, framerate=

              (fraction)30000/1001, codec_data=(buffer)000000016764001fac248805005ba100000303e

              90000ea600f1832a00000000168ee3cb0

Track 01 [audio_000000] Enabled

        Duration: 0:01:36.595000000

        Language: und

        Mime:

              audio/mpeg, mpegversion=(int)4, channels=(int)2, rate=(int)44100, bitrate=(int)1

              46528, framed=(boolean)true, stream-format=(string)raw, codec_data=(buffer)1210

[INFO]  bitstreamMode 1, chromaInterleave 1, mapType 0, tiled2LinearEnable 0

[WARN]  VPU iram is less than needed, some parts don't use iram

>>V4L_SINK: Actually buffer status:

        hardware buffer : 9

        software buffer : 0

Pipeline is PREROLLED ...

Setting pipeline to PLAYING ...

full screen size:800x480

[V4L Update Display]: left=0, top=0, width=800, height=480

New clock: GstSystemClock

[timothybean]

I discovered the same issue and was confused on why this happens. After debugging the mfw_v4lsink gstreamer element, it turns out that there are 2 issues going one.

1. device=/dev/videoX is not getting set in the parameters until AFTER the display is opened for reading the resolutions.

2. The mfw_v4lsink element is HARD CODED to only open and read fb0. So, whatever the resolution is set to fb0 is what will get picked for both displays.

I have modified my version to fix the setting of the /dev/videoX option and to then open the correct frame buffer to get the proper resolutions. Now, I do not need to set the resolutions manually as suggested as the correct answer. This means video will fill display no matter what the display resolution is set to.

Tim

[LeonardoSandova]

Nice Timothy. In case you have the ltib's patch (or yocto), please share it :smileyhappy:

Leo

[timothybean]

Leonardo,

I do not have it in front of me, but yes I can share it later tonight.

Tim

[holgermoritz]

Dear Timothy,

my name is Holger Moritz & I'm working on fixing some issues in the qmediaplayer from Qt for Yocto on imx6
(I added an option to bring the gui onto the secondary screen).

Can you help me with your patch regarding the gstreamer backend?

(knowing, that I'm very late in this conversation - pardon me)

Regards,

Holger

[jack_mao · Accepted solution]

hi, please try the following command and adjust your parameters

gst-launch filesrc location=1.mp4 typefind=true ! aiurdemux name=demux demux. ! queue max-size-buffers=0 max-size-time=0 ! vpudec ! mfw_v4lsink device="/dev/video16" disp-width=1024 disp-height=768 demux. ! queue max-size-buffers=0 max-size-time=0 ! beepdec ! audioconvert ! 'audio/x-raw-int, channels=2' ! alsasink &

[LeonardoSandova]

All thread's questions have been posted internally. Either me or other Freescale engineer will post a comment soon.

[LeonardoSandova]

Could you try this test:?

gst-launch playbin2 uri=file:///<filename> video-sink="mfw_v4lsink device=/dev/video17" &

gst-launch playbin2 uri=file:///<filename> video-sink="mfw_v4lsink device=/dev/video19"

if works, then change the device property on your pipelines.

[williamtung]

Hi Leonardo,

It doesn't work.

root@freescale /home/demo$ ls /dev/video*

/dev/video16  /dev/video17  /dev/video18

root@freescale /home/demo$ gst-launch playbin2 uri=file:///albertsons.mp4 video-sink="mfw_v4lsink device=/dev/video17"

MFW_GST_V4LSINK_PLUGIN 3.0.2 build on Dec 19 2012 10:56:28.

Setting pipeline to PAUSED ...

ERROR: Pipeline doesn't want to pause.

ERROR: from element /GstURIDecodeBin:uridecodebin0/GstFileSrc:source: Resource not found.

Additional debug info:

gstfilesrc.c(1036): gst_file_src_start (): /GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstFileSrc:source:

No such file "/albertsons.mp4"

Setting pipeline to NULL ...

Freeing pipeline ...

[--->FINALIZE v4l_sink

My original scripts can play just size issue.

[LeonardoSandova]

Which i.MX board are you using? BSP or Ubuntu version? have you tried the same pipeline on a demo image?

Sorry, I have re-read the problem you are facing but I do not fully understand the problem, could you explain a bit more the issue?

Leo

[scottomx6]

We have a Solo and we have only the /dev/video16, video17, and video18.  We recently received a patch from Freescale to fix video playback issue that was causing both displays of our use case to blank.  We have been working with Quad i.mx6 board to develop up until we received our custom i.MX6 Solo in mid April.  There is no /dev/video19 device like was present on i.MX6 Quad.  I would like to understand is this related to only one IPU in the solo versus two in the quad.  One device is an overlay /dev/video16 and the other is primary /dev/video17?  So what is /dev/video18?

Our use case:

- i.MX6 Solo - 1Ghz CPU, 1 GB RAM

- Two LVDS display connected (1024x600 display 1, 800x480 display 2)

- Only require one video (<= 720p MPEG4 H/W decoding) to play at a time, not like original poster's situation which described simultaneous playback of two different video feeds on display 1 and display 2

- Timesys Linux 3.0.35 1.x.x (Embedded Linux) with BSP patches for board (Basic I/O nothing out of the ordinary)

- Qt 4.8.1 based application (with X Windows) heavy use of Phonon (gstreamer backend) to play videos on either display (/dev/video18 locks up even after recent patch from Freescale to fix /dev/video17 playback)

- Display user application GUI 1 and video in Qt Phonon managed window (<= 720p mp4) is displayed on Display 1.  GUI 2 or video in Qt Phonon managed Window (<= 720 mp4) is displayed on Display 2.

No issues with above scenario on a i.MX6 Quad board which has /dev/video16, /dev/video17, /dev/video18, and additional device of /dev/video19. We switched our Phonon/Gstreamer to use /dev/video18 since NO /dev/video19 on i.MX6 Solo and now issues.  With very recent Freescale patch no problems with /dev/video16 and /dev/video17 managing display 1, however display 2 locks up and blanks screen after video playback (GUI 2 is fine). If you restart X you can regain control again but that of course kills our Qt Application.  We initially had this problem with /dev/video16 and /dev/video17 until the recent Freescale patch but this does not solve the display 2 (/dev/video18) issue.

What are possible solutions besides going to a i.MX6 Dual/Quad if in fact the only one IPU in i.MX6 Solo is the issue and we must have a /dev/video19 device present under linux.

We really need an understanding of the i.MX6 Solo/Dual/Quad H/W differences specifically in use case above that have precluded video playback without X locking up.  At this point gstreamer is locking up X now only on display 2,   Recent patch fixed gstreamer locking up display 1.

Is there a means to have X refresh /dev/video18 or stop gstreamer from causing X to blank screen and requiring a restart of X?

[LeonardoSandova]

Hi Scott,

I believe you are still facing issues when doing dual playback using QT Phonon. One favour: Could you please re-post this particular issue (perhaps with new updates) on a NEW thread? Please post all your setup configuration, including Xorg.conf, bootargs and the patch to solve the blanking issue on main screen, so we can have all info in one thread. Thanks.

Leo

[williamtung]

Hi Leo,

I can play two video simultaneously but the 2nd display resolution is the same same the first one. Let's say my first display is WVGA (800x480) and 2nd is WSVGA (1024x600). When set ldb=sep0 and play the video on 2nd display at full screen mode, it just outputs 800x480 resolution instead of 1024x600. Is there a way or workaround to fix this?

video1:

gst-launch  filesrc location=/home/demo/albertsons.mp4 typefind=true  ! aiurdemux name=demux demux.  ! queue max-size-buffers=0 max-size-time=0 ! vpudec  ! mfw_v4lsink device=/dev/video16 demux. ! queue max-size-time=0  ! beepdec ! audioconvert ! 'audio/x-raw-int, channels=2' ! alsasink demux.

video2:

gst-launch  filesrc location=/home/demo/albertsons.mp4 typefind=true  ! aiurdemux name=demux demux.  ! queue max-size-buffers=0 max-size-time=0 ! vpudec  ! mfw_v4lsink device=/dev/video18  demux.

[timothybean]

I am seeing similar issues. Were you able to get a resolution?

Thanks

Tim