gstreamer: video fails to play when opened for second time

[lachezarnikolov]

Hello,

I am not very experienced with gstreamer but I am for sure experiencing some strange behavior on my i.MX6 board that I am not experiencing on a normal ubuntu machine. I have an application that does the following:

1. when started initializes the gstreamer framework, using gst_init_check() function.

2. When a command is sent to load a file, the following pipeline is constructed:

filesrc->aiurdemux->...(here the pipeline devides)

...->queue->beepdec->audioconvert->audioresample->volume->alsasink

...->queue->vpudec->ffmpegcolorspace->mfw_v4lsink

the audio and video branch are connected dynamically when aiurdemux emits the pad-added signal.

Also the stream is set first to paused state, then it is played for 2 frames (hack to make the first frame appear) and then it is paused again.

3. The stream is set in PLAYING state. Now everything works perfectly.

4. The stream is stopped by doing the following:

- The state is set to GST_STATE_NULL

- the pipeline is unref-ed (in my tests this unrefs all the elements in the pipeline).

Now when I try to do the same, I don't get video input. The messages from gstreamer say "hardware buffer : 0" (when it is working normally it says "hardware buffer : 9"). Everything begins working again when the application is restarted. Here are the messages from the gstreamer:

MFW_GST_V4LSINK_PLUGIN 3.0.5 build on Oct 22 2013 15:33:00.

set stretch:0.

vpudec versions :smileyhappy:

  plugin: 3.0.5

  wrapper: 1.0.28(VPUWRAPPER_ARM_LINUX Build on Oct 22 2013 15:28:56)

  vpulib: 5.4.10

  firmware: 2.1.8.34588

Aiur: 3.0.5

Core: MKVPARSER_01.04.01  build on Dec 10 2012 15:43:34

  mime: video/x-matroska ; video/webm

  file: /usr/lib/imx-mm/parser/lib_mkv_parser_arm11_elinux.so

Content Info:

  URI:

       file:///jci/mms/test_video

  Idx File:

       /root/.aiur/.jci.mms.test_video.aidx

  Seekable  : Yes

  Size(byte): 211312391

Movie Info:

  Seekable  : Yes

  Live      : No

  Duration  : 0:22:50.721000000

  ReadMode  : Track

  Track     : 5

Track 00 [video_000000] Enabled

  Duration: 0:22:50.721000000

  Language: und

  Mime:

       video/x-h264, parsed=(boolean)true, width=(int)720, height=(int)400, framerate=(

       fraction)29/1, codec_data=(buffer)00000001674d40339e2101686742000007d20001d4c11e

       3065400000000168ee3c80

Track 01 [audio_000000] Enabled

  Duration: 0:22:50.721000000

  Language: eng

  Mime:

       audio/mpeg, mpegversion=(int)1, framed=(boolean)true, channels=(int)2, rate=(int

       )48000, bitrate=(int)0

Track 02 [audio_000001] Enabled

  Duration: 0:22:50.721000000

  Language: jpn

  Mime:

       audio/mpeg, mpegversion=(int)1, framed=(boolean)true, channels=(int)2, rate=(int

       )48000, bitrate=(int)0

Track 03 [subtitle]: Disabled

  Codec: 0, SubCodec: 0

Track 04 [subtitle]: Disabled

  Codec: 0, SubCodec: 0

Beep: 3.0.5

Core: MP3 decoder Wrapper  build on Nov 29 2012 14:50:12

  mime: audio/mpeg, mpegversion = (int)1

  file: /usr/lib/imx-mm/audio-codec/wrap/lib_mp3d_wrap_arm12_elinux.so

CODEC: BLN_MAD-MMCODECS_MP3D_ARM_02.13.00_CORTEX-A8  build on Dec  5 2012 09:45:19.

>>V4L_SINK: Actually buffer status:

  hardware buffer : 9

  software buffer : 0

full screen size:800x480

[V4L Update Display]: left=0, top=20, width=800, height=440

Running time 0:00:00.064000000 render fps 31.250

full screen size:800x480

Running time 0:00:05.216000000 render fps 28.949

Total rendered:151

[--->FINALIZE aiurdemux

MFW_GST_V4LSINK_PLUGIN 3.0.5 build on Oct 22 2013 15:33:00.

set stretch:0.

vpudec versions :smileyhappy:

  plugin: 3.0.5

  wrapper: 1.0.28(VPUWRAPPER_ARM_LINUX Build on Oct 22 2013 15:28:56)

  vpulib: 5.4.10

  firmware: 2.1.8.34588

Aiur: 3.0.5

Core: MKVPARSER_01.04.01  build on Dec 10 2012 15:43:34

  mime: video/x-matroska ; video/webm

  file: /usr/lib/imx-mm/parser/lib_mkv_parser_arm11_elinux.so

Content Info:

  URI:

       file:///jci/mms/test_video

  Idx File:

       /root/.aiur/.jci.mms.test_video.aidx

  Seekable  : Yes

  Size(byte): 211312391

Movie Info:

  Seekable  : Yes

  Live      : No

  Duration  : 0:22:50.721000000

  ReadMode  : Track

  Track     : 5

Track 00 [video_000000] Enabled

  Duration: 0:22:50.721000000

  Language: und

  Mime:

       video/x-h264, parsed=(boolean)true, width=(int)720, height=(int)400, framerate=(

       fraction)29/1, codec_data=(buffer)00000001674d40339e2101686742000007d20001d4c11e

       3065400000000168ee3c80

Track 01 [audio_000000] Enabled

  Duration: 0:22:50.721000000

  Language: eng

  Mime:

       audio/mpeg, mpegversion=(int)1, framed=(boolean)true, channels=(int)2, rate=(int

       )48000, bitrate=(int)0

Track 02 [audio_000001] Enabled

  Duration: 0:22:50.721000000

  Language: jpn

  Mime:

       audio/mpeg, mpegversion=(int)1, framed=(boolean)true, channels=(int)2, rate=(int

       )48000, bitrate=(int)0

Track 03 [subtitle]: Disabled

  Codec: 0, SubCodec: 0

Track 04 [subtitle]: Disabled

  Codec: 0, SubCodec: 0

Beep: 3.0.5

Core: MP3 decoder Wrapper  build on Nov 29 2012 14:50:12

  mime: audio/mpeg, mpegversion = (int)1

  file: /usr/lib/imx-mm/audio-codec/wrap/lib_mp3d_wrap_arm12_elinux.so

CODEC: BLN_MAD-MMCODECS_MP3D_ARM_02.13.00_CORTEX-A8  build on Dec  5 2012 09:45:19.

>>V4L_SINK: Actually buffer status:

  hardware buffer : 0

  software buffer : 0

full screen size:800x480

[V4L Update Display]: left=0, top=20, width=800, height=440

Running time 0:00:01.024000000 render fps 0.000

Total rendered:0

[--->FINALIZE aiurdemux

[lachezarnikolov]

Anyone has some other idea why this is happening?

[YixingKong]

Lachezar

This discussion is closed since no activity. If you still need help, please feel free to reply with an update to this discussion, or create another discussion.

Thanks,

Yixing

[YixingKong]

Lachezar

This discussion is closed since no activity. If you still need help, please feel free to reply with an update to this discussion, or create another discussion.

Thanks,

Yixing

[YixingKong]

Lachezar

Had your issue got resolved? If yes, we are going to close the discussion in 3 days. If you still need help, please feel free to reply with an update to this discussion.

Thanks,

Yixing

[LeonardoSandova]

Do you observe the same issue when you use mfw_ipucsc instead of ffmpegcolorspace?

[lachezarnikolov]

Yes, the same issue is observed. What is mfw_ipucsc? Should I use it instead of ffmpegcolorspace?

Also is this line: hardware buffer : 0 significant? Because when it is zero then no picture is displayed.

I forgot to add that both queues have these attributes set:

max-size-buffers = 0

max-size-time = 0

I saw this in some example. If i leave them to their default values the video playback on some video files stops after some time (i don't know why). Can this have anything to do with the problem?

[LeonardoSandova]

Yes, use mfw_* as color space element, it is HW accelerated. I can not say much about the 'hardware buffer : 0'. Can you try  to replicate the issue just including video elements? Try removing 'useless' elements for the moment, like queues or even the color space convertor, which I think you do not need it.

[lachezarnikolov]

Hi

I left the pipeline looking like this:

filesrc->aiurdemux->vpudec->mfw_v4lsink

And now the issue is no longer present. Here is the log:

MFW_GST_V4LSINK_PLUGIN 3.0.5 build on Oct 22 2013 15:33:00.

set stretch:0.

vpudec versions :smileyhappy:

  plugin: 3.0.5

  wrapper: 1.0.28(VPUWRAPPER_ARM_LINUX Build on Oct 22 2013 15:28:56)

  vpulib: 5.4.10

  firmware: 2.1.8.34588

Aiur: 3.0.5

Core: MKVPARSER_01.04.01  build on Dec 10 2012 15:43:34

  mime: video/x-matroska ; video/webm

  file: /usr/lib/imx-mm/parser/lib_mkv_parser_arm11_elinux.so

Content Info:

  URI:

      file:///jci/mms/test_video

  Idx File:

      /root/.aiur/.jci.mms.test_video.aidx

  Seekable  : Yes

  Size(byte): 211312391

Movie Info:

  Seekable  : Yes

  Live      : No

  Duration  : 0:22:50.721000000

  ReadMode  : Track

  Track    : 5

Track 00 [video_000000] Enabled

  Duration: 0:22:50.721000000

  Language: und

  Mime:

      video/x-h264, parsed=(boolean)true, width=(int)720, height=(int)400, framerate=(

      fraction)29/1, codec_data=(buffer)00000001674d40339e2101686742000007d20001d4c11e

      3065400000000168ee3c80

Track 01 [audio_000000] Enabled

  Duration: 0:22:50.721000000

  Language: eng

  Mime:

      audio/mpeg, mpegversion=(int)1, framed=(boolean)true, channels=(int)2, rate=(int

      )48000, bitrate=(int)0

Track 02 [audio_000001] Enabled

  Duration: 0:22:50.721000000

  Language: jpn

  Mime:

      audio/mpeg, mpegversion=(int)1, framed=(boolean)true, channels=(int)2, rate=(int

      )48000, bitrate=(int)0

Track 03 [subtitle]: Disabled

  Codec: 0, SubCodec: 0

Track 04 [subtitle]: Disabled

  Codec: 0, SubCodec: 0

>>V4L_SINK: Actually buffer status:

  hardware buffer : 9

  software buffer : 0

full screen size:800x480

[V4L Update Display]: left=0, top=20, width=800, height=440

Running time 0:00:00.036374666 render fps 54.983

full screen size:800x480

Running time 0:01:06.055479006 render fps 29.975

Total rendered:1980

[--->FINALIZE v4l_sink

[--->FINALIZE aiurdemux

MFW_GST_V4LSINK_PLUGIN 3.0.5 build on Oct 22 2013 15:33:00.

set stretch:0.

vpudec versions :smileyhappy:

  plugin: 3.0.5

  wrapper: 1.0.28(VPUWRAPPER_ARM_LINUX Build on Oct 22 2013 15:28:56)

  vpulib: 5.4.10

  firmware: 2.1.8.34588

Aiur: 3.0.5

Core: MKVPARSER_01.04.01  build on Dec 10 2012 15:43:34

  mime: video/x-matroska ; video/webm

  file: /usr/lib/imx-mm/parser/lib_mkv_parser_arm11_elinux.so

Content Info:

  URI:

      file:///jci/mms/test_video

  Idx File:

      /root/.aiur/.jci.mms.test_video.aidx

  Seekable  : Yes

  Size(byte): 211312391

Movie Info:

  Seekable  : Yes

  Live      : No

  Duration  : 0:22:50.721000000

  ReadMode  : Track

  Track    : 5

Track 00 [video_000000] Enabled

  Duration: 0:22:50.721000000

  Language: und

  Mime:

      video/x-h264, parsed=(boolean)true, width=(int)720, height=(int)400, framerate=(

      fraction)29/1, codec_data=(buffer)00000001674d40339e2101686742000007d20001d4c11e

      3065400000000168ee3c80

Track 01 [audio_000000] Enabled

  Duration: 0:22:50.721000000

  Language: eng

  Mime:

      audio/mpeg, mpegversion=(int)1, framed=(boolean)true, channels=(int)2, rate=(int

      )48000, bitrate=(int)0

Track 02 [audio_000001] Enabled

  Duration: 0:22:50.721000000

  Language: jpn

  Mime:

      audio/mpeg, mpegversion=(int)1, framed=(boolean)true, channels=(int)2, rate=(int

      )48000, bitrate=(int)0

Track 03 [subtitle]: Disabled

  Codec: 0, SubCodec: 0

Track 04 [subtitle]: Disabled

  Codec: 0, SubCodec: 0

>>V4L_SINK: Actually buffer status:

  hardware buffer : 9

  software buffer : 0

full screen size:800x480

[V4L Update Display]: left=0, top=20, width=800, height=440

Running time 0:00:00.036184333 render fps 55.273

full screen size:800x480

Running time 0:00:11.045527667 render fps 30.057

full screen size:800x480

Running time 0:00:16.552981001 render fps 30.025

Total rendered:497

[--->FINALIZE v4l_sink

[--->FINALIZE aiurdemux

MFW_GST_V4LSINK_PLUGIN 3.0.5 build on Oct 22 2013 15:33:00.

set stretch:0.

vpudec versions :smileyhappy:

  plugin: 3.0.5

  wrapper: 1.0.28(VPUWRAPPER_ARM_LINUX Build on Oct 22 2013 15:28:56)

  vpulib: 5.4.10

  firmware: 2.1.8.34588

Aiur: 3.0.5

Core: MKVPARSER_01.04.01  build on Dec 10 2012 15:43:34

  mime: video/x-matroska ; video/webm

  file: /usr/lib/imx-mm/parser/lib_mkv_parser_arm11_elinux.so

Content Info:

  URI:

      file:///jci/mms/test_video

  Idx File:

      /root/.aiur/.jci.mms.test_video.aidx

  Seekable  : Yes

  Size(byte): 211312391

Movie Info:

  Seekable  : Yes

  Live      : No

  Duration  : 0:22:50.721000000

  ReadMode  : Track

  Track    : 5

Track 00 [video_000000] Enabled

  Duration: 0:22:50.721000000

  Language: und

  Mime:

      video/x-h264, parsed=(boolean)true, width=(int)720, height=(int)400, framerate=(

      fraction)29/1, codec_data=(buffer)00000001674d40339e2101686742000007d20001d4c11e

      3065400000000168ee3c80

Track 01 [audio_000000] Enabled

  Duration: 0:22:50.721000000

  Language: eng

  Mime:

      audio/mpeg, mpegversion=(int)1, framed=(boolean)true, channels=(int)2, rate=(int

      )48000, bitrate=(int)0

Track 02 [audio_000001] Enabled

  Duration: 0:22:50.721000000

  Language: jpn

  Mime:

      audio/mpeg, mpegversion=(int)1, framed=(boolean)true, channels=(int)2, rate=(int

      )48000, bitrate=(int)0

Track 03 [subtitle]: Disabled

  Codec: 0, SubCodec: 0

Track 04 [subtitle]: Disabled

  Codec: 0, SubCodec: 0

>>V4L_SINK: Actually buffer status:

  hardware buffer : 9

  software buffer : 0

full screen size:800x480

[V4L Update Display]: left=0, top=20, width=800, height=440

Running time 0:00:00.036323666 render fps 55.061

Maybe the queue creates problems. But I will need both the video and audio.

Also I tried to use decodebin2 instead of manually constructing the pipeline and the issue is reproducible.

I also tested the pipeline as it is but without the audio branch. The problem is NOT reproducible then.

[LeonardoSandova]

Have you used the playbin2 element? That would construct a similar pipeline including audio and video elements.

[lachezarnikolov]

I cannot use playbin2 element as autovideosink element is missing from my environment. I've used decodebin2 and the problem still occurs. Even if playbin2 worked I cannot use it in my situation because I need some flexibility in chosing streams in the file.