IMX6q and Gstreamer : How to play smoothly a TS stream ?

fredericduchass · ‎10-10-2017

Hello,

I try using gstreamer on IMX6q (Yocto based OS) in order to play a TS stream. This stream is located on a closed network using UDP.

I use this pipeline:

gst-launch-1.0 udpsrc address=239.0.0.1 port=1234 buffer-size=20000000 ! tsdemux name=d program-number=257 ! multiqueue name=q use-buffering=1 d. ! q. q. ! queue ! decodebin ! queue ! imxg2dvideosink q. ! queue ! decodebin ! queue ! audioconvert ! volume volume=10 ! alsasink

The program number 257 is H264 1080i @ 30fps video with ac-3 audio.

The problem is that using this pipeline, i get freeze sometimes and i get message from imxg2dvideosink about dropping frame because too late.

Have you got an idea about how i can improve my pipeline ?

(I try to delete all the queues but it's bader than before)

BR

Fred

igorpadykov · ‎10-10-2017

Hi Frederic

one can try with nxp gstreamer plugins using sect.7.3.13 RTP/UDP MPEGTS streaming

attached Linux Guide. Other documents can be obtained from linux documentation:

https://www.nxp.com/webapp/Download?colCode=L4.1.15_2.1.0_LINUX_DOCS

As imxg2dvideosink is part of gstreamer-imx plugins, it is supported by community and

issue may be posted on meta-fsl-arm mailing list, so that someone familiar with it could try to assist you.
https://lists.yoctoproject.org/listinfo/meta-freescale

Difference between community gstreamer-imx plugins and nxp gstreamer plugins described on

trac.gateworks.com/wiki/Yocto/gstreamer

Best regards
igor
-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

fredericduchass · ‎10-11-2017

root@apalis-imx6:/home# gst-launch-1.0 udpsrc do-timestamp=false uri=udp://239.0.0.1:1234 caps="video/mpegts" ! aiurdemux streaming_latency=400 name=d ! queue ! decodebin ! queue ! imxg2dvideosink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...

====== AIUR: 4.0.9 build on Oct 11 2017 10:52:36. ======
Core: BLN_MAD-MMLAYER_MPG2PARSER_04.04.47 build on Feb 3 2016 05:52:16
file: /usr/lib/imx-mm/parser/lib_mpg2_parser_arm11_elinux.so.3.1
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Mpeg2CreateParser:parser created successfully
------------------------
Track 00 [video_0] Enabled
Duration: 0:00:00.000000000
Language: und
Mime:
video/x-h264, parsed=(boolean)true, alignment=(string)au, stream-format=(string)byte-stream, width=(int)176, height=(int)144, framerate=(fraction)30/1
------------------------
------------------------
Track 01 [audio_0] Enabled
Duration: 0:00:00.000000000
Language: fre
Mime:
audio/eac3, channels=(int)2, rate=(int)44100, bitrate=(int)0
------------------------
------------------------
Track 02 [audio_1] Enabled
Duration: 0:00:00.000000000
Language: qad
Mime:
audio/eac3, channels=(int)2, rate=(int)44100, bitrate=(int)0
------------------------
------------------------
Track 03 [audio_2] Enabled
Duration: 0:00:00.000000000
Language: qaa
Mime:
audio/eac3, channels=(int)2, rate=(int)44100, bitrate=(int)0
------------------------
[INFO] bitstreamMode 1, chromaInterleave 0, mapType 0, tiled2LinearEnable 0
[INFO] bitstreamMode 1, chromaInterleave 0, mapType 0, tiled2LinearEnable 0

In Fact, my video stream is 1920x1080i@25fps H264.

igorpadykov · ‎10-11-2017

Hi Frederic

what bsp used in the case, one can try with nxp releases described in i.MX_Yocto_Project_User's_Guide.pdf

Table 1. Freescale Yocto project images included in doc package

https://www.nxp.com/webapp/Download?colCode=L4.1.15_2.1.0_LINUX_DOCS

Best regards
igor

fredericduchass · ‎10-13-2017

Hi Igor,

I don't really know if it's a clue but i try to read buffer level when I see problems on the screen :

bufferdecodebinvideo level : 1995915
buffervideo level : 7311362
bufferdecodebinaudio level : 0
bufferaudio level : 540672
bufferdecodebinvideo level : 1792253
buffervideo level : 7311362
bufferdecodebinaudio level : 0
bufferaudio level : 602112
bufferdecodebinvideo level : 2160186
buffervideo level : 3655681
bufferdecodebinaudio level : 0
bufferaudio level : 577536
bufferdecodebinvideo level : 2151265
buffervideo level : 0
bufferdecodebinaudio level : 0
bufferaudio level : 565248
bufferdecodebinvideo level : 2033225
buffervideo level : 3655681
bufferdecodebinaudio level : 0
bufferaudio level : 552960
bufferdecodebinvideo level : 2153602
buffervideo level : 0
bufferdecodebinaudio level : 0
bufferaudio level : 540672
bufferdecodebinvideo level : 2028605
buffervideo level : 3655681
bufferdecodebinaudio level : 0
bufferaudio level : 589824

SO I can conclude that the buffer after video decoding can be empty sometimes.

So the problem come from video decoder ?

Have you got an idea ?

BR

Frederic

igorpadykov · ‎10-13-2017

Hi Frederic

if you are using bsp from toradex, this may be posted on toradex support,

as it uses community gstreamer-imx plugins which are different from nxp gstreamer plugins

Toradex - Customer Support

more about differencies in trac.gateworks.com/wiki/Yocto/gstreamer

Best regards
igor

fredericduchass · ‎10-17-2017

Hi Igor,

I've asked to Toradex to look at this post. I hope we will find a solution

:-((((((((

Setting debug level of IMX, i can get these messages :

0:02:44.228679353 369 0x9472f0 DEBUG imxvpuapi imxvpuapi_vpulib.c:2171:imx_vpu_dec_decode: not enough input data was available
0:02:44.228850353 369 0x9472f0 DEBUG imxvpudecoder decoder.c:581:gst_imx_vpu_decoder_handle_frame:<imxvpudecoder0> input gstframe is incomplete; discarding this gstframe
0:02:44.281267019 369 0x9472f0 DEBUG imxvpuapi imxvpuapi_vpulib.c:2171:imx_vpu_dec_decode: not enough input data was available
0:02:44.281410353 369 0x9472f0 DEBUG imxvpudecoder decoder.c:581:gst_imx_vpu_decoder_handle_frame:<imxvpudecoder0> input gstframe is incomplete; discarding this gstframe

Can you explain me what it means ?

BR

Frederic

fredericduchass · ‎10-24-2017

Hi Igor,

Can you help me to debug my problem please ?

I send the question to Toradex but i get no news...

Now i get this message debugging imxvpudec :

0:00:43.572198005 [332m 1357 [00m 0xcb8690 [37mDEBUG [00m [00m imxvpudecoder decoder.c:581:gst_imx_vpu_decoder_handle_frame:<imxvpudecoder0> [00m input gstframe is incomplete; discarding this gstframe
0:00:43.608671672 [332m 1357 [00m 0xcb8690 [37mDEBUG [00m [00m imxvpuapi imxvpuapi_vpulib.c:2171:imx_vpu_dec_decode: [00m not enough input data was available
0:00:43.608792339 [332m 1357 [00m 0xcb8690 [37mDEBUG [00m [00m imxvpudecoder decoder.c:581:gst_imx_vpu_decoder_handle_frame:<imxvpudecoder0> [00m input gstframe is incomplete; discarding this gstframe
0:00:43.648815672 [332m 1357 [00m 0xcb8690 [37mDEBUG [00m [00m imxvpuapi imxvpuapi_vpulib.c:2171:imx_vpu_dec_decode: [00m not enough input data was available
0:00:43.648937339 [332m 1357 [00m 0xcb8690 [37mDEBUG [00m [00m imxvpudecoder decoder.c:581:gst_imx_vpu_decoder_handle_frame:<imxvpudecoder0> [00m input gstframe is incomplete; discarding this gstframe
0:00:43.688845005 [332m 1357 [00m 0xcb8690 [37mDEBUG [00m [00m imxvpuapi imxvpuapi_vpulib.c:2171:imx_vpu_dec_decode: [00m not enough input data was available

Do you know the meaning ?

Thanks in advance

Frederic

igorpadykov · ‎10-24-2017

Hi Frederic

one can try with nxp plugins, check

How to install the fsl codec on linux?

Best regards
igor

fredericduchass · ‎10-25-2017

Hi Igor,

OK I'll try with nxp plugin (toradex image < 2.5) and I'll keep you updated.

Best regards

Frederic

fredericduchass · ‎10-26-2017

Hi Igor,

Using gstreamer0.1 I find other problems : aiurdemux can work with some TS but it doesn't work with others !

I use this pipeline to check :

gst-launch-0.10 udpsrc filesrc location=/home/TNT_M6.ts ! "video/mpegts" ! queue ! aiurdemux streaming_latency=400 name=d d. ! queue max-size-time=0 ! vpudec ! imxv4l2sink

this is the log :

root@apalis-imx6:/home# gst-launch-0.10 udpsrc filesrc location=/home/TNT_M6.ts ! "video/mpegts" ! queue ! aiurdemux streaming_latency=400 name=d d. ! queue max-size-time=0 ! vpudec ! imxv4l2sink
[ 1420.389823] ERROR: v4l2 capture: slave not found! V4L2_CID_HUE
[ 1420.395747] ERROR: v4l2 capture: slave not found! V4L2_CID_HUE
[ 1420.401603] ERROR: v4l2 capture: slave not found! V4L2_CID_HUE
MFW_GST_V4LSINK_PLUGIN 4.0.3 build on Oct 25 2017 12:12:13.
Setting pipeline to PAUSED ...
[INFO] Product Info: i.MX6Q/D/S
vpudec versions :)
plugin: 4.0.3
wrapper: 1.0.58(VPUWRAPPER_ARM_LINUX Build on Oct 25 2017 12:03:31)
vpulib: 5.4.28
firmware: 3.1.1.46063
0:00:00.115699333 788 0xb82cc0 DEBUG aiurdemux /media/sefram/7f7dd387-2c00-4810-babd-e0d994a86f25/oe-core/build/out-glibc/work/apalis_imx6-angstrom-linux-gnueabi/gst-fsl-plugin/4.0.3-r0/gst-fsl-plugins-4.0.3/src/parser/aiur/src/aiurdemux.c:1908:gst_aiurdemux_handle_sink_event:<d> Pushing newseg update 0, rate 1, applied rate 1, format 3, start 0:00:00.000000000, stop 0:00:00.024107616
Pipeline is live and does not need PREROLL ...
Aiur: 4.0.3
Core: BLN_MAD-MMLAYER_MPG2PARSER_04.04.26 build on Jan 20 2015 01:05:12
mime: video/mpeg, mpegversion=(int)[1,2]; video/mpegts, systemstream=true; video/x-cdxa
file: /usr/lib/imx-mm/parser/lib_mpg2_parser_arm11_elinux.so.3.1
Content Info:
URI:
file:///home/TNT_M6.ts
Idx File:
/home/root/.aiur/.home.TNT_M6.ts.aidx
Seekable : Yes
Size(byte): 24107616
Setting pipeline to PLAYING ...
New clock: GstSystemClock
0:00:00.121037333 788 0xb82cf0 WARN aiurdemux /media/sefram/7f7dd387-2c00-4810-babd-e0d994a86f25/oe-core/build/out-glibc/work/apalis_imx6-angstrom-linux-gnueabi/gst-fsl-plugin/4.0.3-r0/gst-fsl-plugins-4.0.3/src/parser/aiur/src/aiurdemux.c:2358:aiurdemux_loop_state_init: API[createParser2] failed, ret=-1
0:00:00.121144333 788 0xb82cf0 LOG aiurdemux /media/sefram/7f7dd387-2c00-4810-babd-e0d994a86f25/oe-core/build/out-glibc/work/apalis_imx6-angstrom-linux-gnueabi/gst-fsl-plugin/4.0.3-r0/gst-fsl-plugins-4.0.3/src/parser/aiur/src/aiurdemux.c:2359:aiurdemux_loop_state_init:<d> aiurdemux_loop_state_init create using createParser2 flag=4
0:00:00.121207000 788 0xb82cf0 LOG aiurdemux /media/sefram/7f7dd387-2c00-4810-babd-e0d994a86f25/oe-core/build/out-glibc/work/apalis_imx6-angstrom-linux-gnueabi/gst-fsl-plugin/4.0.3-r0/gst-fsl-plugins-4.0.3/src/parser/aiur/src/aiurdemux.c:4450:aiurdemux_push_task:<d> pausing task, reason error
0:00:00.121285666 788 0xb82cf0 WARN aiurdemux /media/sefram/7f7dd387-2c00-4810-babd-e0d994a86f25/oe-core/build/out-glibc/work/apalis_imx6-angstrom-linux-gnueabi/gst-fsl-plugin/4.0.3-r0/gst-fsl-plugins-4.0.3/src/parser/aiur/src/aiurdemux.c:4465:aiurdemux_push_task:<d> error: streaming stopped, reason error
ERROR: from element /GstPipeline:pipeline0/GstAiurDemux:d: GStreamer encountered a general stream error.
Additional debug info:
/media/sefram/7f7dd387-2c00-4810-babd-e0d994a86f25/oe-core/build/out-glibc/work/apalis_imx6-angstrom-linux-gnueabi/gst-fsl-plugin/4.0.3-r0/gst-fsl-plugins-4.0.3/src/parser/aiur/src/aiurdemux.c(4465): aiurdemux_push_task (): /GstPipeline:pipeline0/GstAiurDemux:d:
streaming stopped, reason error
Execution ended after 2428000 ns.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Total rendered:0
Freeing pipeline ...
[--->FINALIZE v4l_sink
[--->FINALIZE aiurdemux
root@apalis-imx6:/home#

This is the link you can find my TS if you want to check:

https://drive.google.com/file/d/0B-hlgYw-WA9bb0VmMXJNc2lXa28/view?usp=sharing

BR

Frederic

igorpadykov · ‎10-26-2017

Hi Frederic

GStreamer 0.10 is not supported in latest bsps, please try with GStreamer 1.0

description in attached file.

Best regards
igor

fredericduchass · ‎10-12-2017

Hello Igor,

I'm working on Toradex Apalis Imx6q module. So i use the BSP toradex.

I also tried the Boot2Qt image for this module and same problem.

BTW : I don't really know if it's a good idea to use "aiurdemux". It seems to be depeciated because it has a lot of problems. But as i see in the documentation you gave me, the only big difference (compared with my pipeline) is the use of aiurdemux :

$GSTL udpsrc do-timestamp=false uri=udp://10.192.241.255:10000 caps="video/mpegts"
! aiurdemux streaming_latency=400 name=d d. ! queue ! vpudec
! queue ! overlaysink sync=true d. ! queue ! beepdec ! pulsesink sync=true

Best regards

Frederic

fredericduchass · ‎10-11-2017

Hi Igor,

First thanks a lot for your answer.

Reading the specified chapter, i've seen that aiurdemux was not included in my Yocto image. So i've added the correct recipe to get aiurdemux.

Now i can use "aiurdemux" in my pipeline. But i always have a problem. At the output of aiurdemux, caps of the stream gives me a wrong size for width and height.and so, the decoding is bad....

Have you got an idea about why aiurdemux give me wrong width and height for my stream ???

Frederic

IMX6q and Gstreamer : How to play smoothly a TS stream ?

IMX6q and Gstreamer : How to play smoothly a TS stream ?

i.MX6Quad

Linux

Multimedia

Yocto Project