i.MX ProcessorsのトピックRe: RTSP gstreamer and vpudec

RTSP gstreamer and vpudec

holgerweber — Thu, 23 Jan 2014 16:05:19 GMT

Hello,

I try to display a video stream with gstreamer. I use the following command, which works:

gst-launch-0.10 rtspsrc location=rtsp://192.168.25.60/stream1.sdp ! rtpmp2tdepay ! mpegtsdemux ! h264parse ! ffdec_h264 max-threads=2 ! mfw_v4lsink

The performance is not as good as it should be so I want to use the hw decoder:

gst-launch-0.10 rtspsrc location=rtsp://192.168.25.60/stream1.sdp ! rtpmp2tdepay ! mpegtsdemux ! vpudec ! mfw_v4lsink

But nothing happens, no video is displayed. I only get this output:

MFW_GST_V4LSINK_PLUGIN 3.0.5 build on Dec 14 2012 13:58:00.

Setting pipeline to PAUSED ...

[INFO] Product Info: i.MX6Q/D/S

vpudec versions :smileyhappy:

plugin: 3.0.5

wrapper: 1.0.28(VPUWRAPPER_ARM_LINUX Build on Dec 14 2012 12:23:57)

vpulib: 5.4.10

firmware: 2.1.8.34588

Pipeline is live and does not need PREROLL ...

Setting pipeline to PLAYING ...

New clock: GstSystemClock

[INFO] bitstreamMode 1, chromaInterleave 1, mapType 0, tiled2LinearEnable 0

[WARN] VPU iram is less than needed, some parts don't use iram

>>V4L_SINK: Actually buffer status:

hardware buffer : 24

software buffer : 0

I also tried aiurdemux which replaces mpegtsdemux but there is a bug with rtsp streaming.

Any ideas?

Re: RTSP gstreamer and vpudec

LeonardoSandova — Thu, 23 Jan 2014 17:06:22 GMT

Any reason you did not use h264parse on the second pipeline?

Please enable debugging when launching your pipeline (--gst-debug=*:5)

Leo

Re: RTSP gstreamer and vpudec

holgerweber — Fri, 24 Jan 2014 08:47:40 GMT

You right, I changed the command to:

gst-launch-0.10 --gst-debug=vpudec:5 rtspsrc location=rtsp://192.168.25.60/stream1.sdp ! rtpmp2tdepay ! mpegtsdemux ! h264parse ! vpudec ! mfw_v4lsink

With no success but I get this messages:

MFW_GST_V4LSINK_PLUGIN 3.0.5 build on Dec 14 2012 13:58:00.

Setting pipeline to PAUSED ...

[INFO] Product Info: i.MX6Q/D/S

vpudec versions :smileyhappy:

plugin: 3.0.5

wrapper: 1.0.28(VPUWRAPPER_ARM_LINUX Build on Dec 14 2012 12:23:57)

vpulib: 5.4.10

firmware: 2.1.8.34588

0:00:00.178237667 7627 0x17050 LOG vpudec vpudec.c:392:vpudec_core_mem_alloc_dma_buffer: Call VPU_DecGetMem return 0x0

Pipeline is live and does not need PREROLL ...

Setting pipeline to PLAYING ...

New clock: GstSystemClock

0:00:02.267122001 7627 0x2e700798 INFO vpudec vpudec.c:1812:gst_vpudec_sink_event: Get newsegment event from 0:43:59.915622222to 99:99:99.999999999 pos 0:00:00.000000000

0:00:02.267608335 7627 0x2e700798 INFO vpudec vpudec.c:1181:gst_vpudec_setcaps: Get upstream caps video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, parsed=(boolean)true

0:00:02.268074668 7627 0x2e700798 INFO vpudec vpudec.c:1190:gst_vpudec_setcaps: Get codec std 6

0:00:02.268709668 7627 0x2e700798 INFO vpudec vpudec.c:1230:gst_vpudec_setcaps: got downstream allow caps video/x-raw-yuv, format=(fourcc)NV12, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw-yuv, format=(fourcc)I420, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw-yuv, format=(fourcc)TNVP, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw-yuv, format=(fourcc)YV12, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw-yuv, format=(fourcc)TNVF, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw-yuv, format=(fourcc)YV12, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]

[INFO] bitstreamMode 1, chromaInterleave 1, mapType 0, tiled2LinearEnable 0

0:00:02.272831001 7627 0x2e700798 INFO vpudec vpudec.c:1283:gst_vpudec_setcaps: Use new tsm scheme

0:00:02.273043001 7627 0x2e700798 INFO vpudec vpudec.c:1165:gst_vpudec_setconfig: Set drop policy 0

0:00:02.273333335 7627 0x2e700798 LOG vpudec vpudec.c:1414:gst_vpudec_chain: Chain in with size = 30698

0:00:02.274277001 7627 0x2e700798 LOG vpudec vpudec.c:1467:gst_vpudec_chain: buf status 0x101 data 30698

0:00:02.274480335 7627 0x2e700798 INFO vpudec vpudec.c:1555:gst_vpudec_chain: Got not enough input message!!

0:00:02.281955001 7627 0x2e700798 LOG vpudec vpudec.c:1414:gst_vpudec_chain: Chain in with size = 30514

0:00:02.282935001 7627 0x2e700798 LOG vpudec vpudec.c:1467:gst_vpudec_chain: buf status 0x101 data 30514

0:00:02.283137001 7627 0x2e700798 INFO vpudec vpudec.c:1555:gst_vpudec_chain: Got not enough input message!!

And after some more of them:

0:00:02.413860001 7627 0x2e700798 INFO vpudec vpudec.c:1555:gst_vpudec_chain: Got not enough input message!!

0:00:02.423047001 7627 0x2e700798 INFO vpudec vpudec.c:1181:gst_vpudec_setcaps: Get upstream caps video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, width=(int)640, height=(int)360, framerate=(fraction)15/1, parsed=(boolean)true, pixel-aspect-ratio=(fraction)1/1

0:00:02.423533668 7627 0x2e700798 INFO vpudec vpudec.c:1190:gst_vpudec_setcaps: Get codec std 6

0:00:02.423642668 7627 0x2e700798 INFO vpudec vpudec.c:1194:gst_vpudec_setcaps: Set Width 640

0:00:02.423720335 7627 0x2e700798 INFO vpudec vpudec.c:1198:gst_vpudec_setcaps: Set Height 360

0:00:02.424047001 7627 0x2e700798 INFO vpudec vpudec.c:1230:gst_vpudec_setcaps: got downstream allow caps video/x-raw-yuv, format=(fourcc)NV12, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw-yuv, format=(fourcc)I420, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw-yuv, format=(fourcc)TNVP, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw-yuv, format=(fourcc)YV12, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw-yuv, format=(fourcc)TNVF, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw-yuv, format=(fourcc)YV12, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]

[INFO] bitstreamMode 1, chromaInterleave 1, mapType 0, tiled2LinearEnable 0

0:00:02.439768668 7627 0x2e700798 INFO vpudec vpudec.c:1283:gst_vpudec_setcaps: Use new tsm scheme

0:00:02.439918001 7627 0x2e700798 INFO vpudec vpudec.c:1165:gst_vpudec_setconfig: Set drop policy 0

0:00:02.440091335 7627 0x2e700798 LOG vpudec vpudec.c:1414:gst_vpudec_chain: Chain in with size = 30698

[WARN] VPU iram is less than needed, some parts don't use iram

0:00:02.440803668 7627 0x2e700798 LOG vpudec vpudec.c:1467:gst_vpudec_chain: buf status 0x201 data 30698

0:00:02.440956335 7627 0x2e700798 INFO vpudec vpudec.c:1125:gst_vpudec_core_start: Get min framebuffer count 18

0:00:02.441048001 7627 0x2e700798 INFO vpudec vpudec.c:1025:gst_vpudec_set_downstream_pad: Get initinfo padwidth 640, padheight 368, left 0, right 640, top 0, bottom 360, interlace 0

0:00:02.441275001 7627 0x2e700798 INFO vpudec vpudec.c:1074:gst_vpudec_set_downstream_pad: Try set downstream caps video/x-raw-yuv, format=(fourcc)NV12, width=(int)640, height=(int)368, pixel-aspect-ratio=(fraction)1/1, crop-top=(int)0, crop-left=(int)0, crop-right=(int)0, crop-bottom=(int)8, num-buffers-required=(int)24, framerate=(fraction)15/1, alignment=(int)1

0:00:02.447800001 7627 0x2e700798 LOG vpudec vpudec.c:392:vpudec_core_mem_alloc_dma_buffer: Call VPU_DecGetMem return 0x0

>>V4L_SINK: Actually buffer status:

hardware buffer : 24

software buffer : 0

0:00:02.515363001 7627 0x2e700798 LOG vpudec vpudec.c:1467:gst_vpudec_chain: buf status 0x821 data 0

0:00:02.516016668 7627 0x2e700798 INFO vpudec vpudec.c:1512:gst_vpudec_chain: Got drop information!!

0:00:02.521368335 7627 0x2e700798 LOG vpudec vpudec.c:1414:gst_vpudec_chain: Chain in with size = 30890

0:00:02.521752001 7627 0x2e700798 LOG vpudec vpudec.c:1467:gst_vpudec_chain: buf status 0x1 data 30890

0:00:02.522304335 7627 0x2e700798 LOG vpudec vpudec.c:1467:gst_vpudec_chain: buf status 0x821 data 0

0:00:02.522413668 7627 0x2e700798 INFO vpudec vpudec.c:1512:gst_vpudec_chain: Got drop information!!

0:00:02.525947001 7627 0x2e700798 LOG vpudec vpudec.c:1414:gst_vpudec_chain: Chain in with size = 30698

0:00:02.526490335 7627 0x2e700798 LOG vpudec vpudec.c:1467:gst_vpudec_chain: buf status 0x1 data 30698

0:00:02.527030001 7627 0x2e700798 LOG vpudec vpudec.c:1467:gst_vpudec_chain: buf status 0x821 data 0

0:00:02.527148001 7627 0x2e700798 INFO vpudec vpudec.c:1512:gst_vpudec_chain: Got drop information!!

Re: RTSP gstreamer and vpudec

holgerweber — Fri, 24 Jan 2014 14:24:29 GMT

If I use this command and I switch of low latency encoding of my external hardware encoder.

gst-launch-0.10 --gst-debug=vpudec:5 rtspsrc location=rtsp://192.168.25.60/stream1.sdp ! rtpmp2tdepay ! mpegtsdemux ! h264parse ! vpudec ! mfw_v4lsink

It works... but with much to high latency! I have to use the low latency mode so has anyone an idea what I have to do?

>> Got drop information!!

How can I ignore that information or how can I solve that problem?

Re: RTSP gstreamer and vpudec

LeonardoSandova — Fri, 24 Jan 2014 18:09:01 GMT

do you observe the same issue with 'gst-launch playbin2 uri=rtsp://192.168.25.60/stream1.sdp'? Not sure if it works but lets try playbin2 to create the pipeline for you.

Re: RTSP gstreamer and vpudec

holgerweber — Mon, 27 Jan 2014 08:28:28 GMT

If I use playbin2, gstreamer uses the aiurdemux element.

No video appears and I get this messages (indipendent from low latency encode or normal encode):

Setting pipeline to PAUSED ...

Pipeline is live and does not need PREROLL ...

Setting pipeline to PLAYING ...

New clock: GstSystemClock

0:00:02.246670002 12472 0x17ea60 LOG aiurdemux aiurdemux.c:1836:gst_aiurdemux_handle_sink_event:<aiurdemux0> handling new segment from0

Aiur: 3.0.5
Core: BLN_MAD-MMLAYER_MPG2PARSER_04.04.04 build on Dec 11 2012 13:39:16
mime: video/mpeg, mpegversion=(int)[1,2]; video/mpegts, systemstream=true; video/x-cdxa
file: /usr/lib/imx-mm/parser/lib_mpg2_parser_arm11_elinux.so.3.1
Content Info:
Seekable : No
Size(byte): -1

0:00:02.278108661 12472 0x2e4da0 WARN aiurdemux aiurdemux.c:2335:aiurdemux_add_user_tags: API[getMetaData] failed, ret=-6

0:00:02.278418995 12472 0x2e4da0 WARN aiurdemux aiurdemux.c:2335:aiurdemux_add_user_tags: API[getMetaData] failed, ret=-6

0:00:02.278510328 12472 0x2e4da0 WARN aiurdemux aiurdemux.c:2335:aiurdemux_add_user_tags: API[getMetaData] failed, ret=-6

0:00:02.278571328 12472 0x2e4da0 WARN aiurdemux aiurdemux.c:2335:aiurdemux_add_user_tags: API[getMetaData] failed, ret=-6

0:00:02.278826328 12472 0x2e4da0 WARN aiurdemux aiurdemux.c:2335:aiurdemux_add_user_tags: API[getMetaData] failed, ret=-6

0:00:02.278893661 12472 0x2e4da0 WARN aiurdemux aiurdemux.c:2335:aiurdemux_add_user_tags: API[getMetaData] failed, ret=-6

0:00:02.278945661 12472 0x2e4da0 WARN aiurdemux aiurdemux.c:2335:aiurdemux_add_user_tags: API[getMetaData] failed, ret=-6

0:00:02.278995995 12472 0x2e4da0 WARN aiurdemux aiurdemux.c:2335:aiurdemux_add_user_tags: API[getMetaData] failed, ret=-6

0:00:02.279043661 12472 0x2e4da0 WARN aiurdemux aiurdemux.c:2335:aiurdemux_add_user_tags: API[getMetaData] failed, ret=-6

0:00:02.279093328 12472 0x2e4da0 WARN aiurdemux aiurdemux.c:2335:aiurdemux_add_user_tags: API[getMetaData] failed, ret=-6

0:00:02.279141328 12472 0x2e4da0 WARN aiurdemux aiurdemux.c:2335:aiurdemux_add_user_tags: API[getMetaData] failed, ret=-6

0:00:02.279188995 12472 0x2e4da0 WARN aiurdemux aiurdemux.c:2335:aiurdemux_add_user_tags: API[getMetaData] failed, ret=-6

0:00:02.279237328 12472 0x2e4da0 WARN aiurdemux aiurdemux.c:2335:aiurdemux_add_user_tags: API[getMetaData] failed, ret=-6

0:00:02.279284328 12472 0x2e4da0 WARN aiurdemux aiurdemux.c:2335:aiurdemux_add_user_tags: API[getMetaData] failed, ret=-6

0:00:02.279331661 12472 0x2e4da0 WARN aiurdemux aiurdemux.c:2335:aiurdemux_add_user_tags: API[getMetaData] failed, ret=-6

Mpeg2CreateParser:parser created successfully

Movie Info:
Seekable : No
Live : Yes
Duration : 0:00:00.000000000
ReadMode : File
AutoRetimestamp: 2000ms
programs : 1
Track : 1

0:00:02.279613328 12472 0x2e4da0 WARN aiurdemux aiurdemux.c:3195:aiurdemux_parse_streams: Warning: API[getLanguage] not implement!

0:00:02.279757661 12472 0x2e4da0 INFO aiurdemux aiurdemux.c:2744:aiurdemux_parse_video: Create video pad video_00112c

Track 00 [video_00112c] Enabled
ppid: 1, ppid 300
Duration: 0:00:00.000000000
Language:
Mime:
video/x-h264, parsed=(boolean)true, width=(int)1280, height=(int)720, framerate=
(fraction)30/1, codec_data=(buffer)00000127640028ad840d218e634e30851514420a934ac
899a81e805005bb011000007d00001d4c084000
[INFO] Product Info: i.MX6Q/D/S

vpudec versions :smileyhappy:
plugin: 3.0.5
wrapper: 1.0.28(VPUWRAPPER_ARM_LINUX Build on Dec 14 2012 12:23:57)
vpulib: 5.4.10
firmware: 2.1.8.34588
0:00:02.310173000 12472 0x2e4da0 LOG vpudec vpudec.c:392:vpudec_core_mem_alloc_dma_buffer: Call VPU_DecGetMem return 0x0

0:00:02.311043333 12472 0x2e4da0 ERROR aiurdemux aiurstreamcache.c:335:gst_aiur_stream_cache_seek: Unexpect backward seek addr 0, cachestart 815798, offset 200010

0:00:02.311145333 12472 0x2e4da0 INFO aiurdemux aiurstreamcache.c:360:gst_aiur_stream_cache_seek: stream cache try seek to 0

!!!! Pressing Strg+C !!!

Caught interrupt -- handling interrupt.

Interrupt: Stopping pipeline ...

Execution ended after 9027289665 ns.

Setting pipeline to PAUSED ...

Setting pipeline to READY ...

0:00:09.206715331 12472 0x17050 ERROR vpudec vpudec.c:558:vpudec_core_deinit: Func VPU_DecFlushAll failed!! with ret 3

0:00:09.209800998 12472 0x17050 LOG vpudec vpudec.c:373:vpudec_core_mem_free_dma_buffer: Call VPU_DecFreeMem return 0x0

0:00:09.210019331 12472 0x17050 INFO vpudec vpudec.c:577:vpudec_core_deinit: Stat:

in : 0

out : 0

show: 0

0:00:09.313042333 12472 0x2e4da0 WARN aiurdemux aiurdemux.c:1325:aiurdemux_send_stream_eos: Pad video: Send eos.

0:00:09.313309999 12472 0x2e4da0 LOG aiurdemux aiurdemux.c:4247:aiurdemux_push_task:<aiurdemux0> pausing task, reason unexpected

0:00:09.313476333 12472 0x2e4da0 LOG aiurdemux aiurdemux.c:4256:aiurdemux_push_task:<aiurdemux0> Sending EOS at end of segment

0:00:09.314449666 12472 0x17050 INFO aiurdemux aiurdemux.c:2007:aiurdemux_release_resource: stream video[0] interleave queue max size 0

Setting pipeline to NULL ...

[--->FINALIZE aiurdemux
Freeing pipeline ...

Re: RTSP gstreamer and vpudec

holgerweber — Tue, 28 Jan 2014 09:32:42 GMT

The problem was decoding of low latency streams (intra-refresh streams). I had to recompile the vpu wrapper lib as discussed here: Re: H.264 decoding using ffmpeg + using GPU for display acceleration

Playback is now running with this pipeline:

gst-launch --gst-debug=vpudec:0 udpsrc uri=udp://192.168.24.39:4444 ! mpegtsdemux ! h264parse ! vpudec low-latency=true framedrop=true ! mfw_isink axis-top=100 axis-left=100 disp-width=640 disp-height=360 sync=false

But the latency is much to high! It's about 1000 ms we want to have a latency of 200ms or better.

Any ideas how I can reduce the latency?

Re: RTSP gstreamer and vpudec

holgerweber — Tue, 28 Jan 2014 17:10:06 GMT

Some time later....

I read this thread https://community.freescale.com/message/358922#358922
which mentioned to change the vpudec.c file. It's also possible to change the value sDecOpenParam.reorderEnable to 0 in the vpu_wrapper.c file.

This reduced the delay of the whole system from about 1sec. to 200ms.

PATCH:

open vpu_wrapper.c
change pObj->iframesearch_allowed=1 to 0
change sDecOpenParam.reorderEnable=1 to 0

Re: RTSP gstreamer and vpudec

reinholdfrederi — Fri, 09 Jan 2015 10:06:32 GMT

Hi,

I would like to reduce latency time without change vpu_wrapper.c.

I have changed in vpudec this parameter: dis-reorder=true (equal to sDecOpenParam.reorderEnable=0).

Now i'm looking for pObj->iframesearch_allowed in vpudec parameters ?

Thanks for your help.