Playing low-latency H.264 video with i.MX6Q

[rebelalliance]

Hi,

Ok I am running into an issue which I haven’t been able to surmount and which I need the answers to in order to qualify this chip for our use:

1) Is this decoder able to play low-latency streams aka "intra-refresh"?  There is some confusion about this so let me clarify.  A low latency stream, as we call it, is one which has no "I" frame per se but only "P" frames.  The "I" Frame is actually embedded inside the "P" frame in slices.  This is not nonstandard but is uncommon because it is used mostly in broadcast mode.  We generate these low-latency streams so we need a decoder which is able to decode such streams. 

What I did was save the streams to a TS file and playback using playbin2.  This works and we have been able to qualify many different streams and observe their error resiliency.  However, the low-latency stream have not worked using playbin2 (they don’t play).  Here is their errored output:

root@freescale ~/test$ gst-launch-0.10 filesrc location=./1080p30_low_latency.ts typefind=true ! queue ! aiurdemux ! queue max-size-time=0 max-size-buffers=0 ! vpudec low-latency=true frame-plus=1 framedrop=false ! mfw_v4lsink

MFW_GST_V4LSINK_PLUGIN 3.0.5 build on Mar 26 2013 11:49:26.

Setting pipeline to PAUSED ...

[INFO]  Product Info: i.MX6Q/D/S

vpudec versions

        plugin: 3.0.5

        wrapper: 1.0.28(VPUWRAPPER_ARM_LINUX Build on Mar 26 2013 11:48:15)

        vpulib: 5.4.10

        firmware: 2.1.8.34588

Pipeline is PREROLLING ...

Aiur: 3.0.5

Core: BLN_MAD-MMLAYER_MPG2PARSER_04.04.04  build on Dec 11 2012 13:39:16

  mime: video/mpeg, mpegversion=(int)[1,2]; video/mpegts, systemstream=true; video/x-cdxa

  file: /usr/lib/imx-mm/parser/lib_mpg2_parser_arm11_elinux.so.3.1

Content Info:

        URI:

              file:///root/test/1080p30_low_latency.ts

        Idx File:

              /root/.aiur/.root.test.1080p30_low_latency.ts.aidx

        Seekable  : Yes

        Size(byte): 21366012

Mpeg2CreateParser:parser created successfully

Movie Info:

        Seekable  : Yes

        Live      : No

        Duration  : 0:00:30.800000000

        ReadMode  : File

        programs    : 1

        Track     : 1

Track 00 [video_001031] Enabled

        ppid: 1, ppid 49

        Duration: 0:00:30.800000000

        Language:

        Mime:

              video/x-h264, parsed=(boolean)true, width=(int)1920, height=(int)1088, framerate

              =(fraction)30/1, codec_data=(buffer)000001674d00299a6280f0044fcb80b5010101400000

              fa00003a983a1800cf900033e52ef2e3430019f200067ca5de5c2800

** (gst-launch-0.10:3380): CRITICAL **: gst_vpudec_chain: assertion `vpudec->context.handle' failed

Pipeline is PREROLLED ...

Setting pipeline to PLAYING ...

New clock: GstSystemClock

Got EOS from element "pipeline0".

Execution ended after 446333 ns.

Setting pipeline to PAUSED ...

Running time 0:00:00.001071000 render fps 0.000

Setting pipeline to READY ...

Setting pipeline to NULL ...

Total rendered:0

Freeing pipeline ...

[--->FINALIZE v4l_sink

[--->FINALIZE aiurdemux

Can someone verify if the decoder with the i.MX6Q able to decode these low-latency streams?  Or is the above definitive conclusion that it cannot do it?

PS: Attached is the low latency TS file which I'm attempting to play back.

PPS: Using a Sabre SDB Development Board with Linux.

Best Regards,

Umar

Original Attachment has been moved to: 1080p30_low_latency.ts.zip

[rebelalliance · Accepted solution]

Solution to this issue is here:

https://community.freescale.com/message/342913#342913

[juangutierrez]

I have received the analysis made over the clip attached from the MM experts

The conclusion is that this kind of clip is not support by our current design. At least one I-frame is needed at the start or seek point:

   There are two reasons based on preliminary analysis:

   (1) In our mpeg2 parser, it always find the key frame from the start point (or seek point)

   (2) In our decoder, we always enable vpu internal skipmode to skip non-I frame at start point(or seek point).

   The design is aimed to avoid unacceptable visual quality.

   And it is difficult to be changed, we have designed framework for most popular clips, but not several special clips.

   For the clip provided, it is too special, no any I frame type exist in the whole clip. As result, all frames will be dropped.

   Another way, even you want to decrease delay(low latency), you still can insert some I frames into clip.

[rebelalliance]

Gracias, senor.

[LeonardoSandova]

I have played the video and got the same error. Actually, enabling debug flags I see this error

0:00:07.087301333  2982	0x17050 ERROR	vpudec vpudec.c:558:vpudec_core_deinit: Func VPU_DecFlushAll failed!! with ret 3
0:00:07.089887667  2982	0x17050 LOG	vpudec vpudec.c:373:vpudec_core_mem_free_dma_buffer: Call VPU_DecFreeMem return 0x0
0:00:07.090034667  2982	0x17050 INFO	vpudec vpudec.c:577:vpudec_core_deinit: Stat:

So seems that this format is not supported but need to be confirmed.

[rebelalliance]

Can you confirm officially the i.MX decoder cannot play low latency (intra-refresh) streams?

[LeonardoSandova]

Hi, this issue is under investigation. Just one thing: could you please try with other(s) low-latency media files?

Leo

[rebelalliance]

I have tried it with other low latency media.  Here's another file attached.

[rebelalliance]

Thank you for your response.

How can this be confirmed so the matter can be put to rest?

[LeonardoSandova]

Correct. All posts remain at rest some time, and are escalated to Freescale specialists if these have not been responded during this period.

Leo