i.MX6Q: How to play mp4 using VPU and qtdemux

[daisuketsuchiya]

Hello,

I'm trying to play mp4 on GStreamer and VPU of i.MX6Q.  And I want to use qtdemux and isink, but an error occurs.

How can I play using qtdemux and isink?

My operation is below:
gst-launch filesrc location=/mnt/floppy/1sec_10frame.mp4 ! 'video/quicktime' ! qtdemux ! queue ! vpudec ! mfw_isink

It's no problem when using aiurdemux.
gst-launch filesrc location=/mnt/floppy/1sec_10frame.mp4 ! 'video/quicktime' ! aiurdemux ! queue ! vpudec ! mfw_isink

filesrc:    0.10.35
aiurdemux:  3.0.5
vpudec:     3.0.5
qtdemux:    0.10.30

The error occurs in vpu_DecGetInitialInfo() of vpu_lib.c.

=====
{
            :

    BitIssueCommand(pCodecInst, SEQ_INIT);
    while (VpuReadReg(BIT_BUSY_FLAG)) ;

    val = VpuReadReg(RET_DEC_SEQ_SUCCESS);

    if (cpu_is_mx6x()) {
#ifdef MEM_PROTECT
        if (val & (1 << 31)) {
            :
        }
#endif
->      if (pDecInfo->openParam.bitstreamMode && (val & (1 << 4))) {
->          UnlockVpu(vpu_semap);

->          return RETCODE_FAILURE;
        }
    }

=====

Best Regards,

Daisuke

[LeonardoSandova]

As recommended by DaianeAngolini, have you tried any FFMPEG demuxer?

[kchiba · Accepted solution]

Hello

I'm a Tsuchiya's subordinate.

I've read all of your comments and advice.

I've not tried FFMPEG demux yet however, it can be one of the solutions.

Thank you for all of your advice and cooperation .

Today, I  could play a specific mp4 file by modifying the source code of qtdemux.

I'd like to share the way and get comment about it.

1.Solve the differences of start code prefix of NAL

Thanks to Katagiri-san's advice, I noticed this difference.

In the case of aiurdemux, the first 4 byte is "00 00 00 01".

But in the case of qtdemux, it's the length of the each NAL data.

I modified qtdemux and solve this difference.

2.Solve the difference of "codec_data"

I noticed this difference by putting "-v" to the command line like below

gst-launch -v filesrc location=/mnt/floppy/1sec_10frame.mp4 ! 'video/quicktime' ! qtdemux ! queue ! vpudec ! mfw_isink

I saw the difference of the value of "codec_data".

/GstPipeline:pipeline0/GstVpuDec:vpudec0.GstPad:sink: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)2, profile=(string)baseline,

codec_data=(buffer)0142e00dffe100162742e00da918283f600d418041adb7a02f01e97bdf0101000428ce0988,

width=(int)320, height=(int)240, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1

In the case of aiurdemux

      codec_data=(buffer)000000012742e00da918283f600d418041adb7a02f01e97bdf01 0000000128ce0988

In the case of qtdemux

      codec_data=(buffer)0142e00dffe100162742e00da918283f600d418041adb7a02f01e97bdf0101000428ce0988

I don't know why this happens, so I just modified qtdemux to make it output the "codec_data" of aiurdemux above regardless of the read mp4 file on trial.

The two modification above solved the problem.

I mean I could play a specific mp4 file using qtdemux + vpudec.(non-aiurdemux)

(I cannot play other mp4 file because qtdemux decide "codec_data" statically on trial)

I guess "codec_data" is avcC Data which is in mp4 file.

In the case of qtdemux, "codec_data" equalls avcC, but in the case of aiurdemux, it's little bit modified.

According to the aiurdemux element source code, "codec_data" is decided by FslParserGetDecSpecificInfo() which is fsl parser API.

Could anyone advice me the way to convert "codec_data" of qtdemux to aiurdemux so that qtdemux can decide "codec_data" same as the aiurdemux and play any mp4 files?

Kuniaki

[LeonardoSandova]

Please put all your codes changes into a patch, so it can be easily applied by other (of course, once you solve the codec_data issue). BTW, How codec_data is extracted on a ffmpeg demuxer? that would be interesting to check.

Leo

[LeonardoSandova]

shigenobukatagiri-b39676, did you find the reason of this problem?

Leo

[shigenobukatagi]

Hi Tuchiya-san, Leo,

According to my test, it seems qtdemux doesn’t generate complete H.264 ES.

When I use aiurdemux, there are many “00 00 01” start code prefix of NAL in the demuxed H.264 ES file, VPU and VLC (works on PC) can play back.

However in case of qtdemux, demuxed data has few “00 00 01” start code prefix words, neither VPU nor VLC play back

You may see same result by comparing files generated by following command lines.

$ gst-launch -v filesrc location=<file.mp4> ! 'video/quicktime' ! aiurdemux name=demux ! filesink location=raw_aiur.h264

$ gst-launch -v filesrc location=<file.mp4> ! 'video/quicktime' ! qtdemux name=demux ! filesink location=raw_qt.h264

I found a similar discussion as follows. qtdemux may not be the solution to demux H.264 container.

http://gstreamer-devel.966125.n4.nabble.com/QT-Demux-with-strange-ouput-ES-data-td970679.html

Leo,

Could you please give us your comments?

Best Regards,

Shigenobu

[LeonardoSandova]

Hi,

Let me investigate further but for the moment, lets force the playback to use another demuxer. Move the plugin/element /usr/lib/gstreamer-0.10/libmfw_gst_aiur_demux.so to your $HOME, then run the following pipeline with playbin2:

gst-launch playbi2 uri=file://<media full path> --gst-debug=GST_ELEMENT_FACTORY:3

Check the log.Which demuxer did playbin2 use? Customer may use that one instead of qtdemux.

Leo

[shigenobukatagi]

Hi Leo,

Thank you for your help.

I tried your test which removing aiurdemux,

playbin2 looks for aiurdemux first.

  -> if aiurdemux is not found, qtdemux is the next target.  -> playbin2_1.log

      -> if qtdemux is not found either, playbin2 outputs error.  -> playbin2_2.log

I've confirmed it by removing libgstisomp4.so.

Best Regards,

Shigenobu

[LeonardoSandova]

jackmao / DaianeAngolini, I need help on this one :smileyhappy:

Leo

[daiane_angolini]

Sorry, what´s the problem here?

I see there is a discussion regarding using qtdemux or airdemux, but, I lost the goal of this discussion.

LeonardoSandovalGonzalez, What´s the goal?

[LeonardoSandova]

Hi Daiane.

Customer must use a non-aiurdemux element, so qtdemux may be an option. However, when using qtdemux, they got errors and playback is not possible. For some reason, the vpudec does not like what is coming from qtdemux, some missing headers as mentioned in previous post:

Re: i.MX6Q: How to play mp4 using VPU and qtdemux

Leo

[daiane_angolini]

oh, I got it.

So aiurdemux is not a option. The question is which other mainline demuxer is suitable for the customer´s usecase.

I only know ffmpeg plugins for gstreamer, but I´m not sure how easy would be for including it on the rootfs and if it would be an option.