i.MX6Q: How to play mp4 using VPU and qtdemux

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

i.MX6Q: How to play mp4 using VPU and qtdemux

Jump to solution
4,109 Views
daisuketsuchiya
Contributor II

Hello,


I'm trying to play mp4 on GStreamer and VPU of i.MX6Q.  And I want to use qtdemux and isink, but an error occurs.

How can I play using qtdemux and isink?

My operation is below:
gst-launch filesrc location=/mnt/floppy/1sec_10frame.mp4 ! 'video/quicktime' ! qtdemux ! queue ! vpudec ! mfw_isink

It's no problem when using aiurdemux.
gst-launch filesrc location=/mnt/floppy/1sec_10frame.mp4 ! 'video/quicktime' ! aiurdemux ! queue ! vpudec ! mfw_isink

filesrc:    0.10.35
aiurdemux:  3.0.5
vpudec:     3.0.5
qtdemux:    0.10.30

The error occurs in vpu_DecGetInitialInfo() of vpu_lib.c.

=====
{
            :

    BitIssueCommand(pCodecInst, SEQ_INIT);
    while (VpuReadReg(BIT_BUSY_FLAG)) ;


    val = VpuReadReg(RET_DEC_SEQ_SUCCESS);

    if (cpu_is_mx6x()) {
#ifdef MEM_PROTECT
        if (val & (1 << 31)) {
            :
        }
#endif
->      if (pDecInfo->openParam.bitstreamMode && (val & (1 << 4))) {
->          UnlockVpu(vpu_semap);

->          return RETCODE_FAILURE;
        }
    }

=====

Best Regards,

Daisuke

Labels (1)
0 Kudos
1 Solution
2,306 Views
kchiba
Contributor I
Hello
I'm a Tsuchiya's subordinate.
I've read all of your comments and advice.
I've not tried FFMPEG demux yet however, it can be one of the solutions.
Thank you for all of your advice and cooperation .

Today, I  could play a specific mp4 file by modifying the source code of qtdemux.
I'd like to share the way and get comment about it.
1.Solve the differences of start code prefix of NAL
Thanks to Katagiri-san's advice, I noticed this difference.
In the case of aiurdemux, the first 4 byte is "00 00 00 01".
But in the case of qtdemux, it's the length of the each NAL data.
I modified qtdemux and solve this difference.

2.Solve the difference of "codec_data"
I noticed this difference by putting "-v" to the command line like below
gst-launch -v filesrc location=/mnt/floppy/1sec_10frame.mp4 ! 'video/quicktime' ! qtdemux ! queue ! vpudec ! mfw_isink
I saw the difference of the value of "codec_data".

/GstPipeline:pipeline0/GstVpuDec:vpudec0.GstPad:sink: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)2, profile=(string)baseline,

codec_data=(buffer)0142e00dffe100162742e00da918283f600d418041adb7a02f01e97bdf0101000428ce0988,

width=(int)320, height=(int)240, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1

In the case of aiurdemux

      codec_data=(buffer)000000012742e00da918283f600d418041adb7a02f01e97bdf01 0000000128ce0988

In the case of qtdemux

      codec_data=(buffer)0142e00dffe100162742e00da918283f600d418041adb7a02f01e97bdf0101000428ce0988

I don't know why this happens, so I just modified qtdemux to make it output the "codec_data" of aiurdemux above regardless of the read mp4 file on trial.

The two modification above solved the problem.

I mean I could play a specific mp4 file using qtdemux + vpudec.(non-aiurdemux)

(I cannot play other mp4 file because qtdemux decide "codec_data" statically on trial)

I guess "codec_data" is avcC Data which is in mp4 file.

In the case of qtdemux, "codec_data" equalls avcC, but in the case of aiurdemux, it's little bit modified.

According to the aiurdemux element source code, "codec_data" is decided by FslParserGetDecSpecificInfo() which is fsl parser API.

Could anyone advice me the way to convert "codec_data" of qtdemux to aiurdemux so that qtdemux can decide "codec_data" same as the aiurdemux and play any mp4 files?

Kuniaki

View solution in original post

0 Kudos
11 Replies
2,306 Views
LeonardoSandova
Specialist I

As recommended by DaianeAngolini, have you tried any FFMPEG demuxer?

0 Kudos
2,307 Views
kchiba
Contributor I
Hello
I'm a Tsuchiya's subordinate.
I've read all of your comments and advice.
I've not tried FFMPEG demux yet however, it can be one of the solutions.
Thank you for all of your advice and cooperation .

Today, I  could play a specific mp4 file by modifying the source code of qtdemux.
I'd like to share the way and get comment about it.
1.Solve the differences of start code prefix of NAL
Thanks to Katagiri-san's advice, I noticed this difference.
In the case of aiurdemux, the first 4 byte is "00 00 00 01".
But in the case of qtdemux, it's the length of the each NAL data.
I modified qtdemux and solve this difference.

2.Solve the difference of "codec_data"
I noticed this difference by putting "-v" to the command line like below
gst-launch -v filesrc location=/mnt/floppy/1sec_10frame.mp4 ! 'video/quicktime' ! qtdemux ! queue ! vpudec ! mfw_isink
I saw the difference of the value of "codec_data".

/GstPipeline:pipeline0/GstVpuDec:vpudec0.GstPad:sink: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)2, profile=(string)baseline,

codec_data=(buffer)0142e00dffe100162742e00da918283f600d418041adb7a02f01e97bdf0101000428ce0988,

width=(int)320, height=(int)240, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1

In the case of aiurdemux

      codec_data=(buffer)000000012742e00da918283f600d418041adb7a02f01e97bdf01 0000000128ce0988

In the case of qtdemux

      codec_data=(buffer)0142e00dffe100162742e00da918283f600d418041adb7a02f01e97bdf0101000428ce0988

I don't know why this happens, so I just modified qtdemux to make it output the "codec_data" of aiurdemux above regardless of the read mp4 file on trial.

The two modification above solved the problem.

I mean I could play a specific mp4 file using qtdemux + vpudec.(non-aiurdemux)

(I cannot play other mp4 file because qtdemux decide "codec_data" statically on trial)

I guess "codec_data" is avcC Data which is in mp4 file.

In the case of qtdemux, "codec_data" equalls avcC, but in the case of aiurdemux, it's little bit modified.

According to the aiurdemux element source code, "codec_data" is decided by FslParserGetDecSpecificInfo() which is fsl parser API.

Could anyone advice me the way to convert "codec_data" of qtdemux to aiurdemux so that qtdemux can decide "codec_data" same as the aiurdemux and play any mp4 files?

Kuniaki

0 Kudos
2,306 Views
LeonardoSandova
Specialist I

Please put all your codes changes into a patch, so it can be easily applied by other (of course, once you solve the codec_data issue). BTW, How codec_data is extracted on a ffmpeg demuxer? that would be interesting to check.

Leo

0 Kudos
2,306 Views
LeonardoSandova
Specialist I

shigenobukatagiri-b39676, did you find the reason of this problem?

Leo

0 Kudos
2,306 Views
shigenobukatagi
NXP Employee
NXP Employee

Hi Tuchiya-san, Leo,

According to my test, it seems qtdemux doesn’t generate complete H.264 ES.

When I use aiurdemux, there are many “00 00 01” start code prefix of NAL in the demuxed H.264 ES file, VPU and VLC (works on PC) can play back.

However in case of qtdemux, demuxed data has few “00 00 01” start code prefix words, neither VPU nor VLC play back

You may see same result by comparing files generated by following command lines.

$ gst-launch -v filesrc location=<file.mp4> ! 'video/quicktime' ! aiurdemux name=demux ! filesink location=raw_aiur.h264

$ gst-launch -v filesrc location=<file.mp4> ! 'video/quicktime' ! qtdemux name=demux ! filesink location=raw_qt.h264

I found a similar discussion as follows. qtdemux may not be the solution to demux H.264 container.

http://gstreamer-devel.966125.n4.nabble.com/QT-Demux-with-strange-ouput-ES-data-td970679.html

Leo,

Could you please give us your comments?

Best Regards,

Shigenobu

0 Kudos
2,306 Views
LeonardoSandova
Specialist I

Hi,

Let me investigate further but for the moment, lets force the playback to use another demuxer. Move the plugin/element /usr/lib/gstreamer-0.10/libmfw_gst_aiur_demux.so to your $HOME, then run the following pipeline with playbin2:

gst-launch playbi2 uri=file://<media full path> --gst-debug=GST_ELEMENT_FACTORY:3

Check the log.Which demuxer did playbin2 use? Customer may use that one instead of qtdemux.

Leo

0 Kudos
2,306 Views
shigenobukatagi
NXP Employee
NXP Employee

Hi Leo,

Thank you for your help.

I tried your test which removing aiurdemux,

playbin2 looks for aiurdemux first.

  -> if aiurdemux is not found, qtdemux is the next target.  -> playbin2_1.log

      -> if qtdemux is not found either, playbin2 outputs error.  -> playbin2_2.log

I've confirmed it by removing libgstisomp4.so.

Best Regards,

Shigenobu

0 Kudos
2,306 Views
LeonardoSandova
Specialist I

jackmao / DaianeAngolini, I need help on this one :smileyhappy:

Leo

0 Kudos
2,306 Views
daiane_angolini
NXP Employee
NXP Employee

Sorry, what´s the problem here?

I see there is a discussion regarding using qtdemux or airdemux, but, I lost the goal of this discussion.

LeonardoSandovalGonzalez, What´s the goal?

0 Kudos
2,306 Views
LeonardoSandova
Specialist I

Hi Daiane.

Customer must use a non-aiurdemux element, so qtdemux may be an option. However, when using qtdemux, they got errors and playback is not possible. For some reason, the vpudec does not like what is coming from qtdemux, some missing headers as mentioned in previous post:

Re: i.MX6Q: How to play mp4 using VPU and qtdemux

Leo

0 Kudos
2,306 Views
daiane_angolini
NXP Employee
NXP Employee

oh, I got it.

So aiurdemux is not a option. The question is which other mainline demuxer is suitable for the customer´s usecase.

I only know ffmpeg plugins for gstreamer, but I´m not sure how easy would be for including it on the rootfs and if it would be an option.

0 Kudos