gstreamer vpuenc H.264 encoder (codec=6) output not compressed

[normancheung]

I use gstremer to test vpu encoding and found that for a 10 second video, the H.264 encoder is 20+X bigger than mpeg video..

On imx6 solo with MIPI camera, Yocto Linux 3.0.50 4.0.0

This is my pipeline:

gst-launch mfw_v4lsrc capture-mode=4 ! queue ! vpuenc codec=6 ! matroskamux ! filesink location=test_h264.mkv

For codec=0 (mpeg) 10 second video is 2MB, and codec=6 (H.264) it is 48MB.

It seems that the vpuenc element is not encoding the stream. 

I want to get H.264 video, and appreciate any idea to fix this.

Thank you

Norman

[XXiao1z]

any update on the h.264 vpnenc bug? I need do the same here.

[daiane_angolini]

The QP range for mpeg4(1-31) and h.264(0-51) are different.

As a result, there will be big gap between output size for the same default qp.

So, we suggest user encode clip with appointed bitrate instead of default qp.

>gst-launch mfw_v4lsrc num-buffers=100 capture-mode=4 ! queue ! vpuenc  codec=6 bitrate=4000000 ! matroskamux ! filesink location=test_h264_100.mkv

Please, let me know your results.

[jparrish88]

Hi Daiane,

Do you and others feel that this issue has been fixed?

If so, does that mean the the fix is a bitrate adjustment only or is there something else that may not have been mentioned?

If not, what is the status on a patch or fix?

[daiane_angolini]

Sorry for the delay jparrish88, I should have missed your update notification...

I don´t think the issue is "fixed", because so far I haven´t completely understood this as a bug.

I don´t have one technical argument to disagree with the message I got internally shown in Re: gstreamer vpuenc H.264 encoder (codec=6) output not compressed

And I really don´t understand yet the point in here Re: gstreamer vpuenc H.264 encoder (codec=6) output not compressed

Look, it´s not about believes. It´s about technical arguments. Today I don´t have any. In case you do understand this issue deeper, please, let me know and I will work internally to get more help.

[normancheung]

Daiane,

See my post in this thread on Sep 16, 2013 2:16 PM.   I have tried fixed bitrate and it is good.  But I was hoping to use auto adjust variable bitrate.

I have done more experiments, and found that this only happens which the vpuenc is set to Auto bitrate (i.e. use the default =0).  The frame size for fix bitrate is good, I tried 1Mbps and 1.5Mbps all give reasonable;e frame size.

This is not the news I would like to hear.   If the issue is in the QP, than it is likely to be in the VPU and it is not likely to be patchable. 

Thanks anyway.

Norman

[daiane_angolini]

Sorry, Norman

I did not understand your point. Could you, please, elaborate?

[daiane_angolini]

Using the following command lines:

number of buffers before EOF = 100

gst-launch mfw_v4lsrc num-buffers=100 capture-mode=4 ! queue ! vpuenc codec=6 ! matroskamux ! filesink location=test_h264_100.mkv

gst-launch mfw_v4lsrc num-buffers=100 capture-mode=4 ! queue ! vpuenc codec=0 ! matroskamux ! filesink location=test_mpeg4_100.mkv

number of buffers before EOF = 200

gst-launch mfw_v4lsrc num-buffers=200 capture-mode=4 ! queue ! vpuenc codec=6 ! matroskamux ! filesink location=test_h264_200.mkv

gst-launch mfw_v4lsrc num-buffers=200 capture-mode=4 ! queue ! vpuenc codec=0 ! matroskamux ! filesink location=test_mpeg4_200.mkv

number of buffers before EOF = 300

gst-launch mfw_v4lsrc num-buffers=300 capture-mode=4 ! queue ! vpuenc codec=6 ! matroskamux ! filesink location=test_h264_300.mkv

gst-launch mfw_v4lsrc num-buffers=300 capture-mode=4 ! queue ! vpuenc codec=0 ! matroskamux ! filesink location=test_mpeg4_300.mkv

I got the following results:

-rwxr-xr-x    1 root     root       19.4M Sep 16 20:25 test_h264_100.mkv

-rwxr-xr-x    1 root     root       38.8M Sep 16 20:24 test_h264_200.mkv

-rwxr-xr-x    1 root     root       58.1M Sep 16 20:25 test_h264_300.mkv

-rwxr-xr-x    1 root     root      531.1K Sep 16 20:25 test_mpeg4_100.mkv

-rwxr-xr-x    1 root     root        1.0M Sep 16 20:25 test_mpeg4_200.mkv

-rwxr-xr-x    1 root     root        1.5M Sep 16 20:26 test_mpeg4_300.mkv

I'm using this env:

Linux imx6qsabresd 3.0.35-4.1.0+yocto+gbdde708 #1 SMP PREEMPT Mon Sep 16 16:20:23 BRT 2013 armv7l GNU/Linux

U-Boot 2013.10-rc2 (Sep 16 2013 - 16:46:38)

CPU:   Freescale i.MX6Q rev1.2 at 792 MHz

Board: MX6-SabreSD

DRAM:  1 GiB

So, it's not an integration error, it's something related with MM plugins/encoders

karinavalencia, we need to escalate this bug to get fixed on next MM release. (or someone to explain why this is not a bug)

Deactivated user, include plain text is a pain!!!!!!!!!!!!!!  :smileysad::smileycry:

[karina_valencia]

DaianeAngolini please create a CT for follow up.

[daiane_angolini]

Created CT#43609703

[daiane_angolini]

I don´t understand H264 and MPEG4 encoding enough to know if the result file would be "comparable". I really think any conclusion on this issue would be precipitated. I was not able to find a good comparison table for H264 and MPEG 4 (http://scholar.google.com/scholar?q=h+264+mpeg4&btnG=&hl=en&as_sdt=0%2C44)

What I think should be our focus:

* What do you need? What are your constrains?

* Are you able to reproduce the same difference using your PC? Try gstreamer on your PC and generate one "default" H264 and one "default" MPEG4 and compare the file size.

If you like to understand how gstreamer deal with default/possible configuration, you may contact gstreamer mail list.

[normancheung]

Thanks Daiane for looking into this issue.

Our customer wants H264 instead Mpeg4, that limits our choices.  I want to get the plugin vpuenc to work.

I am not talking about small differences between mpeg4 and h264; it is an order of magnitude difference 10X bigger file size.  On my cell phone and on my PC  I got very similar file sizes using H264 and Mpeg4. On paper, H264 supposed to be more efficient than Mpeg4 and hence smaller than Mpeg4. 

The plugin in question is the vpuenc, it  comes from FSL, so I don't think gstreamer forum would help much.  Perhaps Freescale's developer of that plugin can help.  Dumping the frames size seems to show that the VPU is not producing P frames -- only I frames.  I looked at all the available options in this plugin, there is no setting to control the production of P frames.  I think FSL's support/developer of this plugin and/or the support of  MM codec API would be able to help.

Meanwhile I am trying to compare it with a software only plugin (instead of encoding by VPU).  I added gstreamer-plugin-x264enc (an ugly plugin)  in my local.conf for the Yocto build.  It failed in making rootfs.  Perhaps cleanall does not work.  I will delete the build tree and rebuild.  when I get some results I will post it here.

It will be of great help if you could give me some ideas what are something that I can try to get H264 encoding with VPU .

Thanks,

Norman

[daiane_angolini]

I saw you´re using imx6S, and I suppose you´re using meta-fsl-arm. Dylan or master?

I will use your command line on my imx6Q board using master, if I could reproduce the issue, it´s a vpuenc error. Otherwise, it´s some integration mistake from meta-fsl-arm (VPU firmaware may be different for imx6S, for example, and we´ve been using the same from imx6q)

I´m going to start a bitbake for testing this now, and probably I´m able to try tomorrow. If it´s a vpuenc error, here is the best place to handle. If it´s an integration mistake/error, the better place to deal with the issue is meta-freescale (https://lists.yoctoproject.org/listinfo/meta-freescale)

Tomorrrow I come back to this topic and update it.

[normancheung]

Daiane,

I am using the master branch.  My board is wandboard-solo.

I have done more experiments, and found that this only happens which the vpuenc is set to Auto bitrate (i.e. use the default =0).  The frame size for fix bitrate is good, I tried 1Mbps and 1.5Mbps all give reasonable;e frame size.

thanks,

Norman

[LeonardoSandova]

I believe, the capture-mode value is the same value in both pipelines, right?

Leo

[normancheung]

Yes. tests with both encoding I used capture-mode=4 720P 30FPS recording.

I was wondering if that might hav something to do with VBR Versus CBR.  But then it cannot be out by so much.  The only think I could think of is the IPU code is doing a no-op -- just a guess.

Thanks,

Norman

[LeonardoSandova]

DaianeAngolini, any reason of this big delta between codecs?

[normancheung]

Daiane,

I have uploaded the 2 media file here: Online Backup for PC, Mac and iPhone | IDrive

Thanks,

Norman

I have dumped the size of frames and comparing the H264 file with the mpeg4 file.

Below is the fist portion of the dump.  And you can see the pattern.  It looks to me there are 2 issues influencing the size, that is not quite right:

1. It seems that H264 is encoding all I frames;  whereas Mpeg4 encodes one I frame ~ 0.5 second, 2 - 3 I frames per second.

2. The size of the frame is (assuming all are I frames) about 3X the size of Mpeg4.  That is better than 10X file size, but still big.

Thanks for your help,

Norman

   frame #     H264_size         mpeg4_size      

            1        121308        34262

            2        110347         5477

            3        103515         8473

            4        103947         5564

            5        102960        11489

            6        103974         7129

            7        106999         5777

            8        105517         7197

            9        102019         6297

           10        103045         6298

           11        102969         5430

           12        106440         4897

           13        102194         5778

           14        103236         5440

           15        102172         6614

           16        119551        33688

           17        103514         5302

           18        104108         5761

           19        103311         4841

           20        103419         5425

           21        101451         4453

           22        102454         5702

           23        102744         4421

           24        101899         4656

           25        104769         5409

           26        103984         5228

           27        106746         4611

           28        103861         4741

           29        102642         3996

           30        101762         5500

           31        119317        34082

           32        104736         5573

           33        103867         3723

           34        104681         4699

           35        104695         4004

           36        102445         4142

           37        102752         3755

           38        102544         3603

           39        102008         3095

           40        101631         2860

           41        101053         2353

           42        103052         2415

           43        102501         2257

           44        102067         2592

           45        103514         2515

           46        118937        33823

           47        102873         3388

           48        101201         3939

[normancheung]

While waiting for help from Freescale,  I wonder for folks in the forum  what is the experience of using the IPU encoder to encode H264 video?  -- imx6-solo  from MIPI camera in 720P 30 FPS.   I am using the wandboard with OV5640 camera.

Which toolkit did you use? Android or Linux? gstreamer setting?

Any idea for me to try will be helpful.

Thanks,

Norman

[LeonardoSandova]

Hi Norman,

I liked the comparison you did! for your latest post, VPU is the one doing the codec part (IPU is for display, VPU for MM codecs). For Linux, gstreamer is the framework to be used, unless you want to directly use the VPU API.

Leo

[normancheung]

Sorry I meant VPU.  I  was using vpuenc codec=6.  in gstreamer.

I would like to find out from the community is this is a bug in the gstreamer element, or it is a problem with the firmware (if firmware is involved).    I am hoping to see where to start to tackle this issue.  Any pointers to the sample programs to encode directly using VPU api will help.   Knowing others experiences with iMX6 solo in video capture will help me.

Thanks,

Norman