i.MX ProcessorsのトピックRe: gstreamer vpuenc H.264 encoder (codec=6) output not compressed

gstreamer vpuenc H.264 encoder (codec=6) output not compressed

normancheung — Fri, 06 Sep 2013 23:50:04 GMT

I use gstremer to test vpu encoding and found that for a 10 second video, the H.264 encoder is 20+X bigger than mpeg video..

On imx6 solo with MIPI camera, Yocto Linux 3.0.50 4.0.0

This is my pipeline:

gst-launch mfw_v4lsrc capture-mode=4 ! queue ! vpuenc codec=6 ! matroskamux ! filesink location=test_h264.mkv

For codec=0 (mpeg) 10 second video is 2MB, and codec=6 (H.264) it is 48MB.

It seems that the vpuenc element is not encoding the stream.

I want to get H.264 video, and appreciate any idea to fix this.

Thank you

Norman

Re: gstreamer vpuenc H.264 encoder (codec=6) output not compressed

LeonardoSandova — Sun, 08 Sep 2013 02:24:28 GMT

I believe, the capture-mode value is the same value in both pipelines, right?

Leo

Re: gstreamer vpuenc H.264 encoder (codec=6) output not compressed

normancheung — Mon, 09 Sep 2013 19:28:56 GMT

Yes. tests with both encoding I used capture-mode=4 720P 30FPS recording.

I was wondering if that might hav something to do with VBR Versus CBR. But then it cannot be out by so much. The only think I could think of is the IPU code is doing a no-op -- just a guess.

Thanks,

Norman

Re: gstreamer vpuenc H.264 encoder (codec=6) output not compressed

LeonardoSandova — Mon, 09 Sep 2013 19:39:17 GMT

DaianeAngolini, any reason of this big delta between codecs?

Re: gstreamer vpuenc H.264 encoder (codec=6) output not compressed

normancheung — Tue, 10 Sep 2013 15:34:14 GMT

Daiane,

I have uploaded the 2 media file here: Online Backup for PC, Mac and iPhone | IDrive

Thanks,

Norman

I have dumped the size of frames and comparing the H264 file with the mpeg4 file.

Below is the fist portion of the dump. And you can see the pattern. It looks to me there are 2 issues influencing the size, that is not quite right:

1. It seems that H264 is encoding all I frames; whereas Mpeg4 encodes one I frame ~ 0.5 second, 2 - 3 I frames per second.

2. The size of the frame is (assuming all are I frames) about 3X the size of Mpeg4. That is better than 10X file size, but still big.

Thanks for your help,

Norman

frame # H264_size mpeg4_size

1 121308 34262

2 110347 5477

3 103515 8473

4 103947 5564

5 102960 11489

6 103974 7129

7 106999 5777

8 105517 7197

9 102019 6297

10 103045 6298

11 102969 5430

12 106440 4897

13 102194 5778

14 103236 5440

15 102172 6614

16 119551 33688

17 103514 5302

18 104108 5761

19 103311 4841

20 103419 5425

21 101451 4453

22 102454 5702

23 102744 4421

24 101899 4656

25 104769 5409

26 103984 5228

27 106746 4611

28 103861 4741

29 102642 3996

30 101762 5500

31 119317 34082

32 104736 5573

33 103867 3723

34 104681 4699

35 104695 4004

36 102445 4142

37 102752 3755

38 102544 3603

39 102008 3095

40 101631 2860

41 101053 2353

42 103052 2415

43 102501 2257

44 102067 2592

45 103514 2515

46 118937 33823

47 102873 3388

48 101201 3939

Re: gstreamer vpuenc H.264 encoder (codec=6) output not compressed

normancheung — Thu, 12 Sep 2013 17:02:38 GMT

While waiting for help from Freescale, I wonder for folks in the forum what is the experience of using the IPU encoder to encode H264 video? -- imx6-solo from MIPI camera in 720P 30 FPS. I am using the wandboard with OV5640 camera.

Which toolkit did you use? Android or Linux? gstreamer setting?

Any idea for me to try will be helpful.

Thanks,

Norman

Re: gstreamer vpuenc H.264 encoder (codec=6) output not compressed

LeonardoSandova — Thu, 12 Sep 2013 17:08:35 GMT

Hi Norman,

I liked the comparison you did! for your latest post, VPU is the one doing the codec part (IPU is for display, VPU for MM codecs). For Linux, gstreamer is the framework to be used, unless you want to directly use the VPU API.

Leo

Re: gstreamer vpuenc H.264 encoder (codec=6) output not compressed

normancheung — Thu, 12 Sep 2013 17:55:38 GMT

Sorry I meant VPU. I was using vpuenc codec=6. in gstreamer.

I would like to find out from the community is this is a bug in the gstreamer element, or it is a problem with the firmware (if firmware is involved). I am hoping to see where to start to tackle this issue. Any pointers to the sample programs to encode directly using VPU api will help. Knowing others experiences with iMX6 solo in video capture will help me.

Thanks,

Norman

Re: gstreamer vpuenc H.264 encoder (codec=6) output not compressed

daiane_angolini — Fri, 13 Sep 2013 16:55:33 GMT

I don´t understand H264 and MPEG4 encoding enough to know if the result file would be "comparable". I really think any conclusion on this issue would be precipitated. I was not able to find a good comparison table for H264 and MPEG 4 (http://scholar.google.com/scholar?q=h+264+mpeg4&btnG=&hl=en&as_sdt=0%2C44)

What I think should be our focus:

* What do you need? What are your constrains?

* Are you able to reproduce the same difference using your PC? Try gstreamer on your PC and generate one "default" H264 and one "default" MPEG4 and compare the file size.

If you like to understand how gstreamer deal with default/possible configuration, you may contact gstreamer mail list.

Re: gstreamer vpuenc H.264 encoder (codec=6) output not compressed

normancheung — Fri, 13 Sep 2013 19:50:29 GMT

Thanks Daiane for looking into this issue.

Our customer wants H264 instead Mpeg4, that limits our choices. I want to get the plugin vpuenc to work.

I am not talking about small differences between mpeg4 and h264; it is an order of magnitude difference 10X bigger file size. On my cell phone and on my PC I got very similar file sizes using H264 and Mpeg4. On paper, H264 supposed to be more efficient than Mpeg4 and hence smaller than Mpeg4.

The plugin in question is the vpuenc, it comes from FSL, so I don't think gstreamer forum would help much. Perhaps Freescale's developer of that plugin can help. Dumping the frames size seems to show that the VPU is not producing P frames -- only I frames. I looked at all the available options in this plugin, there is no setting to control the production of P frames. I think FSL's support/developer of this plugin and/or the support of MM codec API would be able to help.

Meanwhile I am trying to compare it with a software only plugin (instead of encoding by VPU). I added gstreamer-plugin-x264enc (an ugly plugin) in my local.conf for the Yocto build. It failed in making rootfs. Perhaps cleanall does not work. I will delete the build tree and rebuild. when I get some results I will post it here.

It will be of great help if you could give me some ideas what are something that I can try to get H264 encoding with VPU .

Thanks,

Norman

Re: gstreamer vpuenc H.264 encoder (codec=6) output not compressed

daiane_angolini — Mon, 16 Sep 2013 19:10:09 GMT

I saw you´re using imx6S, and I suppose you´re using meta-fsl-arm. Dylan or master?

I will use your command line on my imx6Q board using master, if I could reproduce the issue, it´s a vpuenc error. Otherwise, it´s some integration mistake from meta-fsl-arm (VPU firmaware may be different for imx6S, for example, and we´ve been using the same from imx6q)

I´m going to start a bitbake for testing this now, and probably I´m able to try tomorrow. If it´s a vpuenc error, here is the best place to handle. If it´s an integration mistake/error, the better place to deal with the issue is meta-freescale (https://lists.yoctoproject.org/listinfo/meta-freescale)

Tomorrrow I come back to this topic and update it.

Re: gstreamer vpuenc H.264 encoder (codec=6) output not compressed

normancheung — Mon, 16 Sep 2013 21:16:18 GMT

Daiane,

I am using the master branch. My board is wandboard-solo.

I have done more experiments, and found that this only happens which the vpuenc is set to Auto bitrate (i.e. use the default =0). The frame size for fix bitrate is good, I tried 1Mbps and 1.5Mbps all give reasonable;e frame size.

thanks,

Norman

Re: gstreamer vpuenc H.264 encoder (codec=6) output not compressed

daiane_angolini — Tue, 17 Sep 2013 12:22:49 GMT

Using the following command lines:

number of buffers before EOF = 100

gst-launch mfw_v4lsrc num-buffers=100 capture-mode=4 ! queue ! vpuenc codec=6 ! matroskamux ! filesink location=test_h264_100.mkv

gst-launch mfw_v4lsrc num-buffers=100 capture-mode=4 ! queue ! vpuenc codec=0 ! matroskamux ! filesink location=test_mpeg4_100.mkv

number of buffers before EOF = 200

gst-launch mfw_v4lsrc num-buffers=200 capture-mode=4 ! queue ! vpuenc codec=6 ! matroskamux ! filesink location=test_h264_200.mkv

gst-launch mfw_v4lsrc num-buffers=200 capture-mode=4 ! queue ! vpuenc codec=0 ! matroskamux ! filesink location=test_mpeg4_200.mkv

number of buffers before EOF = 300

gst-launch mfw_v4lsrc num-buffers=300 capture-mode=4 ! queue ! vpuenc codec=6 ! matroskamux ! filesink location=test_h264_300.mkv

gst-launch mfw_v4lsrc num-buffers=300 capture-mode=4 ! queue ! vpuenc codec=0 ! matroskamux ! filesink location=test_mpeg4_300.mkv

I got the following results:

-rwxr-xr-x 1 root root 19.4M Sep 16 20:25 test_h264_100.mkv

-rwxr-xr-x 1 root root 38.8M Sep 16 20:24 test_h264_200.mkv

-rwxr-xr-x 1 root root 58.1M Sep 16 20:25 test_h264_300.mkv

-rwxr-xr-x 1 root root 531.1K Sep 16 20:25 test_mpeg4_100.mkv

-rwxr-xr-x 1 root root 1.0M Sep 16 20:25 test_mpeg4_200.mkv

-rwxr-xr-x 1 root root 1.5M Sep 16 20:26 test_mpeg4_300.mkv

I'm using this env:

Linux imx6qsabresd 3.0.35-4.1.0+yocto+gbdde708 #1 SMP PREEMPT Mon Sep 16 16:20:23 BRT 2013 armv7l GNU/Linux

U-Boot 2013.10-rc2 (Sep 16 2013 - 16:46:38)

CPU: Freescale i.MX6Q rev1.2 at 792 MHz

Board: MX6-SabreSD

DRAM: 1 GiB

So, it's not an integration error, it's something related with MM plugins/encoders

karinavalencia, we need to escalate this bug to get fixed on next MM release. (or someone to explain why this is not a bug)

Deactivated user, include plain text is a pain!!!!!!!!!!!!!! :smileysad::smileycry:

Re: gstreamer vpuenc H.264 encoder (codec=6) output not compressed

karina_valencia — Tue, 17 Sep 2013 14:06:44 GMT

DaianeAngolini please create a CT for follow up.

Re: gstreamer vpuenc H.264 encoder (codec=6) output not compressed

daiane_angolini — Fri, 20 Sep 2013 13:55:47 GMT

Created CT#43609703

Re: gstreamer vpuenc H.264 encoder (codec=6) output not compressed

daiane_angolini — Mon, 23 Sep 2013 16:16:54 GMT

The QP range for mpeg4(1-31) and h.264(0-51) are different.

As a result, there will be big gap between output size for the same default qp.

So, we suggest user encode clip with appointed bitrate instead of default qp.

>gst-launch mfw_v4lsrc num-buffers=100 capture-mode=4 ! queue ! vpuenc codec=6 bitrate=4000000 ! matroskamux ! filesink location=test_h264_100.mkv

Please, let me know your results.

Re: gstreamer vpuenc H.264 encoder (codec=6) output not compressed

normancheung — Tue, 24 Sep 2013 04:19:58 GMT

Daiane,

See my post in this thread on Sep 16, 2013 2:16 PM. I have tried fixed bitrate and it is good. But I was hoping to use auto adjust variable bitrate.

This is not the news I would like to hear. If the issue is in the QP, than it is likely to be in the VPU and it is not likely to be patchable.

Thanks anyway.

Norman

Re: gstreamer vpuenc H.264 encoder (codec=6) output not compressed

daiane_angolini — Tue, 22 Oct 2013 17:57:11 GMT

Sorry, Norman

I did not understand your point. Could you, please, elaborate?

Re: gstreamer vpuenc H.264 encoder (codec=6) output not compressed

jparrish88 — Sat, 15 Feb 2014 04:34:42 GMT

Hi Daiane,

Do you and others feel that this issue has been fixed?

If so, does that mean the the fix is a bitrate adjustment only or is there something else that may not have been mentioned?

If not, what is the status on a patch or fix?

Re: gstreamer vpuenc H.264 encoder (codec=6) output not compressed

XXiao1z — Sun, 20 Apr 2014 03:35:42 GMT

any update on the h.264 vpnenc bug? I need do the same here.