topic Re: Poor framerates on iMX6 Solo when decoding H264 in i.MX Processors

Poor framerates on iMX6 Solo when decoding H264

ralph__ — Thu, 09 Oct 2014 10:25:42 GMT

I'm using gstreamer to playback some H264 videos - one is 720p60 and one is 1080p30. The framerates are much worse than expected (the iMX can do 1080p30 and 720p60 allegedly....) and I am certain the hardware is doing the decoding as it only has 1-2% CPU utilisation.

Here are the commands I run to test the video. Either:

gst-launch-0.10 filesrc location=/usr/sample.mp4 ! qtdemux ! vpudec ! mfw_isink

or:

gplay /usr/sample.mp4

Both lines give me the same end result which is this:

1080p30 clip runs at 23fps (reported)

720p60 clip runs at 27fps (reported)

Why are the clips running so slowly? If I play back either clip in VLC on Windows the framerates are as expected; i.e. 30fps for the 1080p30 and 60fps for the 1080p60.

Thanks for your help,

Ralph

Re: Poor framerates on iMX6 Solo when decoding H264

igorpadykov — Thu, 09 Oct 2014 13:09:30 GMT

Hi ralph

I would suggest to look at below links where

all video test procedures are well summarized in one place

https://community.freescale.com/docs/DOC-93387/version/4

Testing things out

http://boundarydevices.com/mx6-video-acceleration-raring-debian/

Best regards

igor

Re: Poor framerates on iMX6 Solo when decoding H264

ralph__ — Thu, 09 Oct 2014 15:47:22 GMT

Hi Igor,

I've looked at both those links - thanks for providing them - however none of the mp4-related commands work for me and the "Sintel" clip that they suggest to use isn't even 1080p30! Even then, the "Sintel" clip only runs at 24fps, not at the 30fps that I'd expect.

Ralph

Re: Poor framerates on iMX6 Solo when decoding H264

igorpadykov — Fri, 10 Oct 2014 09:59:23 GMT

Hi ralph

I would suggest to test with Demo Image

L3.0.35_4.1.0_DEMO_IMAGE_BSP : i.MX 6Quad, i.MX 6Dual, i.MX 6DualLite, i.MX 6Solo and i.MX 6Sololite Linux Binary Demo Files.

One can also try kernel parameters : enable_wait_mode=off, maxcpus=0 (nosmp)

and check if SD (if video is located on it) is high class speed.

Best regards

igor

Re: Poor framerates on iMX6 Solo when decoding H264

ralph__ — Tue, 14 Oct 2014 13:51:29 GMT

Hi, I tried those kernel parameters but no luck.

I'll try the demo image and report back.

Thanks,
Ralph

Re: Poor framerates on iMX6 Solo when decoding H264

ralph__ — Mon, 20 Oct 2014 12:31:28 GMT

Sadly those images are all for the Sabre board (whatever that is) and not the IMX6Solo which is what I have. Here is the boot trace before it hangs:

Linux version 3.0.35-2666-gbdde708 (r65388@shlinux3) (gcc version 4.6.2 20110630 (prerelease) (Freescale MAD -- Linaro 2011.07 -- Built at 2011/08/10 09:20) ) #1 SMP PREEMPT Fri Aug 16 10:43:44 CST 2013

CPU: ARMv7 Processor [412fc09a] revision 10 (ARMv7), cr=10c53c7d

CPU: VIPT nonaliasing data cache, VIPT aliasing instruction cache

Machine: Freescale i.MX 6Quad Sabre-Lite Board

Memory policy: ECC disabled, Data cache writealloc

CPU identified as i.MX6DL/SOLO, silicon rev 1.1

Re: Poor framerates on iMX6 Solo when decoding H264

igorpadykov — Mon, 20 Oct 2014 14:44:48 GMT

Hi ralph

if you are using Sabre-Lite then it is better to

check boundary images, like below

http://boundarydevices.com/running-linaro-ubuntu-on-i-mx6-sabre-lite-and-nitrogen6x/

~igor

Re: Poor framerates on iMX6 Solo when decoding H264

ralph__ — Tue, 21 Oct 2014 14:06:47 GMT

After a reasonably massive struggle, I managed to get the original DEMO image up and running. Sadly, there are no H264 or mp4 files on that image and the gstreamer framework fails when I try to get one of my own mp4 files working that I know work on my own system.

Thanks for the Ubuntu link. I'll try that and see if I have better luck.

Re: Poor framerates on iMX6 Solo when decoding H264

xizhouwang — Tue, 21 Oct 2014 15:20:18 GMT

Can you try a different sink like

gst-launch playbin2 uri=file:///video-file.mp4 video-sink="mfw_v4lsink"

to play your clips (change the file name) .

Re: Poor framerates on iMX6 Solo when decoding H264

ralph__ — Tue, 21 Oct 2014 15:25:32 GMT

I've tried this but all I get is:

gst-launch playbin2 uri=file:///nfs/sample.mp4 video-sink="mf

MFW_GST_V4LSINK_PLUGIN 3.0.10 build on Mar 2 2014 17:33:04.

Setting pipeline to PAUSED ...

Pipeline is PREROLLING ...

Aiur: 3.0.10

Core: MPEG4PARSER_06.07.03 build on Nov 15 2013 01:36:39

mime: video/quicktime; audio/x-m4a; application/x-3gp

file: /usr/lib/imx-mm/parser/lib_mp4_parser_arm11_elinux.so.3.2

Content Info:

URI:

file:///nfs/sample.mp4

Idx File:

/home/root/.aiur/.nfs.sample.mp4.aidx

Seekable : Yes

Size(byte): 14621544

Movie Info:

Seekable : Yes

Live : No

Duration : 0:00:52.209000000

ReadMode : Track

Track : 2

Track 00 [video_000000] Enabled

Duration: 0:00:52.208000000

Language: und

Mime:

video/x-h264, parsed=(boolean)true, width=(int)1920, height=(int)1080, framerate

=(fraction)24/1

[ERR] Error in opening firmware binary file

[ERR] Please put bin file to /lib/firmware/vpu folder or export VPU_FW_PATH env

H264D_ARM11_02.06.03 build on Apr 16 2013 13:30:59.

MFW_GST_H264_DECODER_PLUGIN 3.0.10 build on Mar 2 2014 17:33:34.

Track 01 [audio_000000] Enabled

Duration: 0:00:51.946000000

Language: und

Mime:

audio/mpeg, mpegversion=(int)4, channels=(int)2, rate=(int)48000, bitrate=(int)0

, framed=(boolean)true, stream-format=(string)raw, codec_data=(buffer)1190

ERROR: from element /GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstDecodeBin2:decodebin20/MFW_GST_H264DEC_INFO_T:mfw_gst_h264dec_info_t0: fatal error

Additional debug info:

non recoverable error while decoding the H.264 Data

ERROR: pipeline doesn't want to preroll.

Setting pipeline to NULL ...

Total rendered:0

Freeing pipeline ...

[--->FINALIZE v4l_sink

In fact, I always get the error message "Error in opening firmware binary file" and I've tried loads of different command lines with gstreamer. :-( Here are some of the environment variables I have set up:

export GST_PLUGIN_PATH="/usr/lib/gstreamer-0.10/:/usr/lib"

export LD_LIBRARY_PATH="/usr/lib/gstreamer-0.10/:/usr/lib"

export VPU_FW_PATH="/usr/lib/gstreamer-0.10/:/usr/lib:/lib/firmware/vpu"

Ralph

Re: Poor framerates on iMX6 Solo when decoding H264

xizhouwang — Tue, 21 Oct 2014 16:02:02 GMT

do this

gst-inspect |grep mfw

to check what sink are on your system. Here is from one of my fs.

# gst-inspect |grep mfw

ipucsc.imx: mfw_ipucsc: IPU-based video converter

v4lsrc.imx: mfw_v4lsrc: v4l2 based camera src

mpeg2dec.imx: mfw_mpeg2decoder: mpeg2 video decoder

amrdec.imx: mfw_amrdecoder: amr audio decoder

aacdec.imx: mfw_aacdecoder: aac audio decoder

v4lsink.imx: mfw_v4lsink: v4l2 video sink

mpeg4dec.imx: mfw_mpeg4aspdecoder: mpeg4 video decoder

audiopeq.imx: mfw_audio_pp: audio post equalizer

h264.imx: mfw_h264decoder: h264 video decoder

isink.imx: mfw_isink: IPU-based video sink

mp3enc.imx: mfw_mp3encoder: mp3 audio encoder

vorbisdec.imx: mfw_vorbisdecoder: vorbis audio decoder

Re: Poor framerates on iMX6 Solo when decoding H264

ralph__ — Tue, 21 Oct 2014 16:18:03 GMT

Here you go:

root@nitrogen6x:~# gst-inspect | grep mfw

ipucsc.imx: mfw_ipucsc: IPU-based video converter

v4lsrc.imx: mfw_v4lsrc: v4l2 based camera src

h264.imx: mfw_h264decoder: h264 video decoder

mp3enc.imx: mfw_mp3encoder: mp3 audio encoder

isink.imx: mfw_isink: IPU-based video sink

amrdec.imx: mfw_amrdecoder: amr audio decoder

v4lsink.imx: mfw_v4lsink: v4l2 video sink

audiopeq.imx: mfw_audio_pp: audio post equalizer

mpeg2dec.imx: mfw_mpeg2decoder: mpeg2 video decoder

mpeg4dec.imx: mfw_mpeg4aspdecoder: mpeg4 video decoder

I tried using mfw_v4lsink and mfw_mpeg4aspdecoder in various combinations but no luck.

Ralph

Re: Poor framerates on iMX6 Solo when decoding H264

xizhouwang — Wed, 22 Oct 2014 08:07:30 GMT

This link could be helpful for your firmware ERR.

Error in opening firmware binary file for i.mx6 solo wandboard

Re: Poor framerates on iMX6 Solo when decoding H264

ralph__ — Wed, 22 Oct 2014 16:29:52 GMT

Thanks for the link. That looks like it may in fact fix things however after applying the patch in the link, I flashed U-Boot and have currently bricked my board. I'm attempting to unbrick it over USB however it doesn't show up on my PC over USB.... :-(

Re: Poor framerates on iMX6 Solo when decoding H264

xizhouwang — Thu, 23 Oct 2014 09:01:32 GMT

Sorry to hear that and hope you will get it going again soon.

Re: Poor framerates on iMX6 Solo when decoding H264

ralph__ — Thu, 23 Oct 2014 14:27:54 GMT

Hi, I've unbricked my device now and flashed a U-Boot that has all functions that return the CPU revision return 0x61011. The patch in the link did not apply very well at all so I manually edited arch/arm/cpu/armv7/mx6/soc.c so that the revision was returned as a fixed number.

When I rebooted the unit, I found that the hardware revision was only set correctly in /proc/cpuinfo when I booted *my* kernel image. When I tried booting the Freescale DEMO kernel image it came up with a "hardware revision" of 0000.

Why would this be? Both kernel images are booting from the same bootloader.

Thanks,
Ralph

Re: Poor framerates on iMX6 Solo when decoding H264

xizhouwang — Fri, 24 Oct 2014 09:33:55 GMT

Returns from one of my Solo kit. Hope other expert may have advice on the issues.

root@:~# cat /proc/cpuinfo

Processor : ARMv7 Processor rev 10 (v7l)

processor : 0

BogoMIPS : 1581.05

Features : swp half thumb fastmult vfp edsp neon vfpv3

CPU implementer : 0x41

CPU architecture: 7

CPU variant : 0x2

CPU part : 0xc09

CPU revision : 10

Hardware :

Revision : 63000

Serial : 0000000000000000

Re: Poor framerates on iMX6 Solo when decoding H264

ralph__ — Fri, 07 Nov 2014 13:41:24 GMT

Okay, I've rebuilt my system and my results are as follows for a 720p60 clip:

echo "S:1280x720p-60" > /sys/class/graphics/fb0/mode

gst-launch filesrc location=/usr/sample720p60_40Mbps.mp4 typefind=true ! qtdemux ! vpudec framedrop=false ! queue max-size-buffers=10 ! mfw_isink sync=false disp-width=1280 disp-height=720

gplay /usr/sample720p60_40Mbps.mp4

Both the gplay and gst-launch commands give 56fps. If I then switch to a 1080p30 clip:

echo "S:1920x1080p-60" > /sys/class/graphics/fb0/mode

gplay /usr/sample1080p30_40Mbps.mp4

Then I only get 23fps still.

So I still need to get the 56fps for 720p clips up to 60fps, and I need to get the 23fps up to 30fps for 1080p clips.

I've also been trying the daisy, dizzy and master images built from scratch using Yocto. None of these give me the full frame rate results either. :'-(

By the way, if I change the sink to be mfw_v4lsink then I still get the same frame rates.

Ralph

Re: Re: Poor framerates on iMX6 Solo when decoding H264

igorpadykov — Fri, 07 Nov 2014 13:48:38 GMT

I think you should test on Freescale Sabre SD reference board (i.MX6DL procesor)

with latest Demo

L3.0.101_4.1.1_IM6QDLS_BUNDLE : i.MX 6Quad, i.MX 6Dual, i.MX 6DualLite, i.MX 6Solo Linux Binary Demo Files.

If you are using SabreLite board please post this on boundary devices forum

since Freescale images may not work good on Sabrelite board.

~igor

Re: Poor framerates on iMX6 Solo when decoding H264

ralph__ — Fri, 07 Nov 2014 14:01:43 GMT

One thing I haven't mentioned is that on my Nitrogen6_lite Boundary Devices board, I have actually had it running at 1080p30 and 720p60 using the LTIB image. Here is the link where I got the image from:

http://boundarydevices.com/ltib-on-i-mx6-without-x/

gplay /nfs/sample720p60_40Mbps.mp4

[60fps]

echo "S:1920x1080p-60" > /sys/class/graphics/fb0/mode

gplay /nfs/sample1080p30_40Mbps.mp4

[30fps]

There are some strange differences between this image and my own. For LTIB, here is some interesting info:

[kernel boot messages]

Memory: 384MB = 384MB total

Memory: 381008k/381008k available, 143280k reserved, 0K highmem

Linux version 3.0.35-2026-geaaf30e-02079-g2a4aa352 (ericn@ericsam) (gcc version 4.6.2 20110630 (prerelease) (Freescale MAD -- Linaro 2011.07 -- Built at 2011/08/10 09:20) ) #4 SMP PREEMPT Tue Dec 18 08:40:53 MST 2012

CPU identified as i.MX6DL/SOLO, silicon rev 1.1

Virtual kernel memory layout:

vector : 0xffff0000 - 0xffff1000 ( 4 kB)

fixmap : 0xfff00000 - 0xfffe0000 ( 896 kB)

DMA : 0xf4600000 - 0xffe00000 ( 184 MB)

vmalloc : 0xa0800000 - 0xf2000000 (1304 MB)

lowmem : 0x80000000 - 0xa0000000 ( 512 MB)

pkmap : 0x7fe00000 - 0x80000000 ( 2 MB)

modules : 0x7f000000 - 0x7fe00000 ( 14 MB)

.init : 0x80008000 - 0x80038000 ( 192 kB)

.text : 0x80038000 - 0x806d81b0 (6785 kB)

.data : 0x806da000 - 0x8072eb20 ( 339 kB)

.bss : 0x8072eb44 - 0x8077a9a0 ( 304 kB)

root@freescale ~$ cat /proc/cpuinfo

Processor : ARMv7 Processor rev 10 (v7l)

processor : 0

BogoMIPS : 1988.28

Features : swp half thumb fastmult vfp edsp neon vfpv3

CPU implementer : 0x41

CPU architecture: 7

CPU variant : 0x2

CPU part : 0xc09

CPU revision : 10

Hardware : Freescale i.MX 6Quad Sabre-Lite Board

Revision : 61011

Serial : 0000000000000000

root@freescale ~$ cat /proc/meminfo

MemTotal: 381200 kB

MemFree: 358960 kB

...

VmallocTotal: 1335296 kB

VmallocUsed: 2964 kB

VmallocChunk: 1331744 kB

Our kernel parameters are all the same and I have CONFIG_HIGHMEM enabled in my kernel config. For my own board I have this:

[kernel boot messages]

Memory: 512MB = 512MB total

Memory: 237596k/237596k available, 286692k reserved, 0K highmem

Linux version 3.10.17 (ralphc@waldorf) (gcc version 4.8.1 (Sourcery CodeBench Lite 2013.11-33) ) #4 SMP Fri Nov 7 11:44:20 GMT 2014

CPU identified as i.MX6DL, silicon rev 1.1

vector : 0xffff0000 - 0xffff1000 ( 4 kB)

fixmap : 0xfff00000 - 0xfffe0000 ( 896 kB)

vmalloc : 0xe0800000 - 0xff000000 ( 488 MB) ***LTIB is 1304MB***

lowmem : 0xc0000000 - 0xe0000000 ( 512 MB)

modules : 0xbf000000 - 0xc0000000 ( 16 MB)

.text : 0xc0008000 - 0xc0790f9c (7716 kB)

.init : 0xc0791000 - 0xc18e6200 (17749 kB)

.data : 0xc18e8000 - 0xc1924b60 ( 243 kB)

.bss : 0xc1924b60 - 0xc1983dbc ( 381 kB)

root@freescale ~$ cat /proc/cpuinfo

processor : 0

model name : ARMv7 Processor rev 10 (v7l)

BogoMIPS : 790.52

Features : swp half thumb fastmult vfp edsp neon vfpv3 tls

CPU implementer : 0x41

CPU architecture: 7

CPU variant : 0x2

CPU part : 0xc09

CPU revision : 10

Hardware : Freescale i.MX6 Quad/DualLite (Device Tree)

Revision : 61011

Serial : 0000000000000000

# cat /proc/meminfo

MemTotal: 510728 kB

MemFree: 342616 kB

...

VmallocTotal: 499712 kB

VmallocUsed: 7212 kB

VmallocChunk: 316008 kB

I notice that I have no DMA (2MB) or pkmap allocated on mine (184MB) compared to the LTIB board. Also, the LTIB image has more vmalloc memory than there even is on the board!!! (I only have 512MB according to the datasheet). Could the memory allocation setup be the cause of my problems? There is one other difference and that is in the vpu.imx gstreamer component that is used for decoding mp4 is version 3.0.2 for the LTIB board and 3.0.11 for my board.

Ralph