Poor framerates on iMX6 Solo when decoding H264

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Poor framerates on iMX6 Solo when decoding H264

5,124 Views
ralph__
Contributor IV

I'm using gstreamer to playback some H264 videos - one is 720p60 and one is 1080p30. The framerates are much worse than expected (the iMX can do 1080p30 and 720p60 allegedly....) and I am certain the hardware is doing the decoding as it only has 1-2% CPU utilisation.

Here are the commands I run to test the video. Either:

gst-launch-0.10 filesrc location=/usr/sample.mp4 ! qtdemux ! vpudec ! mfw_isink

or:

gplay /usr/sample.mp4

Both lines give me the same end result which is this:

1080p30 clip runs at 23fps (reported)

720p60 clip runs at 27fps (reported)

Why are the clips running so slowly? If I play back either clip in VLC on Windows the framerates are as expected; i.e. 30fps for the 1080p30 and 60fps for the 1080p60.


Thanks for your help,

Ralph

Tags (2)
0 Kudos
25 Replies

2,038 Views
igorpadykov
NXP Employee
NXP Employee

Hi ralph

I would suggest to look at below links where

all video test procedures are well summarized in one place

https://community.freescale.com/docs/DOC-93387/version/4

Testing things out

http://boundarydevices.com/mx6-video-acceleration-raring-debian/

Best regards

igor

0 Kudos

2,038 Views
ralph__
Contributor IV

Hi Igor,

I've looked at both those links - thanks for providing them - however none of the mp4-related commands work for me and the "Sintel" clip that they suggest to use isn't even 1080p30! Even then, the "Sintel" clip only runs at 24fps, not at the 30fps that I'd expect.

Ralph

0 Kudos

2,038 Views
igorpadykov
NXP Employee
NXP Employee

Hi ralph

I would suggest to test with Demo Image

L3.0.35_4.1.0_DEMO_IMAGE_BSP : i.MX 6Quad, i.MX 6Dual, i.MX 6DualLite, i.MX 6Solo and i.MX 6Sololite Linux Binary Demo Files.

One can also try kernel parameters : enable_wait_mode=off, maxcpus=0 (nosmp)

and check if SD (if video is located on it) is high class speed.

Best regards

igor

0 Kudos

2,038 Views
ralph__
Contributor IV

Okay, I've rebuilt my system and my results are as follows for a 720p60 clip:

echo "S:1280x720p-60" > /sys/class/graphics/fb0/mode

gst-launch filesrc location=/usr/sample720p60_40Mbps.mp4 typefind=true ! qtdemux ! vpudec framedrop=false ! queue max-size-buffers=10 ! mfw_isink sync=false disp-width=1280 disp-height=720

gplay /usr/sample720p60_40Mbps.mp4

Both the gplay and gst-launch commands give 56fps. If I then switch to a 1080p30 clip:

echo "S:1920x1080p-60" > /sys/class/graphics/fb0/mode

gplay /usr/sample1080p30_40Mbps.mp4

Then I only get 23fps still.

So I still need to get the 56fps for 720p clips up to 60fps, and I need to get the 23fps up to 30fps for 1080p clips.

I've also been trying the daisy, dizzy and master images built from scratch using Yocto. None of these give me the full frame rate results either. :'-(

By the way, if I change the sink to be mfw_v4lsink then I still get the same frame rates.

Ralph

0 Kudos

2,038 Views
igorpadykov
NXP Employee
NXP Employee

I think you should test on Freescale Sabre SD reference board (i.MX6DL procesor)

with latest Demo

L3.0.101_4.1.1_IM6QDLS_BUNDLE : i.MX 6Quad, i.MX 6Dual, i.MX 6DualLite, i.MX 6Solo Linux Binary Demo Files.

If you are using SabreLite board please post this on boundary devices forum

since Freescale images may not work good on Sabrelite board.

~igor

0 Kudos

2,035 Views
ralph__
Contributor IV

One thing I haven't mentioned is that on my Nitrogen6_lite Boundary Devices board, I have actually had it running at 1080p30 and 720p60 using the LTIB image. Here is the link where I got the image from:

http://boundarydevices.com/ltib-on-i-mx6-without-x/

gplay /nfs/sample720p60_40Mbps.mp4   [60fps]

echo "S:1920x1080p-60" > /sys/class/graphics/fb0/mode

gplay /nfs/sample1080p30_40Mbps.mp4   [30fps]

There are some strange differences between this image and my own. For LTIB, here is some interesting info:

[kernel boot messages]

Memory: 384MB = 384MB total

Memory: 381008k/381008k available, 143280k reserved, 0K highmem

Linux version 3.0.35-2026-geaaf30e-02079-g2a4aa352 (ericn@ericsam) (gcc version 4.6.2 20110630 (prerelease) (Freescale MAD -- Linaro 2011.07 -- Built at 2011/08/10 09:20) ) #4 SMP PREEMPT Tue Dec 18 08:40:53 MST 2012

CPU identified as i.MX6DL/SOLO, silicon rev 1.1

Virtual kernel memory layout:

    vector  : 0xffff0000 - 0xffff1000   (   4 kB)

    fixmap  : 0xfff00000 - 0xfffe0000   ( 896 kB)

    DMA     : 0xf4600000 - 0xffe00000   ( 184 MB)

    vmalloc : 0xa0800000 - 0xf2000000   (1304 MB)

    lowmem  : 0x80000000 - 0xa0000000   ( 512 MB)

    pkmap   : 0x7fe00000 - 0x80000000   (   2 MB)

    modules : 0x7f000000 - 0x7fe00000   (  14 MB)

      .init : 0x80008000 - 0x80038000   ( 192 kB)

      .text : 0x80038000 - 0x806d81b0   (6785 kB)

      .data : 0x806da000 - 0x8072eb20   ( 339 kB)

       .bss : 0x8072eb44 - 0x8077a9a0   ( 304 kB)

root@freescale ~$ cat /proc/cpuinfo

Processor       : ARMv7 Processor rev 10 (v7l)

processor       : 0

BogoMIPS        : 1988.28

Features        : swp half thumb fastmult vfp edsp neon vfpv3

CPU implementer : 0x41

CPU architecture: 7

CPU variant     : 0x2

CPU part        : 0xc09

CPU revision    : 10

Hardware        : Freescale i.MX 6Quad Sabre-Lite Board

Revision        : 61011

Serial          : 0000000000000000

root@freescale ~$ cat /proc/meminfo

MemTotal:         381200 kB

MemFree:          358960 kB

...

VmallocTotal:    1335296 kB

VmallocUsed:        2964 kB

VmallocChunk:    1331744 kB

Our kernel parameters are all the same and I have CONFIG_HIGHMEM enabled in my kernel config. For my own board I have this:

[kernel boot messages]

Memory: 512MB = 512MB total

Memory: 237596k/237596k available, 286692k reserved, 0K highmem

Linux version 3.10.17 (ralphc@waldorf) (gcc version 4.8.1 (Sourcery CodeBench Lite 2013.11-33) ) #4 SMP Fri Nov 7 11:44:20 GMT 2014

CPU identified as i.MX6DL, silicon rev 1.1

    vector  : 0xffff0000 - 0xffff1000   (   4 kB)

    fixmap  : 0xfff00000 - 0xfffe0000   ( 896 kB)

    vmalloc : 0xe0800000 - 0xff000000   ( 488 MB)       ***LTIB is 1304MB***

    lowmem  : 0xc0000000 - 0xe0000000   ( 512 MB)      

    modules : 0xbf000000 - 0xc0000000   (  16 MB)

      .text : 0xc0008000 - 0xc0790f9c   (7716 kB)

      .init : 0xc0791000 - 0xc18e6200   (17749 kB)

      .data : 0xc18e8000 - 0xc1924b60   ( 243 kB)

       .bss : 0xc1924b60 - 0xc1983dbc   ( 381 kB)

root@freescale ~$ cat /proc/cpuinfo

processor       : 0

model name      : ARMv7 Processor rev 10 (v7l)

BogoMIPS        : 790.52

Features        : swp half thumb fastmult vfp edsp neon vfpv3 tls

CPU implementer : 0x41

CPU architecture: 7

CPU variant     : 0x2

CPU part        : 0xc09

CPU revision    : 10

Hardware        : Freescale i.MX6 Quad/DualLite (Device Tree)

Revision        : 61011

Serial          : 0000000000000000

# cat /proc/meminfo

MemTotal:         510728 kB

MemFree:          342616 kB

...

VmallocTotal:     499712 kB

VmallocUsed:        7212 kB

VmallocChunk:     316008 kB

I notice that I have no DMA (2MB) or pkmap allocated on mine (184MB) compared to the LTIB board. Also, the LTIB image has more vmalloc memory than there even is on the board!!! (I only have 512MB according to the datasheet). Could the memory allocation setup be the cause of my problems? There is one other difference and that is in the vpu.imx gstreamer component that is used for decoding mp4 is version 3.0.2 for the LTIB board and 3.0.11 for my board.

Ralph

0 Kudos

2,031 Views
igorpadykov
NXP Employee
NXP Employee

please post this on boundary devices forum

0 Kudos

2,031 Views
ralph__
Contributor IV

They don't have a forum.

The issue I have here is with the Freescale graphic libraries and firmware and the fact that neither me nor anyone else has managed to get it decoding at 720p60 or 1080p30.

0 Kudos

2,031 Views
igorpadykov
NXP Employee
NXP Employee

software for this board requires optimized, specially prepared

BSP, it is provided and supported by boundary devices directly.

You can try meta-fsl-arm mailing list

https://lists.yoctoproject.org/listinfo/meta-freescale

0 Kudos

2,031 Views
ralph__
Contributor IV

I've already posted to the meta-fsl-arm mailing list but didn't get any replies.


I am already using patches from Boundary Devices. I have tried increasing the VPU frequency but this hasn't fixed things either.

Do you have any ideas what I could investigate to improve the decoding speed on my system? I was thinking it might be something to do with my memory but since this is set up the same way each time by my U-Boot I don't think this is the problem.

Thanks,
Ralph

0 Kudos

2,031 Views
igorpadykov
NXP Employee
NXP Employee

Suggest to ask on meta-fsl-arm mailing who is maintainer of this board

and direct questions to him.

There are many boards with i.MX6, but they are maintained and supported

by its vendors. Also note that i.MX6 perfomance given in RM/Datasheet is given without

OS. For obtaining good performance one needs to fine tune BSP.

Freescale provides own reference designs i.MX6 Sabre SD/AI and BSPs for them,

you can try them

i.MX6S|i.MX 6Solo Processors|Multimedia|Freescale

Best regards

igor

0 Kudos

2,038 Views
ralph__
Contributor IV

Hi, I tried those kernel parameters but no luck.

I'll try the demo image and report back.

Thanks,
Ralph

0 Kudos

2,038 Views
ralph__
Contributor IV

Sadly those images are all for the Sabre board (whatever that is) and not the IMX6Solo which is what I have. Here is the boot trace before it hangs:

Linux version 3.0.35-2666-gbdde708 (r65388@shlinux3) (gcc version 4.6.2 20110630 (prerelease) (Freescale MAD -- Linaro 2011.07 -- Built at 2011/08/10 09:20) ) #1 SMP PREEMPT Fri Aug 16 10:43:44 CST 2013

CPU: ARMv7 Processor [412fc09a] revision 10 (ARMv7), cr=10c53c7d

CPU: VIPT nonaliasing data cache, VIPT aliasing instruction cache

Machine: Freescale i.MX 6Quad Sabre-Lite Board

Memory policy: ECC disabled, Data cache writealloc

CPU identified as i.MX6DL/SOLO, silicon rev 1.1

0 Kudos

2,038 Views
igorpadykov
NXP Employee
NXP Employee

Hi ralph

if you are using Sabre-Lite then it is better to

check boundary images, like below

http://boundarydevices.com/running-linaro-ubuntu-on-i-mx6-sabre-lite-and-nitrogen6x/

~igor

0 Kudos

2,038 Views
ralph__
Contributor IV

After a reasonably massive struggle, I managed to get the original DEMO image up and running. Sadly, there are no H264 or mp4 files on that image and the gstreamer framework fails when I try to get one of my own mp4 files working that I know work on my own system.

Thanks for the Ubuntu link. I'll try that and see if I have better luck.

0 Kudos

2,038 Views
xizhouwang
Contributor II

Can you try a different sink like

      gst-launch playbin2 uri=file:///video-file.mp4  video-sink="mfw_v4lsink"

to play your clips (change the file name) .

0 Kudos

2,038 Views
ralph__
Contributor IV

I've tried this but all I get is:

gst-launch playbin2 uri=file:///nfs/sample.mp4  video-sink="mf

MFW_GST_V4LSINK_PLUGIN 3.0.10 build on Mar  2 2014 17:33:04.

Setting pipeline to PAUSED ...

Pipeline is PREROLLING ...

Aiur: 3.0.10

Core: MPEG4PARSER_06.07.03  build on Nov 15 2013 01:36:39

  mime: video/quicktime; audio/x-m4a; application/x-3gp

  file: /usr/lib/imx-mm/parser/lib_mp4_parser_arm11_elinux.so.3.2

Content Info:

        URI:

              file:///nfs/sample.mp4

        Idx File:

              /home/root/.aiur/.nfs.sample.mp4.aidx

        Seekable  : Yes

        Size(byte): 14621544

Movie Info:

        Seekable  : Yes

        Live      : No

        Duration  : 0:00:52.209000000

        ReadMode  : Track

        Track     : 2

Track 00 [video_000000] Enabled

        Duration: 0:00:52.208000000

        Language: und

        Mime:

              video/x-h264, parsed=(boolean)true, width=(int)1920, height=(int)1080, framerate

              =(fraction)24/1

[ERR]   Error in opening firmware binary file

[ERR]   Please put bin file to /lib/firmware/vpu folder or export VPU_FW_PATH env

H264D_ARM11_02.06.03  build on Apr 16 2013 13:30:59.

MFW_GST_H264_DECODER_PLUGIN 3.0.10 build on Mar  2 2014 17:33:34.

Track 01 [audio_000000] Enabled

        Duration: 0:00:51.946000000

        Language: und

        Mime:

              audio/mpeg, mpegversion=(int)4, channels=(int)2, rate=(int)48000, bitrate=(int)0

              , framed=(boolean)true, stream-format=(string)raw, codec_data=(buffer)1190

ERROR: from element /GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstDecodeBin2:decodebin20/MFW_GST_H264DEC_INFO_T:mfw_gst_h264dec_info_t0: fatal error

Additional debug info:

non recoverable error while decoding the H.264 Data

ERROR: pipeline doesn't want to preroll.

Setting pipeline to NULL ...

Total rendered:0

Freeing pipeline ...

[--->FINALIZE v4l_sink

In fact, I always get the error message "Error in opening firmware binary file" and I've tried loads of different command lines with gstreamer. :-( Here are some of the environment variables I have set up:

export GST_PLUGIN_PATH="/usr/lib/gstreamer-0.10/:/usr/lib"

export LD_LIBRARY_PATH="/usr/lib/gstreamer-0.10/:/usr/lib"

export VPU_FW_PATH="/usr/lib/gstreamer-0.10/:/usr/lib:/lib/firmware/vpu"

Ralph

0 Kudos

2,038 Views
xizhouwang
Contributor II

do this

     gst-inspect |grep mfw

to check what sink are on your system. Here is from one of my fs.

# gst-inspect |grep mfw

ipucsc.imx:  mfw_ipucsc: IPU-based video converter

v4lsrc.imx:  mfw_v4lsrc: v4l2 based camera src

mpeg2dec.imx:  mfw_mpeg2decoder: mpeg2 video decoder

amrdec.imx:  mfw_amrdecoder: amr audio decoder

aacdec.imx:  mfw_aacdecoder: aac audio decoder

v4lsink.imx:  mfw_v4lsink: v4l2 video sink

mpeg4dec.imx:  mfw_mpeg4aspdecoder: mpeg4 video decoder

audiopeq.imx:  mfw_audio_pp: audio post equalizer

h264.imx:  mfw_h264decoder: h264 video decoder

isink.imx:  mfw_isink: IPU-based video sink

mp3enc.imx:  mfw_mp3encoder: mp3 audio encoder

vorbisdec.imx:  mfw_vorbisdecoder: vorbis audio decoder

0 Kudos

2,038 Views
ralph__
Contributor IV

Here you go:

root@nitrogen6x:~# gst-inspect | grep mfw

ipucsc.imx:  mfw_ipucsc: IPU-based video converter

v4lsrc.imx:  mfw_v4lsrc: v4l2 based camera src

h264.imx:  mfw_h264decoder: h264 video decoder

mp3enc.imx:  mfw_mp3encoder: mp3 audio encoder

isink.imx:  mfw_isink: IPU-based video sink

amrdec.imx:  mfw_amrdecoder: amr audio decoder

v4lsink.imx:  mfw_v4lsink: v4l2 video sink

audiopeq.imx:  mfw_audio_pp: audio post equalizer

mpeg2dec.imx:  mfw_mpeg2decoder: mpeg2 video decoder

mpeg4dec.imx:  mfw_mpeg4aspdecoder: mpeg4 video decoder

I tried using mfw_v4lsink and mfw_mpeg4aspdecoder in various combinations but no luck.

Ralph

0 Kudos

2,037 Views
xizhouwang
Contributor II

This link could be helpful for your firmware ERR.

Error in opening firmware binary file for i.mx6 solo wandboard

0 Kudos