I'm using gstreamer to playback some H264 videos - one is 720p60 and one is 1080p30. The framerates are much worse than expected (the iMX can do 1080p30 and 720p60 allegedly....) and I am certain the hardware is doing the decoding as it only has 1-2% CPU utilisation.
Here are the commands I run to test the video. Either:
gst-launch-0.10 filesrc location=/usr/sample.mp4 ! qtdemux ! vpudec ! mfw_isink
or:
gplay /usr/sample.mp4
Both lines give me the same end result which is this:
1080p30 clip runs at 23fps (reported)
720p60 clip runs at 27fps (reported)
Why are the clips running so slowly? If I play back either clip in VLC on Windows the framerates are as expected; i.e. 30fps for the 1080p30 and 60fps for the 1080p60.
Thanks for your help,
Ralph
Hi ralph
I would suggest to look at below links where
all video test procedures are well summarized in one place
https://community.freescale.com/docs/DOC-93387/version/4
Testing things out
http://boundarydevices.com/mx6-video-acceleration-raring-debian/
Best regards
igor
Hi Igor,
I've looked at both those links - thanks for providing them - however none of the mp4-related commands work for me and the "Sintel" clip that they suggest to use isn't even 1080p30! Even then, the "Sintel" clip only runs at 24fps, not at the 30fps that I'd expect.
Ralph
Hi ralph
I would suggest to test with Demo Image
L3.0.35_4.1.0_DEMO_IMAGE_BSP : i.MX 6Quad, i.MX 6Dual, i.MX 6DualLite, i.MX 6Solo and i.MX 6Sololite Linux Binary Demo Files.
One can also try kernel parameters : enable_wait_mode=off, maxcpus=0 (nosmp)
and check if SD (if video is located on it) is high class speed.
Best regards
igor
Okay, I've rebuilt my system and my results are as follows for a 720p60 clip:
echo "S:1280x720p-60" > /sys/class/graphics/fb0/mode
gst-launch filesrc location=/usr/sample720p60_40Mbps.mp4 typefind=true ! qtdemux ! vpudec framedrop=false ! queue max-size-buffers=10 ! mfw_isink sync=false disp-width=1280 disp-height=720
gplay /usr/sample720p60_40Mbps.mp4
Both the gplay and gst-launch commands give 56fps. If I then switch to a 1080p30 clip:
echo "S:1920x1080p-60" > /sys/class/graphics/fb0/mode
gplay /usr/sample1080p30_40Mbps.mp4
Then I only get 23fps still.
So I still need to get the 56fps for 720p clips up to 60fps, and I need to get the 23fps up to 30fps for 1080p clips.
I've also been trying the daisy, dizzy and master images built from scratch using Yocto. None of these give me the full frame rate results either. :'-(
By the way, if I change the sink to be mfw_v4lsink then I still get the same frame rates.
Ralph
I think you should test on Freescale Sabre SD reference board (i.MX6DL procesor)
with latest Demo
L3.0.101_4.1.1_IM6QDLS_BUNDLE : i.MX 6Quad, i.MX 6Dual, i.MX 6DualLite, i.MX 6Solo Linux Binary Demo Files.
If you are using SabreLite board please post this on boundary devices forum
since Freescale images may not work good on Sabrelite board.
~igor
One thing I haven't mentioned is that on my Nitrogen6_lite Boundary Devices board, I have actually had it running at 1080p30 and 720p60 using the LTIB image. Here is the link where I got the image from:
http://boundarydevices.com/ltib-on-i-mx6-without-x/
gplay /nfs/sample720p60_40Mbps.mp4 | [60fps] |
echo "S:1920x1080p-60" > /sys/class/graphics/fb0/mode
gplay /nfs/sample1080p30_40Mbps.mp4 | [30fps] |
There are some strange differences between this image and my own. For LTIB, here is some interesting info:
[kernel boot messages]
Memory: 384MB = 384MB total
Memory: 381008k/381008k available, 143280k reserved, 0K highmem
Linux version 3.0.35-2026-geaaf30e-02079-g2a4aa352 (ericn@ericsam) (gcc version 4.6.2 20110630 (prerelease) (Freescale MAD -- Linaro 2011.07 -- Built at 2011/08/10 09:20) ) #4 SMP PREEMPT Tue Dec 18 08:40:53 MST 2012
CPU identified as i.MX6DL/SOLO, silicon rev 1.1
Virtual kernel memory layout:
vector : 0xffff0000 - 0xffff1000 ( 4 kB)
fixmap : 0xfff00000 - 0xfffe0000 ( 896 kB)
DMA : 0xf4600000 - 0xffe00000 ( 184 MB)
vmalloc : 0xa0800000 - 0xf2000000 (1304 MB)
lowmem : 0x80000000 - 0xa0000000 ( 512 MB)
pkmap : 0x7fe00000 - 0x80000000 ( 2 MB)
modules : 0x7f000000 - 0x7fe00000 ( 14 MB)
.init : 0x80008000 - 0x80038000 ( 192 kB)
.text : 0x80038000 - 0x806d81b0 (6785 kB)
.data : 0x806da000 - 0x8072eb20 ( 339 kB)
.bss : 0x8072eb44 - 0x8077a9a0 ( 304 kB)
root@freescale ~$ cat /proc/cpuinfo
Processor : ARMv7 Processor rev 10 (v7l)
processor : 0
BogoMIPS : 1988.28
Features : swp half thumb fastmult vfp edsp neon vfpv3
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x2
CPU part : 0xc09
CPU revision : 10
Hardware : Freescale i.MX 6Quad Sabre-Lite Board
Revision : 61011
Serial : 0000000000000000
root@freescale ~$ cat /proc/meminfo
MemTotal: 381200 kB
MemFree: 358960 kB
...
VmallocTotal: 1335296 kB
VmallocUsed: 2964 kB
VmallocChunk: 1331744 kB
Our kernel parameters are all the same and I have CONFIG_HIGHMEM enabled in my kernel config. For my own board I have this:
[kernel boot messages]
Memory: 512MB = 512MB total
Memory: 237596k/237596k available, 286692k reserved, 0K highmem
Linux version 3.10.17 (ralphc@waldorf) (gcc version 4.8.1 (Sourcery CodeBench Lite 2013.11-33) ) #4 SMP Fri Nov 7 11:44:20 GMT 2014
CPU identified as i.MX6DL, silicon rev 1.1
vector : 0xffff0000 - 0xffff1000 ( 4 kB)
fixmap : 0xfff00000 - 0xfffe0000 ( 896 kB)
vmalloc : 0xe0800000 - 0xff000000 ( 488 MB) ***LTIB is 1304MB***
lowmem : 0xc0000000 - 0xe0000000 ( 512 MB)
modules : 0xbf000000 - 0xc0000000 ( 16 MB)
.text : 0xc0008000 - 0xc0790f9c (7716 kB)
.init : 0xc0791000 - 0xc18e6200 (17749 kB)
.data : 0xc18e8000 - 0xc1924b60 ( 243 kB)
.bss : 0xc1924b60 - 0xc1983dbc ( 381 kB)
root@freescale ~$ cat /proc/cpuinfo
processor : 0
model name : ARMv7 Processor rev 10 (v7l)
BogoMIPS : 790.52
Features : swp half thumb fastmult vfp edsp neon vfpv3 tls
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x2
CPU part : 0xc09
CPU revision : 10
Hardware : Freescale i.MX6 Quad/DualLite (Device Tree)
Revision : 61011
Serial : 0000000000000000
# cat /proc/meminfo
MemTotal: 510728 kB
MemFree: 342616 kB
...
VmallocTotal: 499712 kB
VmallocUsed: 7212 kB
VmallocChunk: 316008 kB
I notice that I have no DMA (2MB) or pkmap allocated on mine (184MB) compared to the LTIB board. Also, the LTIB image has more vmalloc memory than there even is on the board!!! (I only have 512MB according to the datasheet). Could the memory allocation setup be the cause of my problems? There is one other difference and that is in the vpu.imx gstreamer component that is used for decoding mp4 is version 3.0.2 for the LTIB board and 3.0.11 for my board.
Ralph
please post this on boundary devices forum
They don't have a forum.
The issue I have here is with the Freescale graphic libraries and firmware and the fact that neither me nor anyone else has managed to get it decoding at 720p60 or 1080p30.
software for this board requires optimized, specially prepared
BSP, it is provided and supported by boundary devices directly.
You can try meta-fsl-arm mailing list
I've already posted to the meta-fsl-arm mailing list but didn't get any replies.
I am already using patches from Boundary Devices. I have tried increasing the VPU frequency but this hasn't fixed things either.
Do you have any ideas what I could investigate to improve the decoding speed on my system? I was thinking it might be something to do with my memory but since this is set up the same way each time by my U-Boot I don't think this is the problem.
Thanks,
Ralph
Suggest to ask on meta-fsl-arm mailing who is maintainer of this board
and direct questions to him.
There are many boards with i.MX6, but they are maintained and supported
by its vendors. Also note that i.MX6 perfomance given in RM/Datasheet is given without
OS. For obtaining good performance one needs to fine tune BSP.
Freescale provides own reference designs i.MX6 Sabre SD/AI and BSPs for them,
you can try them
i.MX6S|i.MX 6Solo Processors|Multimedia|Freescale
Best regards
igor
Hi, I tried those kernel parameters but no luck.
I'll try the demo image and report back.
Thanks,
Ralph
Sadly those images are all for the Sabre board (whatever that is) and not the IMX6Solo which is what I have. Here is the boot trace before it hangs:
Linux version 3.0.35-2666-gbdde708 (r65388@shlinux3) (gcc version 4.6.2 20110630 (prerelease) (Freescale MAD -- Linaro 2011.07 -- Built at 2011/08/10 09:20) ) #1 SMP PREEMPT Fri Aug 16 10:43:44 CST 2013
CPU: ARMv7 Processor [412fc09a] revision 10 (ARMv7), cr=10c53c7d
CPU: VIPT nonaliasing data cache, VIPT aliasing instruction cache
Machine: Freescale i.MX 6Quad Sabre-Lite Board
Memory policy: ECC disabled, Data cache writealloc
CPU identified as i.MX6DL/SOLO, silicon rev 1.1
Hi ralph
if you are using Sabre-Lite then it is better to
check boundary images, like below
http://boundarydevices.com/running-linaro-ubuntu-on-i-mx6-sabre-lite-and-nitrogen6x/
~igor
After a reasonably massive struggle, I managed to get the original DEMO image up and running. Sadly, there are no H264 or mp4 files on that image and the gstreamer framework fails when I try to get one of my own mp4 files working that I know work on my own system.
Thanks for the Ubuntu link. I'll try that and see if I have better luck.
Can you try a different sink like
gst-launch playbin2 uri=file:///video-file.mp4 video-sink="mfw_v4lsink"
to play your clips (change the file name) .
I've tried this but all I get is:
gst-launch playbin2 uri=file:///nfs/sample.mp4 video-sink="mf
MFW_GST_V4LSINK_PLUGIN 3.0.10 build on Mar 2 2014 17:33:04.
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Aiur: 3.0.10
Core: MPEG4PARSER_06.07.03 build on Nov 15 2013 01:36:39
mime: video/quicktime; audio/x-m4a; application/x-3gp
file: /usr/lib/imx-mm/parser/lib_mp4_parser_arm11_elinux.so.3.2
Content Info:
URI:
file:///nfs/sample.mp4
Idx File:
/home/root/.aiur/.nfs.sample.mp4.aidx
Seekable : Yes
Size(byte): 14621544
Movie Info:
Seekable : Yes
Live : No
Duration : 0:00:52.209000000
ReadMode : Track
Track : 2
Track 00 [video_000000] Enabled
Duration: 0:00:52.208000000
Language: und
Mime:
video/x-h264, parsed=(boolean)true, width=(int)1920, height=(int)1080, framerate
=(fraction)24/1
[ERR] Error in opening firmware binary file
[ERR] Please put bin file to /lib/firmware/vpu folder or export VPU_FW_PATH env
H264D_ARM11_02.06.03 build on Apr 16 2013 13:30:59.
MFW_GST_H264_DECODER_PLUGIN 3.0.10 build on Mar 2 2014 17:33:34.
Track 01 [audio_000000] Enabled
Duration: 0:00:51.946000000
Language: und
Mime:
audio/mpeg, mpegversion=(int)4, channels=(int)2, rate=(int)48000, bitrate=(int)0
, framed=(boolean)true, stream-format=(string)raw, codec_data=(buffer)1190
ERROR: from element /GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstDecodeBin2:decodebin20/MFW_GST_H264DEC_INFO_T:mfw_gst_h264dec_info_t0: fatal error
Additional debug info:
non recoverable error while decoding the H.264 Data
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Total rendered:0
Freeing pipeline ...
[--->FINALIZE v4l_sink
In fact, I always get the error message "Error in opening firmware binary file" and I've tried loads of different command lines with gstreamer. :-( Here are some of the environment variables I have set up:
export GST_PLUGIN_PATH="/usr/lib/gstreamer-0.10/:/usr/lib"
export LD_LIBRARY_PATH="/usr/lib/gstreamer-0.10/:/usr/lib"
export VPU_FW_PATH="/usr/lib/gstreamer-0.10/:/usr/lib:/lib/firmware/vpu"
Ralph
do this
gst-inspect |grep mfw
to check what sink are on your system. Here is from one of my fs.
# gst-inspect |grep mfw
ipucsc.imx: mfw_ipucsc: IPU-based video converter
v4lsrc.imx: mfw_v4lsrc: v4l2 based camera src
mpeg2dec.imx: mfw_mpeg2decoder: mpeg2 video decoder
amrdec.imx: mfw_amrdecoder: amr audio decoder
aacdec.imx: mfw_aacdecoder: aac audio decoder
v4lsink.imx: mfw_v4lsink: v4l2 video sink
mpeg4dec.imx: mfw_mpeg4aspdecoder: mpeg4 video decoder
audiopeq.imx: mfw_audio_pp: audio post equalizer
h264.imx: mfw_h264decoder: h264 video decoder
isink.imx: mfw_isink: IPU-based video sink
mp3enc.imx: mfw_mp3encoder: mp3 audio encoder
vorbisdec.imx: mfw_vorbisdecoder: vorbis audio decoder
Here you go:
root@nitrogen6x:~# gst-inspect | grep mfw
ipucsc.imx: mfw_ipucsc: IPU-based video converter
v4lsrc.imx: mfw_v4lsrc: v4l2 based camera src
h264.imx: mfw_h264decoder: h264 video decoder
mp3enc.imx: mfw_mp3encoder: mp3 audio encoder
isink.imx: mfw_isink: IPU-based video sink
amrdec.imx: mfw_amrdecoder: amr audio decoder
v4lsink.imx: mfw_v4lsink: v4l2 video sink
audiopeq.imx: mfw_audio_pp: audio post equalizer
mpeg2dec.imx: mfw_mpeg2decoder: mpeg2 video decoder
mpeg4dec.imx: mfw_mpeg4aspdecoder: mpeg4 video decoder
I tried using mfw_v4lsink and mfw_mpeg4aspdecoder in various combinations but no luck.
Ralph
This link could be helpful for your firmware ERR.
Error in opening firmware binary file for i.mx6 solo wandboard