i.MX6Quad 1080p60 H.264 decode/playback support?

[brettkuehner]

We've been doing some testing with the Nitrogen6 board, with the LTIB build, and we are seeing some problems with the 1080p60 decode/render support via gstreamer. Has anyone gotten this to work with a stable 60fps frame rate? We are seeing frame rates averaging 50-55fps, with dips as low at 40fps.

Various Freescale documents claim 1080p60 decode support, for example:

http://cache.freescale.com/files/32bit/doc/fact_sheet/IMX6SRSFS.pdf

but the release notes in the most recent IMX_MMCODEC_3.0.35_4.1.0_BUNDLE_CODA list the max frame rate for H.264 1920x1080 as 30fps

However, that is in section 5.1, which says it is for "i.MX 6Dual/6Quad and i.MX 6Solo/6DualLite", so it is hard to tell if that really applies to the 6Quad.

We've seen other (usually technical) documents that similarly seem to indicate 30fps as the max, but those seem to conflict with the (usually marketing) documents that specifically say 1080p60.

Does anyone have a clear answer one way or the other? Or is anyone having similar problems?

[david_kosir]

We are in process of choosing SoC, can you guys confirm that it works nicely (or doesn't work well) with 1080p60 decoding, using Gstreamer?

Its quite big decision and if Freescale lied about that in docs, that would be huge turn-off from us.

[LeonardoSandova]

Hi David. It is true. Test it yourself

sudo cp video.mp4 /dev/shm/tmp_video
time gst-launch filesrc location=/dev/shm/tmp_video typefind=true ! aiurdemux ! vpudec output-format=4  framedrop=false !  queue  max-size-buffers=2 ! mfw_v4lsink sync=false

Check the log.

[david_kosir]

Leonardo, that doesn't look like good example. Copying file to RAM and then using playback with video sink option sync=false isn't near production setup where you have network stream that needs to be received and demuxed. So, I guess 1080p60 is on the edge of possible. But, it probably depends on video itself, codec and bitrate.

[LeonardoSandova]

Hi David, You are right, this scenario is not for production. I will escalate the issue to a specialist.

Leo

[brettkuehner]

I'll post our results when we have a chance to test 1080p60 with gstreamer and do some analysis. But for the Freescale guys out there, I think Freescale should give a clear answer on this, since there is conflicting information on the issue and it is very important to have a definitive statement saying it is/isn't officially supported, and what the requirements are for it.

[david_kosir]

Brett, have you tested it?

[brettkuehner]

No, sorry. We're focused on working with 720p and 1080i right now.

I did read the VPU doc that gets into fair detail on the capabilities and the direct VPU API, and once again it mentions 30 fps as a max:

Performance

• All video decoder standards up to 1920x1088 @ 30 fps at 266 MHz

• H264 encoder standards up to 1920x1088 @ 30 fps at 266 MHz, MPEG4 encoder up to 720p@30fps at 266MHz

• MJPG decoder on 4:4:4 supports 120M pixel per second @ 266MHz

• MJPG encoder on 4:4:4 supports 160M pixel per second @ 266MHz

(from http://hands.com/~lkcl/eoma/iMX6/VPU_API_RM_L3.0.35_1.1.0.pdf )

That's why I wish Freescale would make a clear statement here on officially supported p60 resolutions.

[david_kosir]

Thanks for your update! Are 720p50 and 1080i25 smooth at least, or you need

to tweak libs for them? If that is the case that it is really bad

situation...

On Dec 6, 2013 1:32 AM, "Brett Kuehner" <admin@community.freescale.com>

[brettkuehner]

We haven't tried those framerates at all, sorry. We do run 720p60 OK, though we are running at the higher (352MHz) VPUCLK.

[brettkuehner]

Note: We are using the Quad (on the Nitrogen board).

Our playback pipeline for 720p60 from a file is:

gst-launch filesrc location=file.ts typefind=true ! mpegtsdemux ! vpudec framerate-nu=60000 framerate-de=1001 framedrop=false ! mfw_v4lsink device=/dev/video17

In our setup, /dev/video17 is HDMI, you may need to change that for your setup.

We saw some slow frame rates when running from NFS, so make sure your test files are on the local SD card.

Also, we found that for some files/sizes/framerates that aiurdemux worked better than mpegtsdemux

[stormhunter]

Hi Brett/Stephen,

Have you (or anyone?) managed to get 1080p60 decoding running on the iMx6D or Q? Under what conditions (simply raising the VPUCLK to 352Mhz as discussed here and adding a heatsink)?

We are have just selected the Imx6 Dual based on all the marketing documentation stating 1080p60 decode. Is this spec a  false claim from Freescale?

[joshkurland]

Huh, never noticed that before, but I see the same thing when I test with a 1080p@59.94 MPEG2.  I am running a wandboard-quad with VPUCLK set to 352MHz (just learned I can do that today!).  I will see if I can hook it up to a scope tomorrow and measure the actual output, maybe its just a gstreamer reporting problem?

[brettkuehner]

We have 720p60 going, and we have heard verbally from an FAE that p60 is definitely supported (though it took a while to get through to someone who would confirm that).

We are using the higher-frequency VPU setting in the kernel, but I'm not certain if that is absolutely required. It does take some fiddling with gstreamer pipelines to get things right. Also, we have not yet validated that the playback from file is precisely 60fps, we sometimes see gstreamer reporting slightly lower fps, but we aren't sure that is meaningful (the fps calculations it is doing may include some setup and teardown that we don't care about).

We do see stable 720p60 when we decode and display incoming RTP network streams.

We also have had success getting 1080p60 from an RTP stream, but haven't tested that as thoroughly.

[stormhunter]

We had updated information today from Freescale though our distributor that 1080p60 is only supported on bare metal (not with OS running)... If anyone has contrary evidence to this it would be good to hear.

Interesting that you have had some success with 1080p60 from RTP... are you running the output to a TV through HDMI? - the TV should be quite fussy on frame rate/timing.

We have a Sabre Platform board arriving soon for evaluation.

[LeonardoSandova]

Aran, can you do what I have suggested?

[LeonardoSandova]

Hi Brett, I am not a VPU/IPU expert but could you post the playback pipeline. How are you measuring the FPS? Also, it may interesting to raise the VPU frequency to VPU=352MHz to check if performance is reached.

[stevengreenberg]

Hi Leonardo-

I'm an engineer working with Brett on this testing. We're actually not even testing 1080p60 yet, as that's not required, we're just trying to get 720p60 running.

We're using a very simple pipeline to play back the file:

gst-launch --gst-debug=vpudec:3  filesrc location=720p60test.mp4   typefind=true !  aiurdemux ! vpudec framedrop=false ! mfw_v4lsink

Part of the resulting log output is as follows:

Running time 0:00:35.102510005 render fps 46.692

Setting pipeline to READY ...

0:00:35.701499338  2546    0x17050 INFO                  vpudec vpudec.c:577:vpudec_core_deinit: Stat:

        in  : 2046

        out : 2041

        show: 2041

Setting pipeline to NULL ...

Total rendered:1639

The expected framerate is 59.94 - as you can see, the rendered framerate is lower than expected. The value shown is consistent with the fact that only 1639 of 2041 frames were actually rendered.

This total rendered & framerate problem is inconsistent. On occasion we've seen values very close to what's expected, but generally the values are much lower, as shown here.

We haven't tested with the increased VPU clock, though we are about to do that.

[LeonardoSandova]

Can you check if there is some difference if the output-format value is change on the vpudec element? output-format=4? Check below:

sudo cp <YOUR MEDIA FILE> /dev/shm/

gst-launch filesrc location=/dev/shm/<YOUR MEDIA FILE> typefind=true ! aiurdemux ! vpudec output-format=4  framedrop=false !  queue  max-size-buffers=2 ! mfw_v4lsink sync=false

Leo