Benjamin Rood

i.MX6Q H.264 video codecs fail GtsExoPlayerTestCases

Discussion created by Benjamin Rood on Oct 17, 2018
Latest reply on Oct 31, 2018 by jimmychan

I currently have a Google-certified tablet product based on the i.MX6Q applications processor and the N7.1.1 Android BSP.  To become Google GMS certified, we had to disable the NXP hardware codecs (specifically: OMX.Freescale.std.video_decoder.avc.v3.hw-based) because it was unable to pass some GtsExoPlayerTestCases (more on which ones later).

 

I have loaded the Oreo BSP (imx-o8.0.0_1.0.0_ga) onto the SABRESD platform and run the GtsExoPlayerTestCases against it to see if a new BSP would fix things and allow us to use the hardware-accelerated media codecs.  It did not, and this BSP has the same issues as the N7.1.1 BSP.

 

The following GtsExoPlayerTestCases fail using i.MX6 hardware-accelerated codecs on the SABRESD i.MX6 development platform (as well as our custom board):

 

1. com.google.android.exoplayer.gts.DashTest#testH264Adaptive

2. com.google.android.exoplayer.gts.DashTest#testH264AdaptiveWithRendererDisabling

3. com.google.android.exoplayer.gts.DashTest#testH264AdaptiveWithSeeking

 

Now, I've been able to get the following tests to pass by applying the timestamp manager patch (attached) AND enabling the VPU_SEEK_ANYPOINT_WORKAROUND flag in fsl_imx_omx/OpenMAXIL/src/component/vpu_wrapper_vpu_wrapper.c (patch attached):

 

1. com.google.android.exoplayer.gts.DashTest#testH264Adaptive

2. com.google.android.exoplayer.gts.DashTest#testH264AdaptiveWithSeeking

 

Which leaves the following GtsExoPlayerTestCases still failing:

 

1. com.google.android.exoplayer.gts.DashTest#testH264AdaptiveWithRendererDisabling

 

I need to use the hardware-accelerated codecs because using software-only codecs results in a choppy user experience, even with the quad-core i.MX6Q.  This is not acceptable by our customers, nor is not being Google GMS certified.

 

A few of my hardware engineers have talked to a NXP representative, Patrick Stilwell (patrick.stilwell@nxp.com), who recommends posting on this forum for support.  Patrick also mentioned that NXP will support a BSP for 1 year after GA release.  Since the Oreo BSP was released earlier this year (~February 2018), it is still within the supposed timeframe to request support.

 

NXP - please help fix this particular issue since it is an issue in the software provided in the BSP, and has been an issue since at least the N7.1.1 BSP.

 

I've run the failing test and captured a bugreport, which includes some verbose output from the codec layer.  The bugreport is attached to this post.

 

EDIT 10/22/18: The GTS failure listed above (com.google.android.exoplayer.gts.DashTest#testH264AdaptiveWithRendererDisabling) can be re-created without access to the GTS test suite by downloading the ExoPlayer repository from GitHub (GitHub - google/ExoPlayer: An extensible media player for Android ) and running the DashStreamTest#testH264AdaptiveWithRendererDisabling unit test found in 'playbacktests' library module.

 

EDIT 10/23/18: I've performed some additional digging around and have a theory about the failure.  The specific GTS/ExoPlayer test case that is failing performs the following sequence:

1. Setup Codec

2. Start playing the stream

3. Enable/disable renderer

4. Attach "dummy" Surface object via a call to setSurface() (ultimately calls handleSetSurface() in ACodec.cpp)

5. Attach "new" Surface object via a call to setSurface() <-- This is where the error happens and the test freezes

 

From my research, it appears that the process of attaching a new surface object via a call to setSurface() will re-assign any allocated buffers from the Surface object that was previously used to the new one, however this doesn't seem to be working correctly with the "OMX.Freescale.std.video_decoder.avc.v3.hw-based" codec since the buffers returned from the codec after the error condition are considered "unregistered" by ACodec.cpp and are ultimately discarded, which results in the test freezing, timing out, and failing.  This post gives some additional context, however not much.  Other people have reported having the same issues using the i.MX6 series processors (see this post specifically).

 

Tagging Ming Zhou in this post since they kindly provided the timestamp manager patch in another post that fixed one of the issues listed above.  Ming Zhou - any thoughts on this issue?

Outcomes