CTSSkQPTestCases failures on Android P9.0.0_2.2.0 BSP for i.MX6 and SabreSD

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

CTSSkQPTestCases failures on Android P9.0.0_2.2.0 BSP for i.MX6 and SabreSD

Jump to solution
4,150 Views
brood
Contributor III

Greetings NXP Community!

I'm working on porting the Android P9.0.0_2.2.0 BSP to our tablet product that is based on the SabreSD i.MX6Q reference design and I'm running into the following failures in the Android CTS test suite, specifically the tests in the CtsSkQPTestCases module.  This module tests the SKIA graphics library, which to my understanding is tied into the graphics 2D functionality available on the i.MX6Q chip.  I know very little about GPUs/graphics in general, so I'm hoping someone has a solution to address the following failing CtsSkQPTestCases:

1. org.skia.skqp.SkQPRunner#gles_arithmode

2. org.skia.skqp.SkQPRunner#gles_colorwheel

3. org.skia.skqp.SkQPRunner#gles_cross_context_image

4. org.skia.skqp.SkQPRunner#gles_deferred_texture_image_medium_decoded

5. org.skia.skqp.SkQPRunner#gles_deferred_texture_image_medium_decoded_indexed

6. org.skia.skqp.SkQPRunner#gles_deferred_texture_image_medium_encoded

7. org.skia.skqp.SkQPRunner#gles_deferred_texture_image_medium_encoded_indexed

8. org.skia.skqp.SkQPRunner#gles_encode_srgb_png

9. org.skia.skqp.SkQPRunner#gles_flippity

10. org.skia.skqp.SkQPRunner#gles_image_from_yuv_textures

11. org.skia.skqp.SkQPRunner#gles_image_surface

12. org.skia.skqp.SkQPRunner#gles_imagemasksubset

13. org.skia.skqp.SkQPRunner#gles_makecolorspace

14. org.skia.skqp.SkQPRunner#gles_shadermaskfilter_image

15. org.skia.skqp.SkQPRunner#gles_tileimagefilter

16. org.skia.skqp.SkQPRunner#unitTest_GrMeshTest

Both the BSP for the SabreSD i.MX6Q and my tablet product are showing the same test case failures, so it's not a porting issue.  My hunch is a bug in the graphics driver, or other related component.  Poking through the external/skia repository shows that NXP made a change related to a potential issue with the graphics driver (NXP issue reference: MA-14657), so I'm thinking there's an outstanding bug that has yet to be fixed or provided.

Attached are the Android CTS test results for both the BSP on the SabreSD and my tablet product for reference.

Labels (3)
0 Kudos
1 Solution
4,049 Views
IvanRuiz
NXP Employee
NXP Employee

A reply was already made to the corresponding case, let's continue the support there.

View solution in original post

0 Kudos
12 Replies
3,646 Views
pareshchaudhary
Contributor I

Hello brood, 

This is not directly relevant to your thread but I see you have worked for GMS certification on imx series Processor. 

I am planning to use iMX8M Plus in our design with Android 11 OS. In our case, GMS is the final requirement but here I am confused about processor selection.  Can you please provide some input based on your experience regarding GMS with iMX or any other NXP processor? 

 

Thanks,

Paresh

 

 

 

0 Kudos
3,583 Views
brood
Contributor III

Hi pareshchaudhary,

Applying the GMS package to the Android BSP is not terribly difficult, however I'm not at liberty to discuss the internal workings of doing so according to our contracts with Google.  Once you get the Google contracts in place and get access to the GMS "stuff", their documentation is sufficient to effectively integrate the package into your build.

Our greatest challenges with using any of the i.MX processors in a GMS-approved device have been with the GPU, media codec, and camera HAL components.  We've experienced a number of issues with the proprietary GPU integration libraries, which were causing failures in the CtsSKQPTestCases and CtsUIRenderingTestCases modules in the Android CTS test suite.  We've also experienced failures in the CtsMediaTestCases module due to issues with the hardware media codec implementation.  We've managed to get fixes for the proprietary GPU integration libraries from NXP, you will have to open a support ticket with them to get the patch(es).  For the CtsMediaTestCases, it really came down to disabling or adjusting the supported resolutions of the codecs to either make the Android CTS test skip testing the codec, or make the codec effectively unsupported.  While not the best solution, it at least gets us through the tests.

With regards to the camera HAL, there are some failures in the Android CtsCameraTestCases that will fail, however if you mess with the supported resolutions of the camera(s), you can eventually get them to pass.  This will depend on the type of camera that you're using, of course.  Looking into the future with Android 11, NXP has provided a "legacy" HAL implementation, which is not allowed by the latest Android 11 requirements.  I have no idea what a "Full" HAL implementation looks like, nor what it will take to put on in place that will pass all of the tests.  But, this is likely to be the biggest challenge using a NXP part.

I can't say which NXP part is right for you, it really comes down to what type of device you're designing.  Personally, I generally want the most "processing power" for our application, as it buys us the most useful time the product can be used.

0 Kudos
3,783 Views
brood
Contributor III

Hi everyone,

I just wanted to update those who may be interested in this issue, that you may also have some issues passing the CtsUiRenderingTestCases module as well as the CtsSkQPTestCases module that was mentioned in this post.  I have been able to get a patch to the OpenGL/GPU 'binary blobs' provided by the BSP via support ticket #00322122 that have addressed all failures in these two modules.  Since these are proprietary libraries provided in binary format by NXP, I'm pretty sure I cannot provide the patch here, however I would recommend opening your own support ticket and referencing my support ticket #00322122, which contains the appropriate patch needed to pass the tests in both modules.

Also, for those looking to pass all of the Android CTS media test cases for the P9.0.0_2.2.0 BSP, I would recommend disabling the VP9 codec as it cannot pass the tests in its current state.

0 Kudos
3,777 Views
amsaleem
Contributor II

thank you for the update.

We find few other media specific CTS case failures as well. Do you get the same? Listed below are some of them.

android.media.cts.DecodeAccuracyTest#testGLViewDecodeAccuracy[12] fail junit.framework.AssertionFailedError:
With the best matched border crop
(0.0, 0.0), greatest pixel difference is
183 at (462, 465) which is over the
allowed difference 90
android.media.cts.DecodeAccuracyTest#testGLViewDecodeAccuracy[28] fail junit.framework.AssertionFailedError:
With the best matched border crop
(0.0, 0.0), greatest pixel difference is
183 at (461, 450) which is over the
allowed difference 90
android.media.cts.DecodeAccuracyTest#testGLViewDecodeAccuracy[44] fail junit.framework.AssertionFailedError:
With the best matched border crop
(0.125, 0.0), greatest pixel difference
is 184 at (459, 467) which is over the
allowed difference 90
android.media.cts.DecodeAccuracyTest#testGLViewDecodeAccuracy[4] fail junit.framework.AssertionFailedError:
With the best matched border crop
(0.0, 0.0), greatest pixel difference is
226 at (467, 454) which is over the
allowed difference 90

 

 

0 Kudos
3,767 Views
brood
Contributor III

Hi ansaleem,

If I remember correctly, these tests are failing due to issues with the VP9 codec.  I believe if you reduce the supported resolution (so these tests are skipped), or disable the codec altogether, will allow them to pass.

3,684 Views
amsaleem
Contributor II

it worked!. thank you. Two more cases are failing in CtsMediaTestCase module for us. 

1. android.media.cts.DecoderTest#testH264ColorAspects

2. android.media.cts.DecoderTest#testMPEG2ColorAspectsTV

Please let me know if you have any solution for it.

0 Kudos
3,574 Views
brood
Contributor III

Hi @amsaleem ,

We managed to find a workaround for the two CtsMediaTestCases you have mentioned.  If you set the minimum supported resolution of the H264 and MPEG2 hardware codecs to 320x240 (QGVA), the CTS tests will effectively skip testing the hardware codecs because the file it is using to test them is at a resolution of 176x144.  Since the file has a lower resolution than the lowest supported by the codecs, the tests will skip over testing the codecs and now pass.

0 Kudos
3,765 Views
amsaleem
Contributor II

thank you. I will try it.

0 Kudos
4,050 Views
IvanRuiz
NXP Employee
NXP Employee

A reply was already made to the corresponding case, let's continue the support there.

0 Kudos
4,017 Views
amsaleem
Contributor II

We have the same issue. I am trying to access the solution and not able to access it. Could you please help? Thank you

0 Kudos
3,996 Views
brood
Contributor III

Greetings!

I had to submit a support ticket to NXP via their portal at nxp.com/support

If you are having the same issue, I would recommend submitting a support ticket this way and mentioning that a fix for this exact issue has been provided in case #00317323.

I would normally post the appropriate patch if I discovered the solution on my own, but since the fix was supplied by NXP I'm not sure if I'm able to share it since it affects the OpenGL/GPU 'binary blobs' supplied in the BSP.

0 Kudos
3,986 Views
amsaleem
Contributor II

thank you

0 Kudos