How to increase the number of inputBuffer for MediaCodec on Android4.4.3

cancel
Showing results for 
Search instead for 
Did you mean: 

How to increase the number of inputBuffer for MediaCodec on Android4.4.3

5,711 Views
yuuki
Senior Contributor I

Dear all,

We run Android4.4.3 with MX6Q-SABRE-SDB board.

(This Android4.4.3 is IMX6_KK443_200_ANDROID_SOURCE_BSP of NXP.)

And, Extended Feature Packages(omx player package) are implemented.

On this system, We want to use Android MediaCodec API Demo.

https://github.com/vecio/MediaCodecDemo

This MediaCodec seems to use only one InputBuffer for the decode.

Can we increase the number of inputBuffer?

If it is possible, would you tell me the method (procedure)?

In the following cord.

==========================================================

ByteBuffer[] inputBuffers = decoder.getInputBuffers();

ByteBuffer[] outputBuffers = decoder.getOutputBuffers();

BufferInfo info = new BufferInfo();

boolean isEOS = false;

long startMs = System.currentTimeMillis();

while (!Thread.interrupted()) {

              if (!isEOS) {

                            int inIndex = decoder.dequeueInputBuffer(100000);

==========================================================

When the dequeueInputBuffer function was called, value "0" is returned to Index, and this processing takes an about 20msec.

It seems to be caused by the fact that this inputBuffer is SingleBuffer.

We want to shorten the processing time by increasing InputBuffer.

May I have advice?

Best Regards,

Yuuki

Labels (3)
34 Replies

723 Views
yifangguo-b4310
NXP Employee
NXP Employee

Hi Gurtaj Singh,

   "fsl_vpu_omx " package is only a  OpenMax plugin worked for Android Multimedia stagefright.

   Do you confirm that  you  are using fsl_vpu_omx to try , not    fsl_imx_omx at present ?

if yes,   I suggest  to apply " Extended Feature Packages(omx player package)-fsl_imx_omx"  to test   MediaCodec API demo.  Because fsl_imx_omx package implement another Multimedia framework "omx_player"   instead of "stagefright" .

So, I mean you can test your MediaCodec API demo on two different Multimedia frameworks :

1)  On Android  default Multimedia framework---stagefright, it utilizes OpenMax VPU component plugin in  "fsl_vpu_omx" package,

2)  Or FSL Multimedia framework---omx_player,  it is implemented by FSL instead of "stagefright" in "fsl_imx_omx" package. In "fsl_imx_omx" package, OMX_player use OpenMAX component "OMX.Freescale.std.video_render.surface.sw-based" as render component.

723 Views
gurtajs4
Contributor II

Hi,

Sir, If I apply fsl_imx_omx patches, then the changes effect only mediaplayerservice and the Gallery will by default start using OMXPlayer instead of Awesome/NuPlayer.

But as you can see in the picture above, we wanted to use MediaCodec API to decode raw h264.

And at no point from the above diagram I see MediaCodec using Freescale renderer, it always uses Awesome renderer and which could be the issue.

Is my understanding correct???

Regards,

Gurtaj

0 Kudos

723 Views
yifangguo-b4310
NXP Employee
NXP Employee

If   apply fsl_imx_omx patches, then the changes effect only mediaplayerservice and the Gallery will by default start using OMXPlayer instead of Awesome/NuPlayer.

[A] Yes , it's right.

Can you confirm that   your  "MediaCodec API"  can call in to "OMXPlayer"  flow or not ?

As "i.MX_Android_Extended_Codec_Release_Notes.pdf" spec,

Untitled.png

Android  Codec spec of H.264 1080p decoder  can only guarantee the MAX framerate 30fps . Don't guarantee more than 30fps  in Android System expect  an baremetal(no os) demo.

0 Kudos

723 Views
gurtajs4
Contributor II

Hi yifangguo-b43107

I am using the Freescale Android L5.1.1_2.1.0 GA, for our development.

""Can you confirm that   your  "MediaCodec API"  can call in to "OMXPlayer"  flow or not ?""

[Gurtaj] Can you verify as my knowledge is scare with respect to OpenMAX.

"Android  Codec spec of H.264 1080p decoder  can only guarantee the MAX framerate 30fps . Don't guarantee more than 30fps  in Android System expect  an baremetal(no os) demo."

[Gurtaj] I know that the spec says no guarantee for more than 30FPS, Can you answer the below questions

A. Have a look here Slow H264 1080P@60fps Decoding on Android Lollipop 5.0.2 SabreSD .

B. How can JB was able to decode and render 60FPS video and Lollipop or Kitkat don't.

C. Why there is no effect in the OMXPlayer or GMPlayer ( the green path in above sample) and they are able to decode the given video, But only the MediaCodec Path got effected?

Looking forward to a great discussion and logical answers to above question.

Regards,

Gurtaj

0 Kudos

723 Views
yifangguo-b4310
NXP Employee
NXP Employee

Hi,

   OMXPlayer (GMPlayer) is implemented by FSL as Media player framework.  It use  openMAX  VPU decoder component, and re-construct render method in  "OMX.Freescale.std.video_render.surface.sw-based" component although send to surfaceflinger too.  So,  OMXPlayer 's play frame rate maybe can reach to about 60fps under some low bitrate use case.

   But Media Codec API    use stagefright  multimedia framework ,  can you mask the stagefright render code, don't display video frame, just only use  VPU component decode( or save every decoded frame in FS by write an opened file), then test the frame rate.  If it can reach to 60fsp, so the issue is in  Android stagefright render part.

0 Kudos

723 Views
gurtajs4
Contributor II

Hi yifangguo-b43107

Thanks for your explanation and keeping patience with me so far.

Yes, We wanted to use MediaCodec to only Decode (just only use  VPU component decode and no rendering). Later save the decoded YUV frames to a file.

As you can see from the starting of the thread, I am asking for help to do this approach

Media Player Suggested Approach.jpg

Can you help me with best way and some test code to do

1. The YUV to RGB color conversion.

2. Render the converted image to framebuffer ( Zero Copy Approach ??).

If, i have a test code to perform the two methods in the minimum time, let say 2-3 millisecond. Then we will be able to narrow down and isolate the issue.

Regards,

Gurtaj

0 Kudos

722 Views
yifangguo-b4310
NXP Employee
NXP Employee

Hi,

    The approach  of YUV  data  rendered  to NativeWindow  is a standard way in Android.  But in your test,  the render method can't reached to 60fps.

As your diagram,  If you want to high performace of render,  suggest the decoded YUV data is transferred to NativeWindow API directly  in Android Frameworks(Native development ), don't export YUV data to  Jave  space, that is  inefficient.

0 Kudos

721 Views
gurtajs4
Contributor II

Hi,

On

"As your diagram,  If you want to high performace of render,  suggest the decoded YUV data is transferred to NativeWindow API directly  in Android Frameworks(Native development ), don't export YUV data to  Jave  space, that is  inefficient."

Can you please share some sample code to do this.

Regards,

Gurtaj

0 Kudos

721 Views
yifangguo-b4310
NXP Employee
NXP Employee

Hi,

   There is no special sample code for your  this case,  but "external\fsl_imx_omx\OpenMAXIL\src\component\surface_render\SurfaceRender.cpp" is using Native Window API for render, maybe you can have a reference.

0 Kudos

721 Views
gurtajs4
Contributor II

Hi,

Thanks for the information.

Do you have some Internal document or something which can be used to understand the fsl_imx_omx Architecture or design

Also, Can quickly understand how SurfaceRender.cpp works.

Some sequence,flow diagram to start with.

Also, Don't you think this is a serious problem, as we were able to achieve 60FPS decoding+rendering in JeallyBean and not in Lollipop ???

Regards,

Gurtaj

0 Kudos

721 Views
yifangguo-b4310
NXP Employee
NXP Employee

No, there is no internal document to introduce this, i need them too if you got.

This maybe Android render issue,  we can't cover it too much.  If you need some customized software, you could apply for Software Service by marketing which is paid servcie.

Thanks.

0 Kudos

721 Views
gurtajs4
Contributor II

Hi,

We do share premium account with freescale.

Jian Li's team is into this. But on a parallel activity I wanted to confirm, if rendering is a issue by not using MediaCodec Rendering.

Can you give us some suggestion here.

Regards,

Gurtaj

0 Kudos

721 Views
yifangguo-b4310
NXP Employee
NXP Employee

At present, no more.

0 Kudos

721 Views
gurtajs4
Contributor II

yifangguo-b43107​,

Thanks for your valuable help here, I will pursue the path as you suggested and will let you know if there is any update.

Regards,

Gurtaj

0 Kudos