How to increase the number of inputBuffer for MediaCodec on Android4.4.3

[yuuki]

Dear all,

We run Android4.4.3 with MX6Q-SABRE-SDB board.

(This Android4.4.3 is IMX6_KK443_200_ANDROID_SOURCE_BSP of NXP.)

And, Extended Feature Packages(omx player package) are implemented.

On this system, We want to use Android MediaCodec API Demo.

https://github.com/vecio/MediaCodecDemo

This MediaCodec seems to use only one InputBuffer for the decode.

Can we increase the number of inputBuffer?

If it is possible, would you tell me the method (procedure)?

In the following cord.

==========================================================

ByteBuffer[] inputBuffers = decoder.getInputBuffers();

ByteBuffer[] outputBuffers = decoder.getOutputBuffers();

BufferInfo info = new BufferInfo();

boolean isEOS = false;

long startMs = System.currentTimeMillis();

while (!Thread.interrupted()) {

              if (!isEOS) {

                            int inIndex = decoder.dequeueInputBuffer(100000);

==========================================================

When the dequeueInputBuffer function was called, value "0" is returned to Index, and this processing takes an about 20msec.

It seems to be caused by the fact that this inputBuffer is SingleBuffer.

We want to shorten the processing time by increasing InputBuffer.

May I have advice?

Best Regards,

Yuuki

[subashkumar]

Thank you. We will try out the patches and  Grafika ​and come back in case of any clarification.

[subashkumar]

Hi,

We are facing issues in enabling the hardware acceleration with stagefright. As per the above discussion, Freescale uses fsl_vpu_omx to interface hardware codec with libstagefright. Is it true that we need to just integrate the fsl_vpu_omx patches to enable the stagefright HW acceleration, or do we need to additional work to achieve this?

Best Regards,

Subash kumar.

[gurtajs4]

Hi Subhash,

Are you using the Android 4.4.3 Repo from freescale? If that's the case, then fsl_vpu_omx should already been patched.

look for folder into $ANDROID/external/fsl_vpu_omx. You can run mm command to confirm if it build.

Regards,

Gurtaj

[subashkumar]

Thanks for your reply. We are using Android 5.0. I will check the patches, but just want to confirm that if this patch is included then we could use HW acceleration with stagefright, right?

[gurtajs4]

Yes, It is included. You should be able to use HW acceleration feature to encode/decode video. The question is what is  the Highest Resolution/FPS you are targeting

-Gurtaj.

[subashkumar]

Hi Gurtaj,

Sorry for the delayed response. Our target is to decode two 1080p videos @30 FPS.

[gurtajs4]

Oh great, You can refer Grafika ​

Double Decode Specially. https://github.com/google/grafika/blob/master/src/com/android/grafika/DoubleDecodeActivity.java

Regards,

Gurtaj

[yuuki]

Dear all,

We found the point that defined the number of InputBuffer.

external/fsl_imx_omx/OpenMAXIL/src/component/vpu_dec_v2/VpuDecComponent.cpp

===============================================================

#define DEFAULT_BUF_IN_CNT                                    0x1

#define DEFAULT_BUF_IN_SIZE                     (1024*1024)

#define DEFAULT_BUF_OUT_POST_ZEROCNT (0)

#define DEFAULT_BUF_OUT_POST_CNT        (5)          //5 will bring better performance than 4

#define DEFAULT_BUF_OUT_DEC_CNT          (3)

===============================================================

We tried the change of the value of DEFAULT_BUF_IN_CNT.

However, it was not improved.

Our problem is that 1080p60fps video contents are played back at about 50fps if MediaCodec was used.

Which part of MediaCodec should we confirm?

May I have advice?

Best Regards,

Yuuki

[yifangguo-b4310]

Hi,

Do you try to use "external\fsl_vpu_omx"  codec  package instead of "external/fsl_imx_omx"  package ?

[gurtajs4]

HI Everyone,

We are facing the similar issue and see that the input buffers are allocates using the useBuffer

Cross Reference: /frameworks/av/media/libstagefright/ACodec.cpp  which is not allocating on hardware side.

Is someone already have achieved the solution, Can someone help me on increasing the Input Buffers.

Regards,

Gurtaj

[jamesbone]

Hello Gurtaj,

Which package codec are you using?  AS you can see from the thread we are recommending to use the "external\fsl_vpu_omx"   which platform are you using and what version of BSP?

[gurtajs4]

Hi James,

We are using Android L5.1.1_2.1.0 GA. We have Android Multimedia Extended Codec as a part of BSP.

I was able to increase the buffer from DEFAULT_BUF_IN_CNT  in VpuDecComponent.cpp.

Currently we are facing the issue with 60FPS video playback using Mediacodec API. The same used to work in Android JB4.2.2_1.0.0-ga.

We are trying to isolate the problem and in the process we see decoding or rendering could be slow.

Can you help me understanding the mechanism of how decoded data is rendered onto the surfaceflinger after MediaCodec.releaseOutputBuffer(true) is called.

Also, can you speculate on what exactly was changed in between JB4.2.2_1.0.0-ga. and next version with reference to rendering.

Thanks & Regards,

Gurtaj

[yifangguo-b4310]

Hi,

  This is about  OpenMax VPU decoder  and  surface render component performace , have created JIRA:

http://sw-jira.freescale.net/browse/MMFMWK-7080

[gurtajs4]

Hi,

I am not able to access the link being in company network. So can you shed some light on how the problem can be solved.

Regards,

Gurtaj

[yifangguo-b4310]

Hi gurtajs4@gmail.com,

   Do you mean you are using  Mediacodec API which Android SDK expose too ?

Actually,  Android use  StageFright  as default Multimedia framework.  Fsl's fsl_imx_omx  package use "OMXPlayer" as Multimedia framework.

So, do you test use "external\fsl_vpu_omx" package to test ?

That's mean don't apply Extended Feature Packages(omx player package) .  That's "HAVE_FSL_IMX_CODEC" is "false".

[gurtajs4]

Hi,

I am using MediaCodec API with BSP Android L5.1.1_2.1.0 GA.

We know that the OMXPlayer uses the fsl_imx_omx. Again the striped version of "fsl_imx_omx" is "fsl_vpu_omx".

So in Android, Freescale uses fsl_vpu_omx to interface hardware codec with libstagefright.

Speaking of which, MediaCodec uses fsl_vpu_omx for decoding.

If that correct then what about the Rendering in MediaCodec ??

Does it uses fsl_vpu_omx  as a renderer?? or Is it differently linked with SurefaceFlinger.?

See, the Green Path works with no issue, but the Mediacodec path are having issue with decoding + rendering the frame at 16.6 fps for 1080p60fps video.

I wanted to isolate the issue. Either its in decoder or renderer.

Can you suggest some way here?

Regards,

Gurtaj

[yifangguo-b4310]

Hi Gurtaj Singh,

  I agree with you,  should distinguish the OpenMax VPU component issue or  surface render display issue.

you said "the Green Path works with no issue" , is that mean  if  only use VPU component(no surface render handlle), the vpu decode  speed can reached 60fps ?  if yes,  the issue happend on rendering mechanism.

In "fsl_imx_omx" package, OMX_player use OpenMAX component "OMX.Freescale.std.video_render.surface.sw-based" as render component.

In "fsl_vpu_omx" package, stagefright  use Android libstagefright default render method "AwesomeRenderer".

[gurtajs4]

Hi Yifang Guo,yifangguo-b43107

Yes, if only VPU Component is used (no surface render handlle), the vpu decode  speed can reached upto 65-75fps. This is with 1080p video 25MB bitrate and 60 FPS. But since we are using Android libstagefright default render, the overall output fps achieve ( both including decoding and rendering ) is less than 60. It is around 50 FPS

I am really doubting the Android libstagefright default render method. Can we verify this and isolate or narrow down the issue.

Thanks & Regards,

Gurtaj

[yifangguo-b4310]

Do you test  to use  "fsl_vpu_omx" package,   delete "fsl_imx_omx" package folder ?

[gurtajs4]

Hi,

Ok I see the point.

I don't have  fsl_imx_omx, we only have fsl_vpu_omx

The problem is I am using MediaCodec API and MediaCodec uses VPU for decoding and Android libstagefright default render method.

Decoding is fine, Rendering is slow and hence the overall video playback with MediaCodec is slow ( 1 Minute video takes 1.5 minutes to complete).

I still want to use MedaiCodec API's . There is no other option to decode h264 elementary stream in JAVA.

How can I speed up the rendering.

Regards,

Gurtaj