Dear all,
We run Android4.4.3 with MX6Q-SABRE-SDB board.
(This Android4.4.3 is IMX6_KK443_200_ANDROID_SOURCE_BSP of NXP.)
And, Extended Feature Packages(omx player package) are implemented.
On this system, We want to use Android MediaCodec API Demo.
https://github.com/vecio/MediaCodecDemo
This MediaCodec seems to use only one InputBuffer for the decode.
Can we increase the number of inputBuffer?
If it is possible, would you tell me the method (procedure)?
In the following cord.
==========================================================
ByteBuffer[] inputBuffers = decoder.getInputBuffers();
ByteBuffer[] outputBuffers = decoder.getOutputBuffers();
BufferInfo info = new BufferInfo();
boolean isEOS = false;
long startMs = System.currentTimeMillis();
while (!Thread.interrupted()) {
if (!isEOS) {
int inIndex = decoder.dequeueInputBuffer(100000);
==========================================================
When the dequeueInputBuffer function was called, value "0" is returned to Index, and this processing takes an about 20msec.
It seems to be caused by the fact that this inputBuffer is SingleBuffer.
We want to shorten the processing time by increasing InputBuffer.
May I have advice?
Best Regards,
Yuuki
Hi Gurtaj Singh,
"fsl_vpu_omx " package is only a OpenMax plugin worked for Android Multimedia stagefright.
Do you confirm that you are using fsl_vpu_omx to try , not fsl_imx_omx at present ?
if yes, I suggest to apply " Extended Feature Packages(omx player package)-fsl_imx_omx" to test MediaCodec API demo. Because fsl_imx_omx package implement another Multimedia framework "omx_player" instead of "stagefright" .
So, I mean you can test your MediaCodec API demo on two different Multimedia frameworks :
1) On Android default Multimedia framework---stagefright, it utilizes OpenMax VPU component plugin in "fsl_vpu_omx" package,
2) Or FSL Multimedia framework---omx_player, it is implemented by FSL instead of "stagefright" in "fsl_imx_omx" package. In "fsl_imx_omx" package, OMX_player use OpenMAX component "OMX.Freescale.std.video_render.surface.sw-based" as render component.
Hi,
Sir, If I apply fsl_imx_omx patches, then the changes effect only mediaplayerservice and the Gallery will by default start using OMXPlayer instead of Awesome/NuPlayer.
But as you can see in the picture above, we wanted to use MediaCodec API to decode raw h264.
And at no point from the above diagram I see MediaCodec using Freescale renderer, it always uses Awesome renderer and which could be the issue.
Is my understanding correct???
Regards,
Gurtaj
If apply fsl_imx_omx patches, then the changes effect only mediaplayerservice and the Gallery will by default start using OMXPlayer instead of Awesome/NuPlayer.
[A] Yes , it's right.
Can you confirm that your "MediaCodec API" can call in to "OMXPlayer" flow or not ?
As "i.MX_Android_Extended_Codec_Release_Notes.pdf" spec,
Android Codec spec of H.264 1080p decoder can only guarantee the MAX framerate 30fps . Don't guarantee more than 30fps in Android System expect an baremetal(no os) demo.
I am using the Freescale Android L5.1.1_2.1.0 GA, for our development.
""Can you confirm that your "MediaCodec API" can call in to "OMXPlayer" flow or not ?""
[Gurtaj] Can you verify as my knowledge is scare with respect to OpenMAX.
"Android Codec spec of H.264 1080p decoder can only guarantee the MAX framerate 30fps . Don't guarantee more than 30fps in Android System expect an baremetal(no os) demo."
[Gurtaj] I know that the spec says no guarantee for more than 30FPS, Can you answer the below questions
A. Have a look here Slow H264 1080P@60fps Decoding on Android Lollipop 5.0.2 SabreSD .
B. How can JB was able to decode and render 60FPS video and Lollipop or Kitkat don't.
C. Why there is no effect in the OMXPlayer or GMPlayer ( the green path in above sample) and they are able to decode the given video, But only the MediaCodec Path got effected?
Looking forward to a great discussion and logical answers to above question.
Regards,
Gurtaj
Hi,
OMXPlayer (GMPlayer) is implemented by FSL as Media player framework. It use openMAX VPU decoder component, and re-construct render method in "OMX.Freescale.std.video_render.surface.sw-based" component although send to surfaceflinger too. So, OMXPlayer 's play frame rate maybe can reach to about 60fps under some low bitrate use case.
But Media Codec API use stagefright multimedia framework , can you mask the stagefright render code, don't display video frame, just only use VPU component decode( or save every decoded frame in FS by write an opened file), then test the frame rate. If it can reach to 60fsp, so the issue is in Android stagefright render part.
Thanks for your explanation and keeping patience with me so far.
Yes, We wanted to use MediaCodec to only Decode (just only use VPU component decode and no rendering). Later save the decoded YUV frames to a file.
As you can see from the starting of the thread, I am asking for help to do this approach
Can you help me with best way and some test code to do
1. The YUV to RGB color conversion.
2. Render the converted image to framebuffer ( Zero Copy Approach ??).
If, i have a test code to perform the two methods in the minimum time, let say 2-3 millisecond. Then we will be able to narrow down and isolate the issue.
Regards,
Gurtaj
Hi,
The approach of YUV data rendered to NativeWindow is a standard way in Android. But in your test, the render method can't reached to 60fps.
As your diagram, If you want to high performace of render, suggest the decoded YUV data is transferred to NativeWindow API directly in Android Frameworks(Native development ), don't export YUV data to Jave space, that is inefficient.
Hi,
On
"As your diagram, If you want to high performace of render, suggest the decoded YUV data is transferred to NativeWindow API directly in Android Frameworks(Native development ), don't export YUV data to Jave space, that is inefficient."
Can you please share some sample code to do this.
Regards,
Gurtaj
Hi,
There is no special sample code for your this case, but "external\fsl_imx_omx\OpenMAXIL\src\component\surface_render\SurfaceRender.cpp" is using Native Window API for render, maybe you can have a reference.
Hi,
Thanks for the information.
Do you have some Internal document or something which can be used to understand the fsl_imx_omx Architecture or design
Also, Can quickly understand how SurfaceRender.cpp works.
Some sequence,flow diagram to start with.
Also, Don't you think this is a serious problem, as we were able to achieve 60FPS decoding+rendering in JeallyBean and not in Lollipop ???
Regards,
Gurtaj
No, there is no internal document to introduce this, i need them too if you got.
This maybe Android render issue, we can't cover it too much. If you need some customized software, you could apply for Software Service by marketing which is paid servcie.
Thanks.
Hi,
We do share premium account with freescale.
Jian Li's team is into this. But on a parallel activity I wanted to confirm, if rendering is a issue by not using MediaCodec Rendering.
Can you give us some suggestion here.
Regards,
Gurtaj