Frame drop at decoding video and scaling

[teranishi_kenji]

Hi all,

I am trying a decoding of H.264 video on i.MX6Q Sabre board using Freescale Yocto Project BSP code (imx-3.14.52-1.1.0_ga).

But, some of the frames are dropped when I try decoding and scaling.

[Command]

> gst-launch-1.0 filesrc location=test_640x480.avi typefind=true ! video/x-msvideo ! aiurdemux ! vpudec ! videoscale method=1 ! video/x-raw,width=800,height=480 ! overlaysink

[Error Message]

> WARNING: from element /GstPipeline:pipeline0/GstOverlaySink:overlaysink0: A lot of buffers are being dropped.

> Additional debug info:

build/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/gstreamer1.0/1.4.5-r0/gstreamer-1.4.5/libs/gst/base/gstbasesink.c(2794): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstOverlaySink:overlaysink0:

> There may be a timestamping problem, or this computer is too slow.

When I try only the decoding and only the scaling, there are not dropped frame.

I think that "vpudec" is hardware-based process and "videoscale" is software-based process.

Therefore, I think that the decoding and the scaling can be parallel processing.

Why some of the frame is dropped?

As additional information, I measure the processing time.

  With decoding     : Scaling time is  61ms / frame

  Only Scaling (*)   : Scaling time is   8ms / frame

  (*) YUV showing (No decoding)

  Command : gst-launch-1.0 filesrc location=test_640x480.yuv ! videoparse format=2 width=640 height=480 framerate=15 ! videoscale method=1 ! video/x-raw,width=800,height=480 ! overlaysink

Why the scaling time is long when decoding and scaling has been processed at the same time?

Best Regards,

Kenji

[Yuri]

Hello,

  Looks like the issue is just a performance issue, mainly because of software “videoscale”.
Please try using one of video conversion plugins (imxvideoconvert_ipu, imxvideoconvert_g2d,
imxvideoconvert_pxp) to resize image.
  Also You may try to insert queue element in Gstreamer pipeline for buffering in order to
improve situation.

Have a great day,
Yuri

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[teranishi_kenji]

Dear Yuri,

Thank you so much for your comment.

I am using the i.MX6Q Sabre board. Therefore, in this case, the software "videoscale" is runing on 1GHz ARM Cortex-A9 processor.

I checked "videoscale" source code, and I think that this processing throughput is not large.

So, I think the performance is enough for "videoscale".

What do you think about this?

BTW, about imxvideoconvert_ipu or imxvideoconvert_g2d, does these plugins have the resize function?

Please tell me a command to resize about these plugin.

I appreciate your cooperation.

Best Regards,

Kenji

[Yuri]

   Please try imxvideoconvert_ipu or imxvideoconvert_g2d, using section
7.3.15 (Video conversion) of "i.MX_Linux_User's_Guide.pdf".

http://www.nxp.com/webapp/Download?colCode=L3.14.52_1.1.0_LINUX_DOCS&Parent_nodeId=13376994810717061...

Also, perhaps it is possible to use more speedy file storage device.

Regards,

Yuri.

[teranishi_kenji]

Dear Yuri,

Thank you so much for your replay.

I was successful according to the manual. Thank you!

About this, do you know the scaling method about imxvideoconvert_ipu or imxvideoconvert_g2d? bicubic or bilinear? The specified is possible?

BTW,

> I am using the i.MX6Q Sabre board. Therefore, in this case, the software "videoscale" is runing on 1GHz ARM Cortex-A9 processor.

> So, I think the performance is enough for "videoscale".

What do you think about this?

Best Regards,

Kenji

[Yuri]

Hello,

  You may look at sources ; please refer to "i.MX_Linux_Release_Notes.pdf" to define

what package contains GStreamer plugins.

  As for performance it depends on application.

Regards,

Yuri.

[teranishi_kenji]

Hello Yuri,

Thank you for your comments.

I can not find the source code of imxvideoconvert_ipu and imxvideoconvert_g2d plugins.

I think that this code is private firmware and binary code.

> As for performance it depends on application.

I am using the original i.MX6Q Sabre board and original Freescale Yocto Project BSP code (imx-3.14.52-1.1.0_ga).

And, I am using "fsl-image-qt5" image.

I do not know why the performance is bad.

Best Regards,

Kenji