i.MX Processors中的主题 Re: Frame drop at decoding video and scaling

Frame drop at decoding video and scaling

teranishi_kenji — Mon, 13 Jun 2016 08:23:39 GMT

Hi all,

I am trying a decoding of H.264 video on i.MX6Q Sabre board using Freescale Yocto Project BSP code (imx-3.14.52-1.1.0_ga).

But, some of the frames are dropped when I try decoding and scaling.

[Command]

> gst-launch-1.0 filesrc location=test_640x480.avi typefind=true ! video/x-msvideo ! aiurdemux ! vpudec ! videoscale method=1 ! video/x-raw,width=800,height=480 ! overlaysink

[Error Message]

> WARNING: from element /GstPipeline:pipeline0/GstOverlaySink:overlaysink0: A lot of buffers are being dropped.

> Additional debug info:

build/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/gstreamer1.0/1.4.5-r0/gstreamer-1.4.5/libs/gst/base/gstbasesink.c(2794): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstOverlaySink:overlaysink0:

> There may be a timestamping problem, or this computer is too slow.

When I try only the decoding and only the scaling, there are not dropped frame.

I think that "vpudec" is hardware-based process and "videoscale" is software-based process.

Therefore, I think that the decoding and the scaling can be parallel processing.

Why some of the frame is dropped?

As additional information, I measure the processing time.

With decoding : Scaling time is 61ms / frame

Only Scaling (*) : Scaling time is 8ms / frame

(*) YUV showing (No decoding)

Command : gst-launch-1.0 filesrc location=test_640x480.yuv ! videoparse format=2 width=640 height=480 framerate=15 ! videoscale method=1 ! video/x-raw,width=800,height=480 ! overlaysink

Why the scaling time is long when decoding and scaling has been processed at the same time?

Best Regards,

Kenji

Re: Frame drop at decoding video and scaling

Yuri — Tue, 14 Jun 2016 02:44:15 GMT

Hello,

Looks like the issue is just a performance issue, mainly because of software “videoscale”.
Please try using one of video conversion plugins (imxvideoconvert_ipu, imxvideoconvert_g2d,
imxvideoconvert_pxp) to resize image.
Also You may try to insert queue element in Gstreamer pipeline for buffering in order to
improve situation.

Have a great day,
Yuri

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

Re: Frame drop at decoding video and scaling

teranishi_kenji — Tue, 14 Jun 2016 04:43:55 GMT

Dear Yuri,

Thank you so much for your comment.

I am using the i.MX6Q Sabre board. Therefore, in this case, the software "videoscale" is runing on 1GHz ARM Cortex-A9 processor.

I checked "videoscale" source code, and I think that this processing throughput is not large.

So, I think the performance is enough for "videoscale".

What do you think about this?

BTW, about imxvideoconvert_ipu or imxvideoconvert_g2d, does these plugins have the resize function?

Please tell me a command to resize about these plugin.

I appreciate your cooperation.

Best Regards,

Kenji

Re: Frame drop at decoding video and scaling

Yuri — Tue, 14 Jun 2016 07:31:15 GMT

Please try imxvideoconvert_ipu or imxvideoconvert_g2d, using section
7.3.15 (Video conversion) of "i.MX_Linux_User's_Guide.pdf".

http://www.nxp.com/webapp/Download?colCode=L3.14.52_1.1.0_LINUX_DOCS&Parent_nodeId=1337699481071706174845&Parent_pageTyp…

Also, perhaps it is possible to use more speedy file storage device.

Regards,

Yuri.

Re: Frame drop at decoding video and scaling

teranishi_kenji — Tue, 14 Jun 2016 09:48:47 GMT

Dear Yuri,

Thank you so much for your replay.

I was successful according to the manual. Thank you!

About this, do you know the scaling method about imxvideoconvert_ipu or imxvideoconvert_g2d? bicubic or bilinear? The specified is possible?

BTW,

> I am using the i.MX6Q Sabre board. Therefore, in this case, the software "videoscale" is runing on 1GHz ARM Cortex-A9 processor.

> So, I think the performance is enough for "videoscale".

What do you think about this?

Best Regards,

Kenji

Re: Frame drop at decoding video and scaling

Yuri — Mon, 04 Jul 2016 07:07:03 GMT

Hello,

You may look at sources ; please refer to "i.MX_Linux_Release_Notes.pdf" to define

what package contains GStreamer plugins.

As for performance it depends on application.

Regards,

Yuri.

Re: Frame drop at decoding video and scaling

teranishi_kenji — Tue, 05 Jul 2016 02:54:57 GMT

Hello Yuri,

Thank you for your comments.

I can not find the source code of imxvideoconvert_ipu and imxvideoconvert_g2d plugins.

I think that this code is private firmware and binary code.

> As for performance it depends on application.

I am using the original i.MX6Q Sabre board and original Freescale Yocto Project BSP code (imx-3.14.52-1.1.0_ga).

And, I am using "fsl-image-qt5" image.

I do not know why the performance is bad.

Best Regards,

Kenji