i.MX Processors中的主题 Re: CPU load when streaming full HD camera input to full HD HDMI display

CPU load when streaming full HD camera input to full HD HDMI display

zhencongtee — Thu, 03 Jul 2014 07:45:15 GMT

Hi all,

Now i am trying to stream a full HD resolution (1920x1080, 30fps) camera input from OV5640 MIPI camera to display it on a full HD HDMI display.

I manage to do so by:

gst-launch mfw_v4lsrc device=/dev/video1 capture-mode=5 ! mfw_v4lsink

However, when i see the CPU load using the 'top' command, the CPU load is near to 100%. Why is the CPU load so high? Isn't it the IPU should kick in in such situation and free the CPU from doing other task?

I am using i.mx6SabreSD board, trying to simulate performance of i.mx6 SOLO by limiting the cpu to 1 core in u-boot boot arguement.

Thank you.

Regards,

Zhen Cong Tee

Re: CPU load when streaming full HD camera input to full HD HDMI display

Yuri — Fri, 04 Jul 2014 07:03:03 GMT

In Your pipeline there are no codecs, this means huge amount of raw ("non-zipped") data
should be read from the cam, copied to display framebuffer 30 times per sec, also frame buffer
should be displayed, say 30 times per sec.
Just for estimation, data throughput : 240 MB/sec ~ 1920x1080 x 30 fps x 4 bytes per pixel.
It should be, at least, doubled, since data are read from the cam to memory and than written
to display. Really system bus is heavy loaded.

Have a great day,
Yuri

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

Re: CPU load when streaming full HD camera input to full HD HDMI display

zhencongtee — Fri, 04 Jul 2014 07:07:17 GMT

Hi Yuri,

Thanks! you cleared my doubt.

In such case, do you recommend me to do the encoding on the input pipeline, and then decode it to the output display?

Is the command something like:

# gst-launch mfw_v4lsrc device=/dev/video1 ! vpuenc codec=6 ! matroskamux ! queue max-size-bytes=0 max-size-time=0 ! vpudec ! mfw_v4lsink

Please correct my pipeline if i am wrong.

Thank you.

Regards,

Zhen Cong Tee

Re: CPU load when streaming full HD camera input to full HD HDMI display

zhencongtee — Mon, 07 Jul 2014 02:48:35 GMT

Hi Yuri,

Some update after doing some experiment.

I found out that the CPU load changed tremendously when i change the sink from mfw_v4lsink to mfw_isink.

I tried with 720p MIPI input, 1080p HDMI output.

When using mfw_v4lsink, the CPU load is about 40%

when using mfw_isink, the CPU load goes down to as low as 5%!

May i know what is the different between v4lsink and isink? what makes the different?

And, i am not able to stream 1080p MIPI input to 1080p HDMI output using mfw_isink. console throw out a lot of errors below:

MAX resolution 1920x1080

set color key

MFW_GST_ISINK_PLUGIN 3.0.11 build on Jun 17 2014 15:54:17.

Setting pipeline to PAUSED ...

Pipeline is live and does not need PREROLL ...

Setting pipeline to PLAYING ...

New clock: GstSystemClock

can not create threadmfw_gst_isink_setcaps:1489

VS0 created. in fmt[I420] win(0,0-1920,1080:1920x1080) out win(0,0-1920,1080:1920x1080)

set deinterlace mode 0

imx-ipuv3 2400000.ipu: IPU Warning - IPU_INT_STAT_10 = 0x00080000

and HDMI screen goes blank.

Any idea why is this happening? I had included the VSALPHA = 1 before running the gst-launch command.

Thanks.

regards,

Zhen Cong Tee

Re: CPU load when streaming full HD camera input to full HD HDMI display

Yuri — Mon, 07 Jul 2014 03:06:38 GMT

Hi,

In comments of the next doc :

https://community.nxp.com/docs/DOC-93788

2. What is basic difference of these two?

Both are similar elements, in the sense that both are video sinks, however, the isink element uses the IPU lib, which allows to have multi-overlay, meaning that several playbacks can run on the same device. This is not possible using the mfw_v4lsink element. Check the properties for each element using the cmd 'gst-inspect'. Also, check the i.MX Linux Multimedia Framework User's Guide for more info.

Have a great day,
Yuri

Re: CPU load when streaming full HD camera input to full HD HDMI display

zhencongtee — Mon, 07 Jul 2014 05:07:08 GMT

Hi Yuri,

Thanks for the reply.

Understand that isink uses IPU lib which allows multi-overlay. How about v4lsink? Does it uses IPU as well?

And, any idea on the error i am getting when trying to stream in 1080p MIPI input?

Thank you.

Regards,

Zhen Cong Tee

Re: CPU load when streaming full HD camera input to full HD HDMI display

Yuri — Thu, 10 Jul 2014 01:46:55 GMT

The mfw_v4lsink uses IPU features, but not optimal for multitasking.

As for IPU warnings in Your log : mxc_ipu mxc_ipu: IPU Error - IPU_INT_STAT_10 = 0x00080000

Have a great day,
Yuri

Re: CPU load when streaming full HD camera input to full HD HDMI display

zhencongtee — Thu, 10 Jul 2014 02:06:53 GMT

Hi Yuri,

I am using Dora branch of the Yocto. It seems like the patch as mentioned is not useful in my case.

Besides that , i tried to include the dmfc=3 in the bootarg, but it does not help either.

I see that you mentioned it's something about the IPU performance. Does it means that the IPU cannot take in 1920x1080p @ 30fps?

Another finding throughout my experiments. I tried to change the vssconfig so that mfs_isink write to fb0 instead of the original fb1, and this does not happen anymore. But i do not understand what is the different between writing to the display framebuffer (fb0) and overlay framebuffer (fb1)? Why does it makes the different?

Regards,

Zhen Cong Tee

Re: CPU load when streaming full HD camera input to full HD HDMI display

Yuri — Tue, 15 Jul 2014 08:15:31 GMT

As for IPU performance - it depends on system load (mainly - DRAM throughput), since other
(system throughput consuming) modules may influence here.