VPU performance in gray scale H.264 encode

[tetsuomaeda]

Hi,

We are trying to implement a gray scale H.264 encoder of 1920 x 1080 30fps using VPU.
Video source is a CMOS camera connected to CSI.

First, 30fps color encoding is successful at the following path.
CSI => DRAM => VPU (H.264 encode)

We are using IC (in IPU) to convert YUV to gray scale (U = V = 0) and the path is as follows.

CSI => DRAM => IC (IPU) => DRAM => VPU (H.264 encode)

Our engineer has modified the device driver of IPU to convert the YUV stream to gray scale.

In case of the resolution 1920 x 1080, the encoded frame rate is as low as around 10fps!
But, in case of the resolution 1280 x 720, 30fps encoding is successful. In this case, the resizing procedure in IC is added.

We have not found the reason of the above low frame rate.

Does any one know the reason of the above phenomena?

I'd appreciate any responses.

Best Regards,

Tetsuo Maeda

[phdm]

Have you tried the gstreamer-imx 0.12.3 plugins ?  gstreamer-imx can now encode directly grayscale images without going in the IC.  It adds two buffers filled with 0x80 bytes and pretends to feed the vpu with I420.  Stream is thus :

Camera => CSI => DRAM

DRAM + static 0x80 zones => VPU.

I have not tested with 1920 x 1080, but for 1280 x 1024, that works perfectly

Philippe

[tetsuomaeda]

Hi, Philippe-san,

Thank you for the fruitful suggestion.

Actually my colleague had thought the same thing and developed the original code.

At first we had planned to change the tone curve and we thought we need to use IC.

Currently we don't need to modify the tone curve and we just need to fill U and V with 0x80.

I'll pass your comment to my colleague.

Thanks!

Tetsuo 

[art]

Generally, VPU should be able to encode 1920x1080@30fps image. The possible cause of the issue is too high data transfer rate on the DRAM bus causing VPU to slow down due to lower memory access rate. Try to optimize your software in the meaning of DRAM use. For example, you can prepare separate U and V buffers, zeroize them before starting decoding and then not touch them at all, using only Y component from the camera.

Have a great day,
Artur

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[tetsuomaeda]

Hi, Artur-san,

Thank you very much for the advice.

Our engineer is trying to implement the gray scale conversion according to your advice and the result seems to be good!

Does your "too high transfer rate on the DRAM bus" mean the limitation of the output rate from IC to the system memory?

There is a description "Rate: up to 100Mpixels/sec (e.g. 1920x1080 @ 30 fps)" on page 2736 of "iMX 6Dual/Quad application Processor Reference manual" such as "IMX6DQRM.pdf".

I'd appreciate your feedback.

Regards,

Tetsuo

[art]

Q. Does your "too high transfer rate on the DRAM bus" mean the limitation of the output rate from IC to the system memory?

A. Yes, this is exactly what I mean.

[tetsuomaeda]

Hi Artur-san,

Thank you for the confirmation.
Because we need to modify the slope of the conversion "Yin => Yout" at the same time when we eliminate U and V, we had planned to use IC in IPU.
When we do the same procedure at the resolution 1280 x 720, we get the encoded stream of 30fps.
We are still looking for the solution of "1920 x 1080 at 30fps" using iMX6, I'd appreciate your advice!

Regards,

Tetsuo.

[art]

Seems that in the CSI => DRAM => IC (IPU) => DRAM => VPU sequence you have to eliminate the extra DRAM access so as the sequence will be as follows: CSI => IC (IPU) => DRAM => VPU. Try that.

[tetsuomaeda]

Hi Artur-san,

At first, we were thinking the following stream flow.

Camera => CSI => IC => DRAM => VPU.

But because of the limitation (up to 1024 x 1024) of IC in the output to DRAM, we have to split the stream of 1920 x 1080.

As the result of the splitting, the frame rate of the video stream of FHD (1920 x 1080) becomes less than 30fps.

We think we have to avoid the splitting so that we can record the FHD, 30fps stream.

At the moment, we have no idea to do so.

Regards,

Tetsuo Maeda

[art]

How do you use IC to convert the YUV camera input to grayscale? Please specify it in more details. Also, please provide your camera documentation to make me able to check the possible cases of making the grayscale input.

Artur