imx8mq: VPU base decoding and display

[jignesh1]

Hello NXP Team,

I use "hx170dec" example to decoder h264 video and created one drm base application that display that NV12 data on HDMI.

I used to read file at 30 fps and display it on HDMI. Everything are working correct but just i see that CPU usage is around 40%.  Here I need to use memcpy to copy VPU decoded data to display.

1. Do we have any example that can avoid above memcopy to reduce CPU usage. ?

2. In another use-case I also need to display encoded data to lcdif. I know that currently lcdif only support RGB format and VPU does not support any post processing. So do we have any library or example GPU base for NV12 to RGB croma conversion ?

Thanks,

Jignesh Patel

[joanxie]

1) why do you need to use memcpy to copy VPU decoded data to display

2) IPU has csc(Color Space Conversion unit) which can convert yuv to rgb, you can refer to it, for your case, you can use CSC in the DP(display port), more detailed information, pls refer to the 37.4.9.5 Color Space Conversion unit - CSC of Reference Manual

[jignesh1]

Hi Joan,

We are using imx8mq evk board. So it has vpu name called Hantro. I validate that VPU required to allocate 5 min buffer at output side to start decoding and as we are using DRM mechanism to display data to HDMI. DRM has only one frame-buffer so I need to do mem-copy there. Is there any other way for linking this ?

I also checked same use-case with Gstreamer and it uses very less CPU usage.

https://community.nxp.com/docs/DOC-343017 

About second point imx8mq does not have IPU unit.

Please guide us if we have any sample code that makes pipeline from VPU decoder to display. I do not find yet any standard example for imx8mq evk board.

[joanxie]

do you want to evaluate imx8mq vpu? if yes, you just connect imx8mq with hdmi, the image supports hdmi output as default, then you can use gstreamer to decode the file outpting to the hdmi, the gstreamer commands, you can refer to the document you send, I post abc three commands  which use gstreamer to decode video by VPU to the display

[jignesh1]

Hi Joan,

I have one quick query.

All the decoder example (like hx170dec) given in buildroot are decoding based on hardware VPU or software ? 

Because these default demo took more CPU usage.  While gstreamer took less cpu usage.

Thanks,

Jignesh Patel

[joanxie]

hx170dec is under vpu, so which should be use hardware, hx170dec is decoding from yuv to h264 or mpeg4, not play video, if you need to play video, pls use gstreamer

[jignesh1]

Hi Joan,

Also if i use use binary build by Yocto which is also using kernel 4.14 then also i see higher CPU usage.

Please help us if anything wrong with this sample application.

Thanks,

Jignesh Patel

[joanxie]

how about use gstreamer  to decode the video? which use vpu hardware with lower cpu usage

[jignesh1]

Hi Joan,

I am okay to use gstreamer if gstreamer can display video on eLCDIF with GUI ?

Do you have any command for this ?

Also we suspect that existing demo "hx170dec"  etc.. are software base decoding and they are not using VPU hantro hardware. please correct me if this is wrong.

[N_Coesel]

Check how the DMA buffers are created. I had a similar problem using the VPU wrapper library. The DMA buffers are not cachable which causes memcpy to take very long. Gstreamer likely creates DMA buffers which are cacheable and thus needs much less CPU time.

[joanxie]

gstreamer is for video playing though vpu on fb0 and QT gui on the fb1, for elcdif, we has already released elcdif dtb file for hdmi or oled, you can refer to it

[jignesh1]

Hi Joan,

I have tested Demo on latest kernel 4.14 and vpu hantro version "mx-vpu-hantro-1.10.0" still I see 100 % or more cpu usage at 30 fps video.

Here i am putting log information:

====================================================

root@imx8mqevk:~# ./hx170dec /run/media/mmcblk0p7/test.h264

X170 H.264 Decoder API v1.1 - SW build: 3.1 - HW build: 67315198

UNABLE TO OPEN INPUT FILE: "tb.cfg"
USING DEFAULT CONFIGURATION
Decoder Macro Block Error Concealment 0
Decoder RLC 0
Decoder Clock Gating 1
Decoder Data Discard 0
Decoder Latency Compensation 0
Decoder Output Picture Endian 1
Decoder Bus Burst Length 16
Decoder Asic Service Priority 0
Decoder Output Format 0
TB Packet by Packet  0
TB Nal Unit Stream 0
TB Seed Rnd 1
TB Stream Truncate 0
TB Stream Header Corrupt 0
TB Stream Bit Swap 0; odds 0
TB Stream Packet Loss 0; odds 0
Open /dev/dri/card0 success
num of fb:0
num of crtc:1
num of encoder:1
num of connector:1
Not found connection
kms info: fb_base = 0x4070e8
w/h=(4219940,0)
bits_per_pixel=105069262
bpp=840554096
screen_buf_size =0
H264DEC_HDRS_RDY
sizeof(dpb) = 4360
offset of dpbs = 2432
Width 1280 Height 720
Cropping params: (0, 0) 1280x720
MonoChrome = 0
Interlaced = 0
DPB mode   = 0
Pictures in DPB = 5
Pictures in Multibuffer PP = 5
Output format = H264DEC_SEMIPLANAR_YUV420
video_range 0, matrix_coefficients 2
Other 9
Waiting for frame buffers
H264DecGetBufferInfo ret 9
buf_to_free (nil), next_buf_size 1612832, buf_num 5
H264DecAddBuffer ret 9
H264DecAddBuffer ret 9
H264DecAddBuffer ret 9
H264DecAddBuffer ret 9
H264DecAddBuffer ret 0

====================================================

Here I am doing only decoding and I have disable the prints and dump logic in demo and still is see high CPU usage.

Also about the use of gstreamer, I need to give this decoded buffer to gpu for croma coversion, scalling and other video post processing. Let me know if any help available on this. So i can use that and skip this exploration.

Thanks,

Jignesh Patel

[jignesh1]

Hi Joan,

We are going to use imx8mq for our new product. We have following requirement.

1. Receive h264 stream and play on HDMI. This can be achieve by gstreamer command. We are okay with this.

2. Receive other h264 stream and display on LCD connected to eLCDIF with GUI. Here we need support of 90/270 degree rotation, scaling and YUV-RGB conversion support from GPU. So we can not use existing gstreamer. Do we have any example for this ?

Right now this second point is blocking point for us.

Thanks,

Jignesh Patel