How to convert simultaneously two video streams (from YUY2 to NV12) using HW CSC?

cancel
Showing results for 
Search instead for 
Did you mean: 

How to convert simultaneously two video streams (from YUY2 to NV12) using HW CSC?

155 Views
RobbieJiang
Contributor IV

 

HW: IMX8QM
BSP: YOCTO 4.19.35


We need to encode 2 camera video streams simultanesouly.

Camera sensor outputs YUY2 raw data, but 'v4l2h264enc' can only accept NV12 data format.
So we have to convert YUY2 to NV12.

There are 3 possible video converters gstreamer elements:
imxvideoconvert_g2d, v4l2convert, videoconvert.


As we prefer to hardware conversion, so 'videoconvert' element will not be considered.
And 'imxvideoconvert_g2d' can NOT convert YUY2 format to NV12 format:

gst-inspect-1.0 imxvideoconvert_g2d
...

SRC template: 'src'
Availability: Always
Capabilities:
video/x-raw
format: { (string)RGB16, (string)RGBx, (string)RGBA, (string)BGRA, (string)BGRx, (string)BGR16, (string)ARGB, (string)ABGR, (string)xRG
B, (string)xBGR, (string)UYVY, (string)YUY2 }
video/x-raw(memory:SystemMemory, meta:GstVideoOverlayComposition)
format: { (string)RGB16, (string)RGBx, (string)RGBA, (string)BGRA, (string)BGRx, (string)BGR16, (string)ARGB, (string)ABGR, (string)xRG
B, (string)xBGR, (string)UYVY, (string)YUY2 }


So the only choice is 'v4l2convert'.

The following pipeline can run without any problem:

gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)YUY2,width=1280,height=960,framerate=30/1" ! v4l2convert ! 'video/x-raw,format=(string)NV12,width=1280,height=960,framerate=(fraction)30/1' ! v4l2h264enc ! fakesink sync=false async=false

However, seems that only 'one' instatnce of 'v4l2convert' can run.
If we tried to run another pipeline on another sensor:

gst-launch-1.0 v4l2src device=/dev/video2 ! "video/x-raw,format=(string)YUY2,width=1280,height=960,framerate=30/1" ! v4l2convert ! 'video/x-raw,format=(string)NV12,width=1280,height=960,framerate=(fraction)30/1' ! v4l2h264enc ! fakesink sync=false async=false

It reports: "WARNING: erroneous pipeline: no element "v4l2convert".
and ‘dmesg’ cmd shows
[ 1381.072904] (null): mxc_isi_capture_open: ISI channel[0] is busy
[ 1381.093449] mxc-isi 58100000.isi: mxc_isi_m2m_open: ISI channel[0] is busy


From 'gst-inspect-1.0 v4l2convert':


device : Device location
flags: readable
String. Default: "/dev/video4"


device-fd : File descriptor of the device
flags: readable
Integer. Range: -1 - 2147483647 Default: -1

Both 'device' and 'device-fd' properties are only readable, not writable.
Seems changing device is not supported.


How can we simultaneously convert two camera video streams (from YUY2 to NV12 raw format) by using hardware color space conversion?

 

0 Kudos
2 Replies

108 Views
joanxie
NXP TechSupport
NXP TechSupport

pls check if your two camera link to the same ISI? it seems the first camera use ISI0 and the second camera link to ISI0 too, you need set the second camera to the different ISI, did you use the nxp imx8qm mek board? and connect one camer via mipi_csi0 and another one via mipi_csi1??

0 Kudos

101 Views
RobbieJiang
Contributor IV

 

Thanks for reply.

Do you mean different ISI channel?
Actually, there is only one ISI in MX8QM SoC,
but this ISI has 8 processing pipelines, which can implement the HW based CSC.

 

However,  seems there is no way to configure different channel using gstreamer element 'v4l2convert':

gst-inspect-1.0 v4l2convert:

...

device : Device location
flags: readable
String. Default: "/dev/video4"


device-fd : File descriptor of the device
flags: readable
Integer. Range: -1 - 2147483647 Default: -1

...

 

Note both  "device" and "device-fd" properties are ONLY READABLE

means no way to change it.

So the potential solution is to set the related configurations in kernel, not in the application.

 

0 Kudos