I use a camera using ISP in IMX8MPlus with MIPI CSI.
v4l2ctl --list-devices
():
/dev/v4l-subdev0
/dev/v4l-subdev2
/dev/v4l-subdev3(csi0):
/dev/v4l-subdev1FSL Capture Media Device (platform:32c00000.bus:camera):
/dev/media0mxc-isi_v1 (platform:32e00000.isi:cap_devic):
/dev/video2VIV (platform:viv0):
/dev/video3
viv_media (platform:vvcam-video.0):
/dev/media1
So when i capture the video from ISI here /dev/video2 , with OpenCV GStreamer, i get CPU usage of about 90%,
But the same video capture from ISP /dev/video3, with OpenCV GStreamer, i get CPU usage of about 125%
Using perf, if we analyze it says lot of cpu processing wasted in SIMD? instructions
So this additional instructions are loaded when using ISP.
Capturing the frames by this way ( with GStreamer)
cv::VideoCapture cap(cam_index)
if (!cap.isOpened())
{
std::cerr << "Failed to open camera " << cam_index << "\n";
return 1;
}
Need to remove this overhead when uisng the ISP and opencV.
Addtl detail on ISP stream
v4l2-ctl -d /dev/video3 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture[0]: 'YUYV' (YUYV 4:2:2)
Size: Stepwise 176x144 - 4096x3072 with step 16/8
[1]: 'NV12' (Y/UV 4:2:0)
Size: Stepwise 176x144 - 4096x3072 with step 16/8
[2]: 'NV16' (Y/UV 4:2:2)
Size: Stepwise 176x144 - 4096x3072 with step 16/8
[3]: 'BA12' (12-bit Bayer GRGR/BGBG)
Size: Stepwise 176x144 - 4096x3072 with step 16/8v4l2-ctl -d /dev/video3 --get-fmt-video
Format Video Capture:
Width/Height : 1280/800
Pixel Format : 'NV16' (Y/UV 4:2:2)
Field : None
Bytes per Line : 1280
Size Image : 2048000
Colorspace : Rec. 709
Transfer Function : Rec. 709
YCbCr/HSV Encoding: Rec. 709
Quantization : Limited Range
Flags : set-csc
Please advise.
pls give me your detailed opencv usage steps, let me try to reproduce this on my side
This was due to the conversion that opencv performs while consuming data.
So tried to use the pure gstreamer to grab the frame with imxvideoconvertor_g2d 2D GPU and it brought the cpu usage by ~50%