i.MX Processors中的主题 OpenCV vs GStreamer Pipeline for SSD-MobileNetV2 Inference on i.MX8MP (BSP 5.15.52) using VX Delegat

OpenCV vs GStreamer Pipeline for SSD-MobileNetV2 Inference on i.MX8MP (BSP 5.15.52) using VX Delegat

ashwanipal01 — Mon, 07 Jul 2025 03:52:32 GMT

Hello NXP Community,

I'm working on an object detection pipeline on the i.MX8MP platform, using NXP Linux BSP 5.15.52, with a quantized SSD-MobileNetV2 model and 480p video input. I’ve integrated the VX Delegate (libvx_delegate.so) into a TFLite runtime pipeline.

1. Current Setup:
Using OpenCV + TFLite with VX Delegate:

python
cap = cv2.VideoCapture(args.video)
if not cap.isOpened():
print(f"Failed to open video: {args.video}")
sys.exit(1)

interpreter = Interpreter(
model_path=args.model,
experimental_delegates=[load_delegate(DELEGATE_PATH)]
)
interpreter.allocate_tensors()
This works, but I observe lower FPS than expected.

2. Questions:
Performance:
Would building a full GStreamer-based pipeline (instead of OpenCV VideoCapture) improve real-time inference FPS when using TFLite + VX Delegate on i.MX8MP?

Model Quality (eIQ Toolkit vs TensorFlow CLI):
I noticed that models created using the eIQ Toolkit show lower performance (accuracy/latency) compared to models converted via TensorFlow command-line tools.
Is this expected? Are there differences in quantization strategy, postprocessing, or layer support between the two?

NXP Model Zoo:
Are there recommended pipelines or examples from the NXP Model Zoo for mobilenet_ssd_v2 that are optimized for VX Delegate on i.MX8MP BSP 5.15.52?

Any guidance, benchmarks, or sample pipelines (especially using GStreamer + TFLite + VX Delegate on i.MX8MP) would be highly appreciated!

Thanks in advance!

Re: OpenCV vs GStreamer Pipeline for SSD-MobileNetV2 Inference on i.MX8MP (BSP 5.15.52) using VX Del

Zhiming_Liu — Mon, 07 Jul 2025 06:43:19 GMT

Hi @ashwanipal01

For the object detection in video, please refer 8 Vision Pipeline with NNStreamer in i.MX Machine Learning User's Guide

https://www.nxp.com/docs/en/user-guide/UG10166.pdf

Best Regards,
Zhiming

Re: OpenCV vs GStreamer Pipeline for SSD-MobileNetV2 Inference on i.MX8MP (BSP 5.15.52) using VX Del

ashwanipal01 — Tue, 08 Jul 2025 12:31:29 GMT

hi @Zhiming_Liu ,

We followed the nnstreamer-example for the pipeline. Modified the pipeline according to the requirement.

self.pipeline = Gst.parse_launch( 'filesrc location=/home/weston/nnstre/detect_example.mp4 ! decodebin ! ' 'videoconvert ! videoscale ! video/x-raw,width=640,height=480,format=RGBA ! tee name=t ' # tee to split for inference and display # ➤ Inference Branch 't. ! queue max-size-buffers=20 leaky=downstream ! ' 'imxvideoconvert_g2d ! video/x-raw,format=RGBA,width=640,height=480 ! ' 'videoscale ! video/x-raw,width=300,height=300,format=RGBA ! ' 'videoconvert ! video/x-raw,format=RGB ! ' 'tensor_converter ! ' 'tensor_transform mode=arithmetic option=typecast:float32,add:-127.5,div:127.5 ! ' 'identity silent=false ! ' 'tensor_filter framework=tensorflow-lite model=' + self.tflite_model + ' ' 'custom=Delegate:External,ExtDelegateLib:libvx_delegate.so name=detection_filter ! ' 'tensor_decoder mode=bounding_boxes option1=mobilenet-ssd ' 'option2=' + self.tflite_label + ' ' 'option3=' + self.tflite_box_prior + ' ' 'option4=640:480 option5=300:300 ! ' 'imxvideoconvert_g2d ! mix. ' # ➤ Display Branch 't. ! queue max-size-buffers=20 leaky=downstream ! ' 'imxvideoconvert_g2d ! video/x-raw,format=RGBA,width=640,height=480 ! ' 'imxcompositor_g2d name=mix sink_0::zorder=2 sink_1::zorder=1 latency=40000000 min-upstream-latency=40000000 ! ' 'fpsdisplaysink name=img_tensor text-overlay=true video-sink=waylandsink sync=true' )

attaching the log file and output video.

we are getting the fps of 25 but the output video is not sync as input video and feels like stuck or freeze.

Re: OpenCV vs GStreamer Pipeline for SSD-MobileNetV2 Inference on i.MX8MP (BSP 5.15.52) using VX Del

Zhiming_Liu — Wed, 09 Jul 2025 01:49:02 GMT

Hi @ashwanipal01

Can you refer the pipieline setting here? Don't use video scale.

https://github.com/nxp-imx-support/nxp-demo-experience-demos-list/tree/lf-6.12.3_1.0.0/scripts/machine_learning/nnstreamer/detection

gst-launch-1.0 v4l2src name=cam_src device=/dev/video3 num-buffers=-1 ! \ video/x-raw,width=640,height=480,framerate=30/1 ! tee name=t t. ! queue name=thread-nn max-size-buffers=2 leaky=2 ! \ imxvideoconvert_g2d ! video/x-raw,width=300,height=300,format=RGBA ! \ videoconvert ! video/x-raw,format=RGB ! tensor_converter ! \ tensor_filter latency=1 framework=tensorflow-lite \ model=/opt/gopoint-apps/downloads/ssdlite_mobilenet_v2_coco_quant_uint8_float32_no_postprocess.tflite \ custom=Delegate:External,ExtDelegateLib:libvx_delegate.so name=detection_filter ! \ tensor_decoder mode=bounding_boxes option1=mobilenet-ssd \ option2=/opt/gopoint-apps/downloads/coco_labels_list.txt \ option3=/opt/gopoint-apps/downloads/box_priors.txt option4=640:480 option5=300:300 ! \ imxvideoconvert_g2d ! mix. \ t. ! queue name=thread-img max-size-buffers=2 leaky=2 ! \ imxcompositor_g2d name=mix sink_0::zorder=2 sink_1::zorder=1 latency=20000000 min-upstream-latency=20000000 ! \ cairooverlay name=perf ! \ fpsdisplaysink name=img_tensor text-overlay=false video-sink=waylandsink sync=false

Best Regards,
Zhiming