Using GStreamer + NNStreamer to run YOLOv5 (nano) model on i.MX 8M Plus

[cadietrich78]

Dear NXP community,

I’m trying an YOLOv5 (nano) model on i.MX 8M Plus (the model follows attached), and one of the options seems to be GStreamer. I’ve found some examples, all around the same construction (using the webcam as input):

gst-launch-1.0 --no-position v4l2src device=/dev/video3 ! \

"video/x-raw,width=640,height=480,framerate=30/1" ! \

tee name=t t. ! queue max-size-buffers=2 leaky=2 ! \

imxvideoconvert_g2d ! video/x-raw,width=640,height=640,format=RGBA ! \

videoconvert ! video/x-raw,format=RGB ! \

tensor_converter ! \

tensor_filter framework=tensorflow-lite model= yolov5n-fp16.tflite \

custom=Delegate:External,ExtDelegateLib:libvx_delegate.so ! \

tensor_decoder mode=bounding_boxes option1=yolov5 option2=coco.names \

option4=640:480 option5=640:640 ! \

mix. t. ! queue max-size-buffers=2 ! \

imxcompositor_g2d name=mix sink_0::zorder=2 sink_1::zorder=1 ! waylandsink

but then GStreamer complains about the format of the input tensor (the pipeline generates int8, the model expects float32). There are also examples of tensor conversion, like

tensor_converter ! \

tensor_transform mode=arithmetic option=typecast:float32,add:-127.5,div:127.5 \

which work, but with a terrible performance. I know the model is still at float32, but is there something that may be optimized in the GStreamer call?

Thanks in advance,

Carlos

[Bio_TICFSL]

Hello,

The default input model should be filled in with the shape from the original model, take a look at the eIQ link for more information on quantization.

https://community.nxp.com/t5/eIQ-Machine-Learning-Software/tkb-p/eiq@tkb

The video pipeline is using gstreamer in the document. Please refer official detect code: https://github.com/ultralytics/yolov5/blob/master/detect.py

regards