IMX8MP ML Example not working

[Vilius]

Hi,
I'm running BSP 5.15.71-2.2.2, and I wanted to try out the machine learning example in https://www.nxp.com/docs/en/user-guide/UG10166.pdf section 8.1

When running this given pipeline, based on the example (only thing that has been changed is the waylandsink to fakevideosink as i'm running headless)

GST_DEBUG=3 gst-launch-1.0 --no-position v4l2src device=/dev/video2 ! video/x-raw,width=640,height=480,framerate=24/1 ! tee name=t t. ! queue max-size-buffers=2 leaky=2 !  imxvideoconvert_g2d ! video/x-raw,width=300,height=300,format=RGBA ! videoconvert ! video/x-raw,format=RGB ! tensor_converter ! tensor_filter framework=tensorflow-lite model=/home/root/ssd_mobilenet_v2_coco_quant_postprocess.tflite  custom=Delegate:External,ExtDelegateLib:libvx_delegate.so ! tensor_decoder mode=bounding_boxes option1=mobilenet-ssd-postprocess option2=/home/root/ssd_labels.txt option3=0:1:2:3,50 option4=640:480 option5=300:300 ! mix. t. ! queue max-size-buffers=2 ! imxcompositor_g2d name=mix latency=30000000 min-upstream-latency=30000000 sink_0::zorder=2 sink_1::zorder=1 ! fakevideosink

 
The hardware accellerated imxcompositor_g2d gives the following error

** Message: 19:38:14.427: accl = cpu
Vx delegate: allowed_cache_mode set to 0.
Vx delegate: device num set to 0.
Vx delegate: allowed_builtin_code set to 0.
Vx delegate: error_during_init set to 0.
Vx delegate: error_during_prepare set to 0.
Vx delegate: error_during_invoke set to 0.
WARNING: Fallback unsupported op 32 to TfLite
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
0:00:00.964549750  1671 0xaaaaedd5fb00 WARN           imxcompositor gstimxcompositor.c:1540:gst_imxcompositor_aggregate_frames:<mix:sink_1> set rotate failed 0xaaaaee03eb40

W [HandleLayoutInfer:274]Op 162: default layout inference pass.
W [HandleLayoutInfer:274]Op 162: default layout inference pass.
0:00:01.005968875  1671 0xaaaaedd5fb00 WARN           imxcompositor gstimxcompositor.c:1540:gst_imxcompositor_aggregate_frames:<mix:sink_1> set rotate failed 0xaaaaee03ec60

0:00:01.050180875  1671 0xaaaaedd5fb00 WARN           imxcompositor gstimxcompositor.c:1540:gst_imxcompositor_aggregate_frames:<mix:sink_1> set rotate failed 0xffff4c013480

0:00:01.089492750  1671 0xaaaaedd5fb00 WARN           imxcompositor gstimxcompositor.c:1540:gst_imxcompositor_aggregate_frames:<mix:sink_1> set rotate failed 0xaaaaee03eea0

From the logs, and other pipelines i have tried, the VX delegate works just fine, it does HW object detection. The only issue that happens is with imxcompositor_g2d. Are there specific limitations that I might not be aware of while using HW image compositors?

[joanxie]

do you mean the pipeline works with waylandsink, then failed with waylandsink to fakevideosink?

[Vilius]

No, the sink doesnt matter, be it fake sink, udp sink, or others. I have identified it to the imxcompositor_g2d component specifically. Because changing it to a software based compositor component, the pipeline works just fine.

[joanxie]

tested this on my side, didn't find any error log, the message as below is warning message, which exist in the old bsp too, this isn't the error message, reported to the GPU team about this warning message before

0:00:01.005968875  1671 0xaaaaedd5fb00 WARN           imxcompositor gstimxcompositor.c:1540:gst_imxcompositor_aggregate_frames:<mix:sink_1> set rotate failed 0xaaaaee03ec60