running neural network streamer example in imx8mp

[greeran]

hi. i am encountering a problem when running the nxp-nnstreamer examples. i am running on imx8mp kirkstone 5.15.71. i am running the example from nxp-nnstreamer-examples but it fails. is the nnstreamer examples able to run on kirkstone 5.15.71 and if so might someone tell me what i am missing to fix the example. 

./example_detection_mobilenet_ssd_v2_tflite -c /dev/video3 -p ${MOBILENETV2_QUANT} -l /storage/nxp-nnstream-dir/downloads/models/object-detection/coco_labels_list.txt -x ${MOBILENETV2_BOXES} -r 1280,1024,25

INFO: Start app...

DEBUG: v4l2src name=cam_src device=/dev/video3 num-buffers=-1 ! video/x-raw,width=1280,height=1024,framerate=25/1 ! tee name=t t. ! queue name=thread-nn max-size-buffers=2 leaky=2 ! imxvideoconvert_g2d name=scale_csc_g2d_0 ! video/x-raw,width=300,height=300,format=RGBA ! videoconvert name=rgb_convert_cpu_1 ! video/x-raw,format=RGB ! tensor_converter ! tensor_filter latency=1 framework=tensorflow-lite model=/storage/nxp-nnstream-dir/downloads/models/object-detection/ssdlite_mobilenet_v2_coco_quant_uint8_float32_no_postprocess.tflite custom=Delegate:External,ExtDelegateLib:libvx_delegate.so name=detection_filter ! tensor_decoder name=tensor_decode_bounding_boxes_2 mode=bounding_boxes option1=mobilenet-ssd option2=/storage/nxp-nnstream-dir/downloads/models/object-detection/coco_labels_list.txt option3=/storage/nxp-nnstream-dir/downloads/models/object-detection/box_priors.txt option4=1280:1024 option5=300:300 ! imxvideoconvert_g2d ! mix.sink_0 t. ! queue name=thread-img max-size-buffers=2 leaky=2 ! mix.sink_1 imxcompositor_g2d name=mix sink_0::zorder=2 sink_1::zorder=1 latency=20000000 min-upstream-latency=20000000 ! waylandsink

** Message: 15:26:38.138: accl = cpu
Vx delegate: allowed_cache_mode set to 0.
Vx delegate: allowed_builtin_code set to 0.
Vx delegate: error_during_init set to 0.
Vx delegate: error_during_prepare set to 0.
Vx delegate: error_during_invoke set to 0.
0:00:00.385170625 1298 0xaaaaed9a0640 WARN v4l2src gstv4l2src.c:794:gst_v4l2src_query:<cam_src> Can't give latency since framerate isn't fixated !
0:00:00.385286625 1298 0xaaaaed9a0640 WARN v4l2src gstv4l2src.c:794:gst_v4l2src_query:<cam_src> Can't give latency since framerate isn't fixated !
0:00:00.385330625 1298 0xaaaaed9a0640 WARN aggregator gstaggregator.c:2099:gst_aggregator_query_latency_unlocked:<mix> Latency query failed
0:00:00.386064500 1298 0xaaaaed9a0640 WARN v4l2src gstv4l2src.c:794:gst_v4l2src_query:<cam_src> Can't give latency since framerate isn't fixated !
0:00:00.386158125 1298 0xaaaaed9a0640 WARN v4l2src gstv4l2src.c:794:gst_v4l2src_query:<cam_src> Can't give latency since framerate isn't fixated !
0:00:00.386201875 1298 0xaaaaed9a0640 WARN aggregator gstaggregator.c:2099:gst_aggregator_query_latency_unlocked:<mix> Latency query failed
0:00:00.386379375 1298 0xaaaaed9a0d80 WARN v4l2 gstv4l2object.c:4683:gst_v4l2_object_probe_caps:<cam_src:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument

DEBUG: Pipeline state changed from NULL to READY.

DEBUG: Pipeline state changed from READY to PAUSED.
0:00:00.391118500 1298 0xaaaaed9a0640 WARN v4l2src gstv4l2src.c:794:gst_v4l2src_query:<cam_src> Can't give latency since framerate isn't fixated !
0:00:00.391224750 1298 0xaaaaed9a0640 WARN v4l2src gstv4l2src.c:794:gst_v4l2src_query:<cam_src> Can't give latency since framerate isn't fixated !
0:00:00.391268125 1298 0xaaaaed9a0640 WARN aggregator gstaggregator.c:2099:gst_aggregator_query_latency_unlocked:<mix> Latency query failed
0:00:00.397494750 1298 0xaaaaed9a0640 WARN v4l2src gstv4l2src.c:794:gst_v4l2src_query:<cam_src> Can't give latency since framerate isn't fixated !
0:00:00.397592000 1298 0xaaaaed9a0640 WARN v4l2src gstv4l2src.c:794:gst_v4l2src_query:<cam_src> Can't give latency since framerate isn't fixated !
0:00:00.397638500 1298 0xaaaaed9a0640 WARN aggregator gstaggregator.c:2099:gst_aggregator_query_latency_unlocked:<mix> Latency query failed
0:00:00.400187500 1298 0xaaaaed9a0640 WARN v4l2src gstv4l2src.c:794:gst_v4l2src_query:<cam_src> Can't give latency since framerate isn't fixated !
0:00:00.400289500 1298 0xaaaaed9a0640 WARN v4l2src gstv4l2src.c:794:gst_v4l2src_query:<cam_src> Can't give latency since framerate isn't fixated !
0:00:00.400333750 1298 0xaaaaed9a0640 WARN aggregator gstaggregator.c:2099:gst_aggregator_query_latency_unlocked:<mix> Latency query failed
0:00:00.407920875 1298 0xaaaaed9a0d80 WARN v4l2bufferpool gstv4l2bufferpool.c:884:gst_v4l2_buffer_pool_start:<cam_src:pool0:src> Uncertain or not enough buffers, enabling copy threshold
0:00:00.871101250 1298 0xaaaaed9a0640 WARN videopool gstvideopool.c:226:video_buffer_pool_set_config:<waylandpool0> Provided size is to small for the caps: 0 < 5242880
ERROR: Invoke called on model that is not ready.

** (example_detection_mobilenet_ssd_v2_tflite:1298): CRITICAL **: 15:26:38.660: Failed to invoke

** (example_detection_mobilenet_ssd_v2_tflite:1298): CRITICAL **: 15:26:38.661: /usr/lib/libnnstreamer-single.so(_backtrace_to_string+0x40) [0xffff703c73c0]/usr/lib/gstreamer-1.0/libnnstreamer.so(+0x3d9ac) [0xffff7042d9ac]/usr/lib/libgstbase-1.0.so.0(+0x482a4) [0xffff847782a4]/usr/lib/libgstbase-1.0.so.0(+0x47964) [0xffff84777964]/usr/lib/libgstreamer-1.0.so.0(+0x8f368) [0xffff854bf368]/usr/lib/libgstreamer-1.0.so.0(+0x911f8) [0xffff854c11f8]/usr/lib/gstreamer-1.0/libnnstreamer.so(+0x1fba0) [0xffff7040fba0]/usr/lib/libgstreamer-1.0.so.0(+0x8f368) [0xffff854bf368]/usr/lib/libgstreamer-1.0.so.0(+0x911f8) [0xffff854c11f8]/usr/lib/libgstbase-1.0.so.0(+0x47a78) [0xffff84777a78]/usr/lib/libgstreamer-1.0.so.0(+0x8f368) [0xffff854bf368]/usr/lib/libgstreamer-1.0.so.0(+0x911f8) [0xffff854c11f8]/usr/lib/libgstbase-1.0.so.0(+0x47a78) [0xffff84777a78]/usr/lib/libgstreamer-1.0.so.0(+0x8f368) [0xffff854bf368]/usr/lib/libgstreamer-1.0.so.0(+0x911f8) [0xffff854c11f8]/usr/lib/libgstbase-1.0.so.0(+0x47a78) [0xffff84777a78]/usr/lib/libgstreamer-1.0.so.0(+0x8f368) [0xffff854bf368]/usr/lib/libgstreamer-1.0.so.0(+0x911f8) [0xffff854c11f8]/usr/lib/libgstbase-1.0.so.0(+0x47a78) [0xffff84777a78]/usr/lib/libgstreamer-1.0.so.0(+0x8f368) [0xffff854bf368]

** (example_detection_mobilenet_ssd_v2_tflite:1298): CRITICAL **: 15:26:38.661: Calling invoke function (inference instance) of the tensor-filter subplugin (tensorflow-lite for /storage/nxp-nnstream-dir/downloads/models/object-detection/ssdlite_mobilenet_v2_coco_quant_uint8_float32_no_postprocess.tflite) has failed with error code (-1).

0:00:00.882050500 1298 0xaaaaed9a0640 WARN waylandsink wlwindow.c:439:gst_wl_window_new_internal: init surface_state fail, fallback to scale=1 fullscreen (1280x992)

DEBUG: Pipeline state changed from PAUSED to PLAYING.
0:00:00.909338875 1298 0xaaaaed9a0d80 WARN v4l2src gstv4l2src.c:1123:gst_v4l2src_create:<cam_src> lost frames detected: count = 1 - ts: 0:00:00.523322125
0:00:00.909574250 1298 0xaaaaed9a0d80 WARN basesrc gstbasesrc.c:3127:gst_base_src_loop:<cam_src> error: Internal data stream error.
0:00:00.909620125 1298 0xaaaaed9a0d80 WARN basesrc gstbasesrc.c:3127:gst_base_src_loop:<cam_src> error: streaming stopped, reason error (-5)
0:00:00.909807500 1298 0xaaaaed9a0d80 WARN queue gstqueue.c:992:gst_queue_handle_sink_event:<thread-nn> error: Internal data stream error.
0:00:00.909849750 1298 0xaaaaed9a0d80 WARN queue gstqueue.c:992:gst_queue_handle_sink_event:<thread-nn> error: streaming stopped, reason error (-5)

ERROR: Error received from element cam_src: Internal data stream error..

ERROR: Use 'gst-device-monitor-1.0 Video/Source' command to check supported camera device resolution and framerate

DEBUG: Debugging information: ../git/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:cam_src:
streaming stopped, reason error (-5).

DEBUG: Closing the main loop.
Total showed frames (2), playing for (0:00:00.524242875), fps (3.815).

INFO: close app...

[greeran]

hi

i was able to run a stream with a module i created from 

https://github.com/NXP/eiq-model-zoo/blob/main/tasks/vision/object-detection/ssdlite-mobilenetv2/REA.... the problems is that it seems to enable the npu but i see 2 cores on 100% usage and the fps is 5.

gst-launch-1.0 -v \

    v4l2src device=$VIDEO_DEVICE ! \

    video/x-raw,format=YUY2,width=1280,height=1024,framerate=25/1 ! \

    videoconvert ! \

    video/x-raw,format=RGB ! \

    tee name=t allow-not-linked=true \

1. ! queue max-size-buffers=30 ! \

        videoconvert ! \

        waylandsink sync=false async=true name=sink1 \

1. ! queue leaky=downstream max-size-buffers=2 ! \

        videoscale ! \

        video/x-raw,width=$INPUT_WIDTH,height=$INPUT_HEIGHT ! \

        videorate ! \

        video/x-raw,framerate=8/1 ! \

        tensor_converter ! \

        tensor_transform mode=arithmetic option=typecast:float32,add:-127.5,div:127.5 ! \

        tensor_filter framework=tensorflow2-lite model=$MODEL_PATH accelerator=true:npu ! \

        queue leaky=downstream max-size-buffers=2 ! \

        tensor_decoder mode=bounding_boxes option1=mobilenet-ssd option2=$LABELS_PATH option3=0.5 option4=1280:1024 option5=300:300 ! \

        fakesink sync=false

the output is: 

./nnstreamer_detector.sh
Starting NNStreamer object detection pipeline...
Model: ssdlite-mobilenetv2.tflite
Device: /dev/video3
Processing every 3 frames with NPU
Press Ctrl+C to stop
** Message: 10:04:49.170: accl = npu
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)1024, framerate=(fraction)25/1, views=(int)1, interlace-mode=(string)progressive, colorimetry=(string)1:4:5:1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)1024, framerate=(fraction)25/1, views=(int)1, interlace-mode=(string)progressive, colorimetry=(string)1:4:5:1
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps = video/x-raw, width=(int)1280, height=(int)1024, framerate=(fraction)25/1, views=(int)1, interlace-mode=(string)progressive, format=(string)RGB, colorimetry=(string)1:1:5:1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw, width=(int)1280, height=(int)1024, framerate=(fraction)25/1, views=(int)1, interlace-mode=(string)progressive, format=(string)RGB, colorimetry=(string)1:1:5:1
/GstPipeline:pipeline0/GstTee:t.GstTeePad:src_0: caps = video/x-raw, width=(int)1280, height=(int)1024, framerate=(fraction)25/1, views=(int)1, interlace-mode=(string)progressive, format=(string)RGB, colorimetry=(string)1:1:5:1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, width=(int)1280, height=(int)1024, framerate=(fraction)25/1, views=(int)1, interlace-mode=(string)progressive, format=(string)RGB, colorimetry=(string)1:1:5:1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, width=(int)1280, height=(int)1024, framerate=(fraction)25/1, views=(int)1, interlace-mode=(string)progressive, format=(string)RGB, colorimetry=(string)1:1:5:1
/GstPipeline:pipeline0/GstTee:t.GstTeePad:src_1: caps = video/x-raw, width=(int)1280, height=(int)1024, framerate=(fraction)25/1, views=(int)1, interlace-mode=(string)progressive, format=(string)RGB, colorimetry=(string)1:1:5:1
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = video/x-raw, width=(int)1280, height=(int)1024, framerate=(fraction)25/1, views=(int)1, interlace-mode=(string)progressive, format=(string)RGB, colorimetry=(string)1:1:5:1
/GstPipeline:pipeline0/GstTee:t.GstPad:sink: caps = video/x-raw, width=(int)1280, height=(int)1024, framerate=(fraction)25/1, views=(int)1, interlace-mode=(string)progressive, format=(string)RGB, colorimetry=(string)1:1:5:1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw, width=(int)1280, height=(int)1024, framerate=(fraction)25/1, views=(int)1, interlace-mode=(string)progressive, format=(string)RGB, colorimetry=(string)1:1:5:1
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:src: caps = video/x-raw, width=(int)1280, height=(int)1024, framerate=(fraction)25/1, views=(int)1, interlace-mode=(string)progressive, format=(string)RGB, colorimetry=(string)1:1:5:1
/GstPipeline:pipeline0/GstVideoConvert:videoconvert1.GstPad:src: caps = video/x-raw, width=(int)1280, height=(int)1024, framerate=(fraction)25/1, views=(int)1, interlace-mode=(string)progressive, format=(string)BGRx, colorimetry=(string)1:1:5:1
/GstPipeline:pipeline0/GstWaylandSink:sink1.GstPad:sink: caps = video/x-raw, width=(int)1280, height=(int)1024, framerate=(fraction)25/1, views=(int)1, interlace-mode=(string)progressive, format=(string)BGRx, colorimetry=(string)1:1:5:1
/GstPipeline:pipeline0/GstWaylandSink:sink1.GstPad:sink: caps = video/x-raw, width=(int)1280, height=(int)1024, framerate=(fraction)25/1, views=(int)1, interlace-mode=(string)progressive, format=(string)BGRx, colorimetry=(string)1:1:5:1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter2.GstPad:src: caps = video/x-raw, width=(int)300, height=(int)300, framerate=(fraction)25/1, views=(int)1, interlace-mode=(string)progressive, format=(string)RGB, colorimetry=(string)1:1:5:1, pixel-aspect-ratio=(fraction)5/4
/GstPipeline:pipeline0/GstVideoConvert:videoconvert1.GstPad:sink: caps = video/x-raw, width=(int)1280, height=(int)1024, framerate=(fraction)25/1, views=(int)1, interlace-mode=(string)progressive, format=(string)RGB, colorimetry=(string)1:1:5:1
/GstPipeline:pipeline0/GstVideoRate:videorate0.GstPad:src: caps = video/x-raw, width=(int)300, height=(int)300, framerate=(fraction)12/1, views=(int)1, interlace-mode=(string)progressive, format=(string)RGB, colorimetry=(string)1:1:5:1, pixel-aspect-ratio=(fraction)5/4
/GstPipeline:pipeline0/GstCapsFilter:capsfilter3.GstPad:src: caps = video/x-raw, width=(int)300, height=(int)300, framerate=(fraction)12/1, views=(int)1, interlace-mode=(string)progressive, format=(string)RGB, colorimetry=(string)1:1:5:1, pixel-aspect-ratio=(fraction)5/4
/GstPipeline:pipeline0/GstTensorConverter:tensorconverter0.GstPad:src: caps = other/tensor, framerate=(fraction)12/1, dimension=(string)3:300:300:1, type=(string)uint8
/GstPipeline:pipeline0/GstTensorTransform:tensortransform0.GstPad:src: caps = other/tensor, framerate=(fraction)12/1, dimension=(string)3:300:300:1, type=(string)float32
/GstPipeline:pipeline0/GstTensorFilter:tensorfilter0.GstPad:src: caps = other/tensors, format=(string)static, num_tensors=(int)2, framerate=(fraction)12/1, dimensions=(string)"4:1:1917:1\,91:1917:1:1", types=(string)"float32\,float32"
/GstPipeline:pipeline0/GstQueue:queue2.GstPad:src: caps = other/tensors, format=(string)static, num_tensors=(int)2, framerate=(fraction)12/1, dimensions=(string)"4:1:1917:1\,91:1917:1:1", types=(string)"float32\,float32"
/GstPipeline:pipeline0/GstTensorDec:tensordec0.GstPad:src: caps = video/x-raw, format=(string)RGBA, width=(int)1280, height=(int)1024, framerate=(fraction)12/1
/GstPipeline:pipeline0/GstFakeSink:fakesink0.GstPad:sink: caps = video/x-raw, format=(string)RGBA, width=(int)1280, height=(int)1024, framerate=(fraction)12/1
/GstPipeline:pipeline0/GstTensorDec:tensordec0.GstPad:sink: caps = other/tensors, format=(string)static, num_tensors=(int)2, framerate=(fraction)12/1, dimensions=(string)"4:1:1917:1\,91:1917:1:1", types=(string)"float32\,float32"
/GstPipeline:pipeline0/GstQueue:queue2.GstPad:sink: caps = other/tensors, format=(string)static, num_tensors=(int)2, framerate=(fraction)12/1, dimensions=(string)"4:1:1917:1\,91:1917:1:1", types=(string)"float32\,float32"
/GstPipeline:pipeline0/GstTensorFilter:tensorfilter0.GstPad:sink: caps = other/tensor, framerate=(fraction)12/1, dimension=(string)3:300:300:1, type=(string)float32
/GstPipeline:pipeline0/GstTensorTransform:tensortransform0.GstPad:sink: caps = other/tensor, framerate=(fraction)12/1, dimension=(string)3:300:300:1, type=(string)uint8
/GstPipeline:pipeline0/GstTensorConverter:tensorconverter0.GstPad:sink: caps = video/x-raw, width=(int)300, height=(int)300, framerate=(fraction)12/1, views=(int)1, interlace-mode=(string)progressive, format=(string)RGB, colorimetry=(string)1:1:5:1, pixel-aspect-ratio=(fraction)5/4
/GstPipeline:pipeline0/GstCapsFilter:capsfilter3.GstPad:sink: caps = video/x-raw, width=(int)300, height=(int)300, framerate=(fraction)12/1, views=(int)1, interlace-mode=(string)progressive, format=(string)RGB, colorimetry=(string)1:1:5:1, pixel-aspect-ratio=(fraction)5/4
/GstPipeline:pipeline0/GstVideoRate:videorate0.GstPad:sink: caps = video/x-raw, width=(int)300, height=(int)300, framerate=(fraction)25/1, views=(int)1, interlace-mode=(string)progressive, format=(string)RGB, colorimetry=(string)1:1:5:1, pixel-aspect-ratio=(fraction)5/4
/GstPipeline:pipeline0/GstCapsFilter:capsfilter2.GstPad:sink: caps = video/x-raw, width=(int)300, height=(int)300, framerate=(fraction)25/1, views=(int)1, interlace-mode=(string)progressive, format=(string)RGB, colorimetry=(string)1:1:5:1, pixel-aspect-ratio=(fraction)5/4
/GstPipeline:pipeline0/GstVideoScale:videoscale0.GstPad:sink: caps = video/x-raw, width=(int)1280, height=(int)1024, framerate=(fraction)25/1, views=(int)1, interlace-mode=(string)progressive, format=(string)RGB, colorimetry=(string)1:1:5:1
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)1024, framerate=(fraction)25/1, views=(int)1, interlace-mode=(string)progressive, colorimetry=(string)1:4:5:1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)1024, framerate=(fraction)25/1, views=(int)1, interlace-mode=(string)progressive, colorimetry=(string)1:4:5:1
Redistribute latency...
Redistribute latency...
0:00:00.9 / 99:99:99.
(gst-launch-1.0:5011): GLib-CRITICAL **: 10:04:50.627: g_array_remove_index: assertion 'index_ < array->len' failed
0:00:01.1 / 99:99:99.
(gst-launch-1.0:5011): GLib-CRITICAL **: 10:04:50.895: g_array_remove_index: assertion 'index_ < array->len' failed
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:02:17.174674500
Setting pipeline to NULL ...
Total showed frames (664), playing for (0:02:17.174697875), fps (4.841).
Freeing pipeline ...

[greeran]

thanks for the quick response. i tried the example in the document in chapter 10 and this is the failure i get . 

gst-launch-1.0 --no-position v4l2src device=/dev/video3 ! video/x-raw,width=1280,height=1024,framerate=25/1 ! tee name=t t. ! queue max-size-buffers=2 leaky=2 ! imxvideoconvert_g2d ! video/x-raw,width=300,height=300,format=RGBA ! videoconvert ! video/x-raw,format=RGB ! tensor_converter ! tensor_filter framework=tensorflow-lite model=${MODEL} custom=Delegate:External,ExtDelegateLib:libvx_delegate.so ! tensor_decoder mode=bounding_boxes option1=tf-ssd option2=${LABELS} option3=0:1:2:3,50 option4=640:480 option5=300:300 ! mix. t. ! queue max-size-buffers=2 ! imxcompositor_g2d name=mix sink_0::zorder=2 sink_1::zorder=1 ! waylandsink
** Message: 20:34:49.169: accl = cpu
Vx delegate: allowed_cache_mode set to 0.
Vx delegate: allowed_builtin_code set to 0.
Vx delegate: error_during_init set to 0.
Vx delegate: error_during_prepare set to 0.
Vx delegate: error_during_invoke set to 0.

** (gst-launch-1.0:953): CRITICAL **: 20:34:49.184: _check_tensors: assertion 'config->info.num_tensors >= limit' failed

** (gst-launch-1.0:953): CRITICAL **: 20:34:49.184: _check_tensors: assertion 'config->info.num_tensors >= limit' failed
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../git/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.002472625
Setting pipeline to NULL ...
Freeing pipeline ...

[joanxie]

pls refer to the chapter 10 Vision Pipeline with NNStreamer of enclosed file, imx8mp can support object detection with this bsp version

[greeran]

sorry my last test was with the wrong tflite file. i ran it again with the file that i downloaded as the document says and i get the same error as my origial test "failed to invoke on module that is not ready"

gst-launch-1.0 --no-position v4l2src device=/dev/video3 ! video/x-raw,width=1280,height=1024,framerate=25/1 ! tee name=t t. ! queue max-size-buffers=2 leaky=2 ! imxvideoconvert_g2d ! video/x-raw,width=300,height=300,format=RGBA ! videoconvert ! video/x-raw,format=RGB ! tensor_converter ! tensor_filter framework=tensorflow-lite model=${MODEL} custom=Delegate:External,ExtDelegateLib:libvx_delegate.so ! tensor_decoder mode=bounding_boxes option1=tf-ssd option2=${LABELS} option3=0:1:2:3,50 option4=640:480 option5=300:300 ! mix. t. ! queue max-size-buffers=2 ! imxcompositor_g2d name=mix sink_0::zorder=2 sink_1::zorder=1 ! waylandsink
** Message: 20:49:03.903: accl = cpu
Vx delegate: allowed_cache_mode set to 0.
Vx delegate: allowed_builtin_code set to 0.
Vx delegate: error_during_init set to 0.
Vx delegate: error_during_prepare set to 0.
Vx delegate: error_during_invoke set to 0.
WARNING: Fallback unsupported op 32 to TfLite

** (gst-launch-1.0:1105): CRITICAL **: 20:49:03.917: bb_getOutCaps: assertion 'BOX_SIZE == dim4[0]' failed

** (gst-launch-1.0:1105): CRITICAL **: 20:49:03.918: bb_getOutCaps: assertion 'BOX_SIZE == dim4[0]' failed
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

** (gst-launch-1.0:1105): CRITICAL **: 20:49:03.931: bb_getOutCaps: assertion 'BOX_SIZE == dim4[0]' failed

** (gst-launch-1.0:1105): CRITICAL **: 20:49:03.931: bb_getOutCaps: assertion 'BOX_SIZE == dim4[0]' failed

** (gst-launch-1.0:1105): CRITICAL **: 20:49:03.932: bb_getOutCaps: assertion 'BOX_SIZE == dim4[0]' failed

** (gst-launch-1.0:1105): CRITICAL **: 20:49:03.932: bb_getOutCaps: assertion 'BOX_SIZE == dim4[0]' failed

** (gst-launch-1.0:1105): CRITICAL **: 20:49:03.933: bb_getOutCaps: assertion 'BOX_SIZE == dim4[0]' failed

(gst-launch-1.0:1105): GStreamer-CRITICAL **: 20:49:03.933: gst_caps_intersect_full: assertion 'GST_IS_CAPS (caps2)' failed

(gst-launch-1.0:1105): GStreamer-CRITICAL **: 20:49:03.933: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:1105): GStreamer-CRITICAL **: 20:49:03.933: gst_caps_is_empty: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:1105): GStreamer-CRITICAL **: 20:49:03.933: gst_mini_object_make_writable: assertion 'mini_object != NULL' failed

(gst-launch-1.0:1105): GStreamer-CRITICAL **: 20:49:03.933: gst_caps_fixate: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:1105): GStreamer-CRITICAL **: 20:49:03.933: gst_caps_is_subset: assertion 'superset != NULL' failed

(gst-launch-1.0:1105): GStreamer-Base-CRITICAL **: 20:49:03.933: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad tensordec0:sink

** (gst-launch-1.0:1105): CRITICAL **: 20:49:03.933: bb_getOutCaps: assertion 'BOX_SIZE == dim4[0]' failed

** (gst-launch-1.0:1105): CRITICAL **: 20:49:03.934: bb_getOutCaps: assertion 'BOX_SIZE == dim4[0]' failed

** (gst-launch-1.0:1105): CRITICAL **: 20:49:03.934: bb_getOutCaps: assertion 'BOX_SIZE == dim4[0]' failed

(gst-launch-1.0:1105): GStreamer-CRITICAL **: 20:49:03.934: gst_caps_intersect_full: assertion 'GST_IS_CAPS (caps2)' failed

(gst-launch-1.0:1105): GStreamer-CRITICAL **: 20:49:03.934: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:1105): GStreamer-CRITICAL **: 20:49:03.934: gst_caps_is_empty: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:1105): GStreamer-CRITICAL **: 20:49:03.934: gst_mini_object_make_writable: assertion 'mini_object != NULL' failed

(gst-launch-1.0:1105): GStreamer-CRITICAL **: 20:49:03.934: gst_caps_fixate: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:1105): GStreamer-CRITICAL **: 20:49:03.935: gst_caps_is_subset: assertion 'superset != NULL' failed

(gst-launch-1.0:1105): GStreamer-Base-CRITICAL **: 20:49:03.935: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad tensordec0:sink

** (gst-launch-1.0:1105): CRITICAL **: 20:49:04.396: bb_getOutCaps: assertion 'BOX_SIZE == dim4[0]' failed

** (gst-launch-1.0:1105): CRITICAL **: 20:49:04.396: bb_getOutCaps: assertion 'BOX_SIZE == dim4[0]' failed

** (gst-launch-1.0:1105): CRITICAL **: 20:49:04.397: bb_getOutCaps: assertion 'BOX_SIZE == dim4[0]' failed

(gst-launch-1.0:1105): GStreamer-CRITICAL **: 20:49:04.397: gst_caps_intersect_full: assertion 'GST_IS_CAPS (caps2)' failed

(gst-launch-1.0:1105): GStreamer-CRITICAL **: 20:49:04.397: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:1105): GStreamer-CRITICAL **: 20:49:04.397: gst_caps_is_empty: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:1105): GStreamer-CRITICAL **: 20:49:04.397: gst_mini_object_make_writable: assertion 'mini_object != NULL' failed

(gst-launch-1.0:1105): GStreamer-CRITICAL **: 20:49:04.397: gst_caps_fixate: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:1105): GStreamer-CRITICAL **: 20:49:04.397: gst_caps_is_subset: assertion 'superset != NULL' failed

(gst-launch-1.0:1105): GStreamer-Base-CRITICAL **: 20:49:04.397: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad tensordec0:sink
ERROR: Invoke called on model that is not ready.

** (gst-launch-1.0:1105): CRITICAL **: 20:49:04.404: Failed to invoke

** (gst-launch-1.0:1105): CRITICAL **: 20:49:04.406: /usr/lib/libnnstreamer-single.so(_backtrace_to_string+0x40) [0xffff780373c0]/usr/lib/gstreamer-1.0/libnnstreamer.so(+0x3d9ac) [0xffff7809d9ac]/usr/lib/libgstbase-1.0.so.0(+0x482a4) [0xffff8bc682a4]/usr/lib/libgstbase-1.0.so.0(+0x47964) [0xffff8bc67964]/usr/lib/libgstreamer-1.0.so.0(+0x8f368) [0xffff8c29f368]/usr/lib/libgstreamer-1.0.so.0(+0x911f8) [0xffff8c2a11f8]/usr/lib/gstreamer-1.0/libnnstreamer.so(+0x1fba0) [0xffff7807fba0]/usr/lib/libgstreamer-1.0.so.0(+0x8f368) [0xffff8c29f368]/usr/lib/libgstreamer-1.0.so.0(+0x911f8) [0xffff8c2a11f8]/usr/lib/libgstbase-1.0.so.0(+0x47a78) [0xffff8bc67a78]/usr/lib/libgstreamer-1.0.so.0(+0x8f368) [0xffff8c29f368]/usr/lib/libgstreamer-1.0.so.0(+0x911f8) [0xffff8c2a11f8]/usr/lib/libgstbase-1.0.so.0(+0x47a78) [0xffff8bc67a78]/usr/lib/libgstreamer-1.0.so.0(+0x8f368) [0xffff8c29f368]/usr/lib/libgstreamer-1.0.so.0(+0x911f8) [0xffff8c2a11f8]/usr/lib/libgstbase-1.0.so.0(+0x47a78) [0xffff8bc67a78]/usr/lib/libgstreamer-1.0.so.0(+0x8f368) [0xffff8c29f368]/usr/lib/libgstreamer-1.0.so.0(+0x911f8) [0xffff8c2a11f8]/usr/lib/libgstbase-1.0.so.0(+0x47a78) [0xffff8bc67a78]/usr/lib/libgstreamer-1.0.so.0(+0x8f368) [0xffff8c29f368]

** (gst-launch-1.0:1105): CRITICAL **: 20:49:04.406: Calling invoke function (inference instance) of the tensor-filter subplugin (tensorflow-lite for ./ssd_mobilenet_v2_coco_quant_postprocess.tflite) has failed with error code (-1).

Redistribute latency...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../git/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason error (-5)
Execution ended after 0:00:00.512036250
Setting pipeline to NULL ...
ERROR: from element /GstPipeline:pipeline0/GstQueue:queue0: Internal data stream error.
Additional debug info:
../git/plugins/elements/gstqueue.c(992): gst_queue_handle_sink_event (): /GstPipeline:pipeline0/GstQueue:queue0:
streaming stopped, reason error (-5)
Total showed frames (2), playing for (0:00:00.512028125), fps (3.906).
Freeing pipeline ...