Need guidance: YOLOv8 output shape mismatch on NNStreamer (i.MX8MP) + pipeline design question (mini

[SiddavatamVishnu]

HARDWARE AND SOFTWARE DETAILS

I.MX8MPlus and Linux BSP LF6.12.34_2.1.0

Goal

I’m building a command-line pipeline (no GUI) where inference, overlay, and display all run in GStreamer and NNStreamer on an i.MX8MP.

Current experiment (command line)

I tried this pipeline:

GST_DEBUG=GStreamer:4,tensor_filter:6,tensor_transform:6,tensor_decoder:7 \ gst-launch-1.0 --no-position \ v4l2src device=/dev/video4 num-buffers=200 ! \ video/x-raw,width=1920,height=1080,format=NV12,framerate=30/1 ! \ imxvideoconvert_g2d ! \ video/x-raw,width=320,height=320,format=RGBA ! \ videoconvert ! \ video/x-raw,width=320,height=320,format=BGR ! \ tensor_converter ! \ tensor_transform mode=arithmetic option=typecast:int8,add:-128 ! \ tensor_filter framework=tensorflow-lite model=${MODEL} custom=Delegate:External,ExtDelegateLib:${VX_LIB} ! \ tensor_transform mode=arithmetic option=typecast:float32,add:128.0,mul:0.004982381127774715 ! \ tensor_transform mode=transpose option=1:0:2 ! \ tensor_decoder mode=bounding_boxes option1=yolov8 option2=${LABELS} option3=0 option4=1920:1080 option5=320:320 ! \ cairooverlay name=overlay ! \ videoconvert ! \ autovideosink

log file link LINK 

Problem

My YOLOv8 TFLite model outputs (1, 7, 2100), but NNStreamer on i.MX8MP expects 7 × 2100 × 1.

I received this explanation:

The YOLOv8 TFLite model outputs (1,7,2100), but NNStreamer’s YOLOv8 decoder for i.MX8MP expects 7×2100×1. This BSP version only supports transpose on 4D tensors, so the model output needs dequantization, reshape to (1,7,2100,1), then transpose.

• input: int8 [1, 320, 320, 3]

• output: int8 [1, 7, 2100]

• scale/zero point

• output correctly contains 3 classes + 4 bbox values

---

Current (slow) approach

Right now the application flow is:

1. GStreamer → BGR → OpenCV

2. NPU inference

3. OpenCV post-processing

4. Back to RTSP pipeline

This causes multiple software videoconvert, and in ideal conditions we reach only ~20 FPS, although the model alone can run 60+ FPS.

---

Proposed new approach

I want to split the pipeline:

Path A — Inference

Convert NV12 → BGR only here
Run NNStreamer

Path B — Overlay + Display

Keep original NV12/YUY2 frames
Draw bounding boxes directly on NV12 (preferably using hardware)

→ Feed NV12 to encoder / RTSP
→ Avoid software videoconvert completely

I’d first like to prototype this using pure gst-launch, then apply the approach in Python (possibly using OpenGL for NV12 overlay).

---

What I need help on

1. How to reshape/transpose (1,7,2100) TFLite output into the format required by NNStreamer’s YOLOv8 decoder on i.MX8MP

   • Any working example using only tensor_filter / transform / decoder?

   • Is there a known workaround for the 3-D output?

2. Best practice for overlay on NV12/YUY2

   • Any NNStreamer-friendly way to draw boxes directly on NV12?

   • Recommended elements (cairooverlay on NV12? OpenGL? v4l2convert? imxvideoconvert_g2d overlays?)

3. General advice: Is the split-pipeline (inference on BGR, overlay on NV12) a reasonable architectural direction on i.MX8MP?

---

[Bio_TICFSL]

Hello,

The issue stems from a mismatch between your YOLOv8 TFLite model output shape (1,7,2100) and the format expected by NNStreamer's YOLOv8 decoder on i.MX8MP (7×2100×1). This is happening because:

1. Your current BSP version (LF6.12.34_2.1.0) supports transpose operations only on 4D tensors
2. The model output needs reshaping and transposing to match the decoder's expectations

## Recommended Solution

For the tensor transformation, you need to apply:
1. Dequantization (if using quantized model)
2. Reshape the output from (1,7,2100) to (1,7,2100,1)
3. Transpose the tensor to the required format (7×2100×1)

## Pipeline Optimization

Your proposed architecture (splitting the pipeline) is a sound approach:
- Convert NV12 → BGR only for inference
- Keep original NV12/YUY2 frames for display/encoding
- Overlay detection results directly on NV12 using hardware acceleration

This will eliminate software videoconvert operations and achieve better performance.

## Recommended Elements for NV12 Pipeline

For drawing bounding boxes directly on NV12:
- Use `imxvideoconvert_g2d` with overlay capability
- Alternative: `cairooverlay` can work with NV12 but may require format adaptation

Example pipeline structure:
```
v4l2src → NV12 → tee → branch1: convert to BGR → inference → detection results
branch2: original NV12 → imxvideoconvert_g2d (with overlay) → encoder/display
```

This approach should significantly improve performance beyond your current ~20 FPS limit, leveraging the NPU's 60+ FPS capability by eliminating unnecessary format conversions.

Regards

[SiddavatamVishnu]

Thanks for the Quick response, I had tried to use the reshape in the tensor_transform before, but it returned an error.
root@imx8mpevk:~# export MODEL=/root/rtsp/testing1/saved_model_triding_320.tflite
root@imx8mpevk:~# export LABELS=/root/rtsp/testing1/labels.txt
root@imx8mpevk:~# gst-launch-1.0 --no-position v4l2src device=/dev/video4 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! imxvideoconvert_g2d ! video/x-raw,width=320,height=320,format=RGBA ! videoconvert ! video/x-raw,width=320,height=320,format=BGR ! tensor_converter ! tensor_transform mode=arithmetic option=typecast:int8,add:-128 ! tensor_filter framework=tensorflow-lite model=${MODEL} ! tensor_transform mode=arithmetic option=typecast:float32,add:128.0,mul:0.004982381 ! tensor_transform mode=reshape option=1:7:2100:1 ! tensor_transform mode=transpose option=1:0:2:3 ! tensor_decoder mode=bounding_boxes option1=yolov8 option2=${LABELS} option3=0 option4=1920:1080 option5=320:320 ! cairooverlay name=overlay ! videoconvert ! autovideosink
** Message: 14:56:17.877: accl = cpu

** (gst-launch-1.0:1335): CRITICAL **: 14:56:17.931: bb_getOutCaps: assertion 'config->info.info[0].type == _NNS_FLOAT32' failed

** (gst-launch-1.0:1335): CRITICAL **: 14:56:17.931: bb_getOutCaps: assertion 'config->info.info[0].type == _NNS_FLOAT32' failed

** (gst-launch-1.0:1335): CRITICAL **: 14:56:17.931: bb_getOutCaps: assertion 'config->info.info[0].type == _NNS_FLOAT32' failed

** (gst-launch-1.0:1335): CRITICAL **: 14:56:17.935: bb_getOutCaps: assertion 'config->info.info[0].type == _NNS_FLOAT32' failed

** (gst-launch-1.0:1335): CRITICAL **: 14:56:17.938: bb_getOutCaps: assertion 'config->info.info[0].type == _NNS_FLOAT32' failed

** (gst-launch-1.0:1335): CRITICAL **: 14:56:17.939: bb_getOutCaps: assertion 'config->info.info[0].type == _NNS_FLOAT32' failed
WARNING: erroneous pipeline: could not set property "mode" in element "tensor_transform" to "reshape"
root@imx8mpevk:~#

Could you please suggest changes to the pipeline to match the expected formats and also for overlaying on the nv12.

Or do i need to downgrade or upgrade my linux bsp version to match them.

If possible please test the pipeline and provide back.

Thanks and Regards
S Vishnu

[Bio_TICFSL]

Hi,

Yes, you need downgrade your linux BSP. I guess with that will work because We have not yet tested pipeline in Yolov8

Regards

[SiddavatamVishnu · Accepted solution]

I have a Update, i have used the mode dimension change (dimchg ) in the tensor_transform element to match the expections of the tensor_decoder.  
 
....  ! tensor_filter framework=tensorflow-lite model=../../vaishnavi/model_calibrated_int8_og_320.tflite custom=Delegate:External,ExtDelegateLib:libvx_delegate.so ! tensor_transform mode=arithmetic option=typecast:float32,add:128.0,mul:0.004982381 ! tensor_transform mode=dimchg option=0:1 ! tensor_decoder mode=bounding_boxes option1=yolov8 option2=labels_over.txt option3=0 option4=1920:1080 option5=320:320 ! ......



Thanks for the replies.

Thanks and Regards
Siddavatam Vishnu