eiq tflite model for imx8mplus

[kamalesh]

I'm currently working on converting a TensorFlow object detection model to a TensorFlow Lite format for inference on imx8mplus. I used the EIQ Toolkit for the conversion which is installed in ubuntu 20.04. The base model used is ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8.

Here are the steps I followed using the EIQ Model Tool:

Selected Model Tool in the EIQ GUI.

Loaded the TensorFlow saved_model.pb file of the model.

Converted the model to TensorFlow Lite (.tflite) using the eiq-converter-tflite.

Enabled quantization with the following settings:

Quantization type: Per Tensor

Input type: uint8

Output type: uint8

Quantization normalization: Unsigned

Calibration dataset: 10 samples from the COCO dataset

Provided a compatible labels.txt file.

After the conversion, when running inference with the TFLite model, I encountered the following error:
IndexError: list index out of range

Post-processing Inference Code:

interpreter.invoke()
labels = load_labels(args.label_file)

scores = np.squeeze(interpreter.get_tensor(output_details[0]['index']))
boxes = np.squeeze(interpreter.get_tensor(output_details[1]['index'])[0])
num_detections = np.squeeze(interpreter.get_tensor(output_details[2]['index'])[0])
classes = np.squeeze(interpreter.get_tensor(output_details[3]['index'])[0])

for i in range(10):
if scores[i] > 0.7:
ymin, xmin, ymax, xmax = boxes[i]
xmin = int(xmin * w0)
ymin = int(ymin * h0)
xmax = int(xmax * w0)
ymax = int(ymax * h0)

class_id = classes[i]
cv2.rectangle(frame, (xmin, ymin), (xmax, ymax), (0, 255, 0), 2)
label = f"{labels[class_id]} {scores[i]:.2f}"
cv2.putText(frame, label, (xmin, max(10, ymin - 5)), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 0, 0), 2)

Is there anything incorrect or missing in the conversion process or the inference/post-processing code that could be causing this IndexError?

I would appreciate any insights or suggestions on how to debug or fix this issue.

Thank you!

[Zhiming_Liu]

Hi,

This error is not about model, your code is trying to get index which is out of range. Please check the code or you can find other reference like this: https://github.com/zafarRehan/object_detection_COCO

Best Regards,
Zhiming

[kamalesh]

I understand that the IndexError is related to accessing indices that are not available. What I am trying to confirm is the structure of the outputs produced by the model after conversion with the EIQ Toolkit.

Could you please clarify:

What is the expected order and shape of the output tensors for SSD models after EIQ conversion?

Is there a reference example of the correct post-processing steps to use with EIQ-converted TFLite models?

Could the small calibration dataset used during quantization have any impact on the outputs?

This information would help me adjust my inference code accordingly. I would really appreciate your guidance on the correct way to handle the outputs from EIQ-converted models.

[Zhiming_Liu]

Hi,

Quantization will not affect the structure, you can find eiq model example from here.

https://github.com/NXP/eiq-model-zoo/blob/main/tasks/vision/object-detection/ssdlite-mobilenetv2/exa...

Best Regards,
Zhiming

[kamalesh]

Thank you for sharing the links. I noticed a key difference between the NXP TFLite model and my model — specifically in their output structures. The NXP TFLite model produces only bounding boxes and class outputs, whereas my model provides bounding boxes, scores, classes, and the number of detections.

NXP Tflite model output:
Outputs: [
{'name': 'concat', 'index': 299, 'shape': array([1, 1917, 1, 4], dtype=int32)},
{'name': 'concat_1', 'index': 300, 'shape': array([1, 1917, 91], dtype=int32)}
]

My model output:
Outputs: [
{'name': 'StatefulPartitionedCall:1', 'index': 339, 'shape': array([1, 10], dtype=int32)},
{'name': 'StatefulPartitionedCall:3', 'index': 337, 'shape': array([1, 10, 4], dtype=int32)},
{'name': 'StatefulPartitionedCall:0', 'index': 340, 'shape': array([1], dtype=int32)},
{'name': 'StatefulPartitionedCall:2', 'index': 338, 'shape': array([1, 10], dtype=int32)}
]
It would be greatly helpful if you could provide suggestions regarding the conversion aspect — particularly on how to align my model’s output structure to match that of the NXP TFLite model.

[Zhiming_Liu]

Hi,

You can use onnx2tf to repalce eiq conversion and quantization, onnx2tf enables conversion and quantization under your complete control.

Best Regards,
Zhiming

[kamalesh]

The reference link provided is based on the Tensorflow1 object detection model. However, I am specifically looking for the Tensorflow2 object detection model. it would be greatly helpful if you could provide the appropriate guidance on converting a TensorFlow 2 object detection model to TensorFlow Lite, ensuring compatibility with the inference code.

[Zhiming_Liu]

Hi,
Please refer these existing tutorials

https://colab.research.google.com/github/marcin-ch/Object_Detection_SSD_MobilenetV3_TFLite/blob/main...

https://github.com/google-coral/tflite/blob/master/python/examples/detection/README.md

https://github.com/tensorflow/examples/blob/master/lite/examples/object_detection/raspberry_pi/READM...

Best Regards,
Zhiming