Problems in identifying model output

[ddm]

Hi, everyone!

I'm using the eIQ toolkit to train a human object detection model – this is only a test problem to understand the main concepts. Mainly I used the ssd_mobilenet_v3 model and eIQ_Toolkit_v1.5.2.179_b221005 (by using eIQ_Toolkit_v1.6.9.310_b230201 each time when I want to generate the tensorflow lite model I got the error "Conversion Failed (2, 'No such file or directory)")

After I trained the model, I exported the model to a tensorflow lite model able to be executed on the NPU embedded on i.MX 8M NavQ Plus board.

Mainly, I have at the output a table of a little more than 2000 lines (2034) and 6 columns.

My problem is that I'm unable to understand the significance of the outputs. Searching on the web, I found that, most likely, the first [0-3] location is the detection coordinates (the box), and the following ones represent the class and the score or the background score and the human class score.

However, any attempt to display the rectangles with the detected subjects (both as presented above and other combinations) did not yield any correct results. However, in eIQ, the human subjects are correctly fitted when validating the model.

Under these conditions, I did the following test, I generated a model to be run within the CPU of the board (thus supporting float32) in the hope that the class scores would be numbers between 0 and 1 and I would correctly identify at least one or two outputs of the model.

=======================================================================

The first model was without quantization with float32 outputs, and I got:

[{'name': 'StatefulPartitionedCall:0',
 'index': 392,
  'shape': array([   1, 2034,    6]),
  'shape_signature': array([   1, 2034,    6]),
  'dtype': <class 'numpy.float32'>,
  'quantization': (0.0, 0),
  'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0},
  'sparsity_parameters': {}}]

By using:

interpreter.allocate_tensors()
output_details = interpreter.get_output_details()
print ('Output: ', output_details)

and the following results:

[[ 2.3530433  -2.180804   -0.27817088 -0.8394966   0.547423   -1.4488211 ]
 [ 2.3681636  -2.1536398   0.24990132  0.47419435  0.5186531   1.0320079 ]
 [ 2.429813   -2.4287374   0.02726804 -0.71517277 -0.62933177 -0.02892369]
 ...

 [ 2.2256145  -1.2317237  -0.8156106   0.24284875 -0.39259493 -0.24716848]
 [ 1.7052491  -1.8843162  -0.49732697  0.2286227   0.3193972   0.28404716]
 [ 2.0460138  -0.8108859   0.28477466  0.14418325 -0.18408567 -0.16082   ]]

As a result of the following lines of python:

output_data = interpreter.get_tensor(output_details[0]['index'])[0]
print('output', output_data)

===========================================================================

Or by using quantization, uint8 as input and float32 as output:

[{'name': 'StatefulPartitionedCall:0',
 'index': 396, 'shape': array([   1, 2034,    6]),
 'shape_signature': array([   1, 2034,    6]),
 'dtype': <class 'numpy.float32'>,
 'quantization': (0.0, 0),
 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0},
'sparsity_parameters': {}}]

 [[ 2.8598952  -2.239677   -0.24119598 -0.6891314   0.75804454 -1.3093497 ]

 [ 2.170764   -2.1018507   0.13782628  0.17228284  0.41347885  0.6891314 ]

 [ 3.0321782  -2.3085902   0.06891315 -0.51684856 -0.51684856 -0.4479354 ]

 ...

 [ 2.1018507  -1.3438063  -0.723588    0.3445657  -0.24119598 -0.03445657]

 [ 1.7572851  -1.6194588  -0.31010914  0.4479354  -0.10336971  0.6546748 ]

 [ 1.6194588  -1.3782628   0.13782628 -0.10336971  0.13782628 -0.10336971]]

=============================================================================

Questions:

1. Apart from the first column, in all other columns, I have negative numbers, and in the first column, I have numbers greater than 1. So what is/are the column(s) of human subject detection scores? What columns show the rectangle(s) with the detected subject(s)?
2. Assuming the first four columns indicate the detected subject, how do I draw this rectangle?