Hi @Ramson,
In principle what you said may sound good to me. I trained an object detector model with the eIQ portal based on the ssd_mobilenet_V3 and my own data set. My model considers two possible classes as output (plus the background?). The output data provides consequently a [1,1,7] tensor:
[1,1,4]-> object location
[1,1,5]-> background score as you suggest or class type as suggested here by MarcinChelminsk (09-06-2021 06:45 AM).
[1,1,6-7]-> corresponding to scores of my two main classes
However, after invoking the inference, I am not able to interpret the output at all. My model, same as yours, results on [1,2034,7] 2034 possible objects found.
...
interpreter = tflite.Interpreter('detection-balanced-mcu-2021-11-15T14-20-27.767Z_in-uint8_out-uint8_channel_mlir_ptq.tflite')
interpreter.allocate_tensors()input_details = interpreter.get_input_details()
print("INPUT DETAILS", input_details)
output_details = interpreter.get_output_details()
print("OUTPUT DETAILS", output_details)
INPUT DETAILS [{'name': 'input_1', 'index': 446, 'shape': array([ 1, 320, 320, 3], dtype=int32), 'shape_signature': array([ -1, 320, 320, 3], dtype=int32), 'dtype': <class 'numpy.uint8'>, 'quantization': (0.007843137718737125, 127), 'quantization_parameters': {'scales': array([0.00784314], dtype=float32), 'zero_points': array([127], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}]
OUTPUT DETAILS [{'name': 'Identity', 'index': 447, 'shape': array([1, 1, 7], dtype=int32), 'shape_signature': array([-1, -1, 7], dtype=int32), 'dtype': <class 'numpy.uint8'>, 'quantization': (0.1662425845861435, 150), 'quantization_parameters': {'scales': array([0.16624258], dtype=float32), 'zero_points': array([150], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}]
# NxHxWxC, H:1, W:2
height = input_details[0]['shape'][1]
width = input_details[0]['shape'][2]
img = Image.open('imgSource/armas(1).jpg').resize((height,width ))# add N dim
input_data = np.expand_dims(img, axis=0)
interpreter.invoke()output_details = interpreter.get_output_details()
print("OUTPUT DETAILS", output_details)
OUTPUT DETAILS [{'name': 'Identity', 'index': 447, 'shape': array([ 1, 2034, 7], dtype=int32), 'shape_signature': array([-1, -1, 7], dtype=int32), 'dtype': <class 'numpy.uint8'>, 'quantization': (0.1662425845861435, 150), 'quantization_parameters': {'scales': array([0.16624258], dtype=float32), 'zero_points': array([150], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}]
output_data = interpreter.get_tensor(output_details[0]['index'])
results = np.squeeze(output_data)
#lets print the first 5 result values as an example
for i in range(5):
print(results[i])
[169 120 138 146 152 145 128]
[175 125 125 148 157 148 154]
[168 124 138 151 157 153 137]
[169 125 128 143 141 134 133]
[177 125 126 151 149 141 149]
How should I interpret these values?
Hi @Ramson!
Thank you for your response and the correction about the order of the output.
In my opinion the output size changes from [1,1,7] to [1,2034,7] after invoking because of some kind of maximum_objects_detection parameter set during the training phase. I could not find it anywhere, but it must be standard as I got exactly the same value as you did. Also, if you convert the model using the toco option (instead of mlir) you will get this value in the output_details even before the invoke.
I tried to interpret the outputs based on the current suppositions but I am yet not able to interpret them. This is what I changed:
# NxHxWxC, H:1, W:2
height = input_details[0]['shape'][1]
width = input_details[0]['shape'][2]
img_name = 'imgSource/armas(17).jpg'
img = Image.open(img_name).resize((height,width ))
print(img_name)
# add N dim
input_data = np.expand_dims(img, axis=0)
#Test if the array has the expected dimmensions (1, 320, 320, 3)
print(input_data.shape)interpreter.set_tensor(input_details[0]['index'], input_data)
startTime = datetime.now()
interpreter.invoke()
delta = datetime.now() - startTime
print("Warm-up time:", '%.1f' % (delta.total_seconds() * 1000), "ms\n")input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()#sort the output array to obtain the best Background, knife and pistol classes scores.
output_data = interpreter.get_tensor(output_details[0]['index'])
results = np.squeeze(output_data)
ind = np.argsort(results, axis=0)[::-1]#print best 5 Background score results and its location:
print("label[0]:" + labels[0])
im_w, im_h = img.size
for i in range(5):
loc = (results[ind[i]][0]) / 255.0
(ymin, xmin, ymax, xmax) = loc[3:]
(left, right, top, bottom) =(xmin*im_w, xmax*im_w, ymin*im_h, ymax*im_h)
print('Score: {}. Loc: {}, {}, {}, {}'.format(loc[0], left, right, top, bottom))#print best 5 knife score results and its location:
print("label[1]:" + labels[1])
for i in range(5):
loc = (results[ind[i]][1]) / 255.0
(ymin, xmin, ymax, xmax) = loc[3:]
(left, right, top, bottom) =(xmin*im_w, xmax*im_w, ymin*im_h, ymax*im_h)
print('Score: {}. Loc: {}, {}, {}, {}'.format(loc[1], left, right, top, bottom))#print best 5 pistol score results and its location:
print("label[2]:" + labels[2])
for i in range(5):
loc = (results[ind[i]][2]) / 255.0
(ymin, xmin, ymax, xmax) = loc[3:]
(left, right, top, bottom) =(xmin*im_w, xmax*im_w, ymin*im_h, ymax*im_h)
print('Score: {}. Loc: {}, {}, {}, {}'.format(loc[2], left, right, top, bottom))
The accuracy of this quantized model is 79.95% according to the validation in the eIQ toolkit . Here however, the maximum accuracy obtained after inference is 0.6235... despite using on purpose an image from the training/validation dataset ( 'imgSource/armas(17).jpg')
In addition, the boxes obtained (object locations) do also not make any sense:
- The input image size is (1300x860) which is first resized to the model´s input size (320x320).
- The output values are then divided by 255.0 (output tensor is of type uint8) and multiplied by the corresponding image sizes: ``(left, right, top, bottom) =(xmin*im_w, xmax*im_w, ymin*im_h, ymax*im_h)´´
I am doing something wrong here, as the boxes are always in pixel ranges between [160 < x < 220] for both height and width (left, right, top, bottom values in the code) which can clearly not be correct.
Any idea, why is this happening or how can I interpret it correctly?
Thank you very much in advance.
best regards,
Jon
Hi Ramson,
Thanks for the clear explanation. I did the test with the non quantized model as suggested:
The changes performed:
Best regards,
Jon
Dear @Ramson,
I included as suggested the first 5 result values as an example, as well as the model's input width and height:
I hope you can help me understand these output values.
Thanks and best regards,
Jon
