I am trying to quantize VGG19 to run it on i.MX 8M Plus NPU and I need multiple convolutional outputs. When I try to quantize it with the following code, input scale is like 3.9e-9 and the responses for different inputs are the same.
def create_vgg19_c(batch_normalization=False, required_layers=None):
base_model = VGG19(include_top=False, weights='imagenet', input_shape=(256, 256, 3))
base_model.trainable = True
if required_layers is None:
required_layers = [2, 5, 8, 13, 18, 19]
outputs = [base_model.layers[i].output for i in required_layers]
model = Model(inputs=base_model.input, outputs=outputs)
return model
model = create_vgg19_c()
quant_aware_model = tfmot.quantization.keras.quantize_model(model)
def representative_data_gen():
_, h, w, _ = quant_aware_model.input_shape
samples = 100
for i in range(samples):
img_in = np.random.uniform(0, 255, (1, h, w, 3)).astype('float32')
img_in = tf.keras.applications.vgg19.preprocess_input(img_in)
yield [img_in]
converter = tf.lite.TFLiteConverter.from_keras_model(quant_aware_model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_data_gen
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.experimental_new_converter = True
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
quantized_tflite_model = converter.convert()
I tried to normalize the values between [-128, 127] for int8 and between [-1,1] for float32. I changed the parameters of the converter, I used just optimizer but scale and same results didn't changed. Also I changed the preprocess function but nothing changed. I want to quantize pretrained VGG19 backbone with specified outputs.
