Model Size is too large

[wklee]

Hi everyone,

I came across a strange problem. I manually trained a small object detection CNN and successfully created the .tflite file. Then I use the eIQ toolkit to convert it into NPU-supported version (shown below). 

We can see that the input and output are in int8, which is great. The total size of this model is just 68KB.

However, when I loaded it to the MCXN947 board, I found that it was bloated up to 512KB!!!

I was using the face detection example (dm-multiple-face-detection-on-mcxn947) from Application Code Hub, with small modifications on the MODEL_GetOpsResolver so that it loads the necessary operations.

Anyone sees this problem before? Did I miss out anything else?

*When I tried with other CNN models, it does not have such a problem.

Thanks!!!

[Harry_Zhang]

Hi @wklee 

When you export the model, do you select the "Export Quantized Model"?

BR

Hang

[wklee]

Dear Hang,

Yes, you can check both the attached file.

helmet_quantized.tflite --> Trained and quantized to 8-bit using python code on my host PC. I did not use the eIQ toolkit to train it.

helmet_quantized_converted.tflite --> This is the converted file using eIQv1.13. I use the eIQ Model Tool to open the helmet_quantized.tflite, then convert to NPU file.

I am using FRDM-MCXN947 board.

No quantization was selected because it was already quantized. You can see that the converted version also have 8-bit input and output. It is only ~67KB in size.

However, when I download the latest face detection demo and replace the original model with my helmet_quantized_converted.tflite , the size is much larger than expected.

*You may try converting my helmet_quantized.tflite, see if you get the same problem.

Thanks.

[Harry_Zhang]

Hi @wklee 

This is the compilation size of the face_detect.tflite model.

This is the compilation size of the helmet_quantized_converted1.tflite model.

BR

Hang

[wklee]

Dear Hang,

The file size for helmet_quantized_converted is 68KB:

But the compile size becomes 406.5KB, what could be the problem?

There are only ~70K parameters in this model.

Thanks.

[Harry_Zhang]

Hi @wklee 

You can check the liker script.

It includes the *(.ezh_code) and *(.model_input_buffer).

So the compile size becomes 406.5KB.

BR

Harry

[wklee]

I don't think that's the reason.

The ezh_code is very small, only 512*4=2KB.

I verify this point by checking the original compilation:

Then modify the ezh_code to only ezh_code[2], which is 8 bytes.

You can see that the SRAMX region is only changed a little.

I think that the problem should be some problems during the conversion from a trained model to the NPU version. If the trained model is in int8 and contains only 70K parameters, there is no reason for the size to grow up to so big (400KB+).

Thanks.

[wklee]

Sorry for not making the problems clear.

In my first post, I mentioned that the model that I trained only has ~67 K parameters, so I expect that after int8 quantization the size should be around 70KB. However, when it is loaded onto the MCU (FRDM-MCXN947), I see the size grows to ~512KB, which is not what I expected, see the serial port information below.

 

[Harry_Zhang]

Hi @wklee 

I tried to convert this model in two ways and ran it on mcxn947, both of which encountered the same error. I think it's a problem with the model itself.

If your model includes operations unsupported by the NPU, increasing the size.

How did you train this model?

BR

Harry

[wklee]

The code can be found in the attachment. It is a customized CNN that performs object detection (helmet).

The training dataset is here:

https://drive.google.com/file/d/1xXwR6-OhXmePOGtyy9Yxn0bdhv3gmu4y/view?usp=sharing

Thanks.

[Harry_Zhang · Accepted solution]

Hi @wklee 

I want to train a model trough your dataset. I downloaded your dataset and train.py.

How do you process the data?

BR

Harry

[wklee]

Hi Harry Zhang,

I choose to compress the model into very small so that it can fit into the MCXN947 MCU. Although this does not solve the issue, it works for my project.

Thanks for your time and effort to support.