topic Re: Model Size is too large in MCX Microcontrollers

Model Size is too large

wklee — Tue, 12 Nov 2024 03:21:21 GMT

Hi everyone,

I came across a strange problem. I manually trained a small object detection CNN and successfully created the .tflite file. Then I use the eIQ toolkit to convert it into NPU-supported version (shown below).

We can see that the input and output are in int8, which is great. The total size of this model is just 68KB.

However, when I loaded it to the MCXN947 board, I found that it was bloated up to 512KB!!!

I was using the face detection example (dm-multiple-face-detection-on-mcxn947) from Application Code Hub, with small modifications on the MODEL_GetOpsResolver so that it loads the necessary operations.

Anyone sees this problem before? Did I miss out anything else?

*When I tried with other CNN models, it does not have such a problem.

Thanks!!!

Re: Model Size is too large

Harry_Zhang — Wed, 13 Nov 2024 08:18:55 GMT

Hi @wklee

When you export the model, do you select the "Export Quantized Model"?

Hang

Re: Model Size is too large

wklee — Wed, 13 Nov 2024 09:01:20 GMT

Dear Hang,

Yes, you can check both the attached file.

helmet_quantized.tflite --> Trained and quantized to 8-bit using python code on my host PC. I did not use the eIQ toolkit to train it.

helmet_quantized_converted.tflite --> This is the converted file using eIQv1.13. I use the eIQ Model Tool to open the helmet_quantized.tflite, then convert to NPU file.

I am using FRDM-MCXN947 board.

No quantization was selected because it was already quantized. You can see that the converted version also have 8-bit input and output. It is only ~67KB in size.

However, when I download the latest face detection demo and replace the original model with my helmet_quantized_converted.tflite , the size is much larger than expected.

*You may try converting my helmet_quantized.tflite, see if you get the same problem.

Thanks.

Re: Model Size is too large

Harry_Zhang — Thu, 14 Nov 2024 03:53:42 GMT

Hi @wklee

This is the compilation size of the face_detect.tflite model.

This is the compilation size of the helmet_quantized_converted1.tflite model.

Hang

Re: Model Size is too large

wklee — Thu, 14 Nov 2024 05:46:59 GMT

Dear Hang,

The file size for helmet_quantized_converted is 68KB:

But the compile size becomes 406.5KB, what could be the problem?

There are only ~70K parameters in this model.

Thanks.

Re: Model Size is too large

Harry_Zhang — Fri, 15 Nov 2024 09:19:44 GMT

Hi @wklee

You can check the liker script.

It includes the *(.ezh_code) and *(.model_input_buffer).

So the compile size becomes 406.5KB.

Harry

Re: Model Size is too large

wklee — Thu, 28 Nov 2024 07:54:12 GMT

I don't think that's the reason.

The ezh_code is very small, only 512*4=2KB.

I verify this point by checking the original compilation:

Then modify the ezh_code to only ezh_code[2], which is 8 bytes.

You can see that the SRAMX region is only changed a little.

I think that the problem should be some problems during the conversion from a trained model to the NPU version. If the trained model is in int8 and contains only 70K parameters, there is no reason for the size to grow up to so big (400KB+).

Thanks.

Re: Model Size is too large

wklee — Fri, 29 Nov 2024 08:43:57 GMT

Sorry for not making the problems clear.

In my first post, I mentioned that the model that I trained only has ~67 K parameters, so I expect that after int8 quantization the size should be around 70KB. However, when it is loaded onto the MCU (FRDM-MCXN947), I see the size grows to ~512KB, which is not what I expected, see the serial port information below.

Re: Model Size is too large

Harry_Zhang — Mon, 02 Dec 2024 10:14:25 GMT

Hi @wklee

I tried to convert this model in two ways and ran it on mcxn947, both of which encountered the same error. I think it's a problem with the model itself.

If your model includes operations unsupported by the NPU, increasing the size.

How did you train this model?

Harry

Re: Model Size is too large

wklee — Tue, 03 Dec 2024 07:49:46 GMT

The code can be found in the attachment. It is a customized CNN that performs object detection (helmet).

The training dataset is here:

https://drive.google.com/file/d/1xXwR6-OhXmePOGtyy9Yxn0bdhv3gmu4y/view?usp=sharing

Thanks.

Re: Model Size is too large

Harry_Zhang — Mon, 09 Dec 2024 09:10:21 GMT

Hi @wklee

I want to train a model trough your dataset. I downloaded your dataset and train.py.

How do you process the data?

Harry

Re: Model Size is too large

wklee — Thu, 19 Dec 2024 03:14:20 GMT

Hi Harry Zhang,

I choose to compress the model into very small so that it can fit into the MCXN947 MCU. Although this does not solve the issue, it works for my project.

Thanks for your time and effort to support.