I run model from sample tflm_cifar10 on MIMRT700 (NPU model). When building the program, I could see the model's size and correspond region size.
In many cases, the region size is 1 MB. As my understanding, the model's size is limited at 1 MB. Is that right?
I did not understand this point. Here is information of MIMRT700 EVK.
I don't know where model is saved on MIMRT700 EVK. And where is the 1 MB for region size? Is it actual limit of model size? Or we can increase model size by some methods.
Do you have any comment for this problem? Because I try to deploy a larger model > 1 MB. I do wait for your response. Thank you.
Thank you so much. Now I understood that we can increase the size of the model by setting region size.
Or If I want to run larger model on external memory, I can follow this document https://docs.nxp.com/bundle/AN14700/page/topics/external_memory.html
Hi @nnxxpp ,
Thank you so much for your interest in our products and for using our community.
Q: I don't know where model is saved on MIMRT700 EVK. And where is the 1 MB for region size? Is it actual limit of model size? Or we can increase model size by some methods.
Do you have any comment for this problem? Because I try to deploy a larger model > 1 MB.
A: The 1 MB shown for modeldata is not a hardware limit of the RT700.
It is only the default linker allocation used in the sample project.
For larger models, this allocation can be adjusted in the project settings, and external XSPI flash can also be used if more storage is needed.
For more detail information, you can refer to this AN14700.
https://docs.nxp.com/bundle/AN14700/page/topics/introduction.html
So, the RT700 is not inherently limited to a 1 MB model. Larger NPU models are supported either by increasing the modeldata memory allocation or by placing the model in external XSPI flash with the appropriate conversion option.
Wish it helps you
Best Regards
May Liu
I want to reopen this topic. Now i am trying to deploy larger model on RT700. The below image is captured when building the program with the small model.
I see that there are 4 memory regions:
- QSPI_flash: external memory
- SRAM: I ask chatgpt and it is for data when running the program (like .data, .bss, stack, heap). Is that correct?
- NCACHE_REGION: it is same ktensorArena (for inputs, intermediate outputs and output)
- modeldata: to save model weights
I see in the memory configuration when I import SDK example. It means that SRAM, NCACHE_REGION and modeldata from SRAM (7.5 MB).
NCACHE_REGION and modeldata should be located in 0x2000_0000 to 0x2058_0000 (5.5 MB) to get best perforemce (SRAM area that can be accessed by the NPU)
But location of SRAM (named SRAM) is 0x20080000 (in the second image) ==> It is also in the range 0x2000_0000 to 0x2058_0000. And by default, it is set about 2.5 MB. It means that NCACHE_REGION + modeldata should be less than (5.5 - 2.5) = 3 MB.
My model size is about 3.5 MB. Beside that I can locate my model on external memory (it results in larger inference time), how I can config memory to still locate my model (3.5 MB) on memory area that NPU can access?
I am curious about whether we can shrink "SRAM" region (in the images 1, 2) or can I move it to another area of RAM (7.5 - 5.5 = 2 MB - the last region in the image 3)?
And how I can estimate the size of "SRAM" region? In the below image, it is 15560 B.
Sorry for my long questions.
Good morning.
Maybe you missed my new above questions.
Hi @nnxxpp ,
Apologies for the delayed response.
If you don’t mind, could you please create a new case for your new issue?
Thank you for your understanding and cooperation.
Best Regards,
May
@mayliu1
Yes, ok. Let me create new issue. Thank you.
I have resolved my problem. We can locate SRAM outof 5.5 MB area for NPU. I locate modeldata and kTensorArena in area 5.5 MB and it worked. The inference time is good.
But If you did not miss my questions, so I can finish soon my tasks. Thank you.
Glad to hear that your issue has been resolved.
Apologies for the delayed response, thank you for your understanding.