2329726_en-US

[#1]

TCM constraints in iMX95

Hi all,

When attempting to convert already quantized TFLite models for i.MX95 using the Neutron converter, I repeatedly encounter the following warning:

WARNING: For this target, the Neutron Software currently only supports operators whose tensors can fit in TCM!

This warning appears many times during the conversion process. The number of occurrences varies depending on the model being converted.

So far, I have observed this behavior with several object detection models trained on my own dataset, including:

• SSD MobileNet v3 (this was taken from the default elQ Toolkit from GUI)

• YOLOv5

• EfficientDet

I exported these models as quantized TFLite (INT8) and tested multiple input resolutions, including:

• 320 × 320

• 224 × 224

However, the warning still appears consistently during conversion, regardless of the model architecture or input size.

Despite the warnings, the converter still produces an output .tflite file.

My questions are:

1. Does this warning indicate a critical issue that would prevent the model from running properly on the i.MX95 NPU?

2. Does it imply that some operators will fall back to CPU execution?

3. Are there recommended guidelines for model architecture or tensor sizes to ensure compatibility with the TCM constraints of the Neutron NPU?

Any suggestions on how to address or mitigate this warning would be greatly appreciated.

Thanks in advance.

Re: TCM constraints in iMX95

Hi @EOF 

For these questions about Neutron converter, please refer the docs in eIQ Neutron SDK 3.0.0

https://www.nxp.com/design/design-center/software/eiq-ai-development-environment/eiq-toolkit-for-end...

- - Standard operators that are supported by NPU are extracted together and mapped to one or multiple `NeutronGraph` custom operators in the converted model to be executed by the NPU.
    - Standard operators that are NOT supported by NPU are left unmodified in the converted model to be executed by the CPU.
- 

The converter attempts to tile each operator's input/output/scratch intermediate tensors and place them into the NPU's TCM. If an operator's tiles cannot be arranged within the TCM, that operator will run on the CPU. For larger models, TCM memory exhaustion may occur during the i.MX95 loading. This is not a error but indicates the need to downsize the model.

For supported operators, please refer 12 Neural Network API Reference in this guide.

https://www.nxp.com/docs/en/user-guide/UG10166.pdf

Best Regards,
Zhiming

Re: TCM constraints in iMX95

Thank you @Zhiming_Liu for the quick reply!