Out of TCM memory in imx95 Neutron delegate

David41 — Thu, 13 Feb 2025 17:21:06 GMT

I'm working on imx95 version 6.6.23, trying to run a large model using neutron delegate.

I'm getting the following errors (several times) when loading the model:

WARNING: De-extracting subgraph 'subgraph_002' because of a failure!
WARNING: The failure occurred with message 'Out of TCM memory!' at line 293 in file '/home/osc/ci/npi-imx95/neutron-software/neutron-converter/src/NeutronTCMAllocator.cpp' in domain 'Allocator'.

The library is not provided in source code so there is no way to track the error.

How do I know what is the problem? What is the limit of what item in the model?

Thanks for the help

Re: Out of TCM memory in imx95 Neutron delegate

Bio_TICFSL — Mon, 17 Feb 2025 14:30:02 GMT

Hello,

That;s an issue with larger models at the moment of load the i.MX95 get the out of memory issue, but that is not a simple error its a warning that you may have to reduce the model. The i.mx95 development still on beta phase and will be investigating issue like this. Thanks for the catch.

Regards

Re: Out of TCM memory in imx95 Neutron delegate

aekam_acldigital — Tue, 18 Feb 2025 07:31:07 GMT

Hello,

I am evaluating i.MX95 EVK for the ML performance. I already have experience with i.MX8M Plus and have developed a couple of applications already.

As a starting point, I considered running sample application under eiq-example.

I followed the steps, however I ran into a problem with 'Out of TCM memory'.

Here is the log for your reference.

root@imx95evk:~/eiq-example/face_recognition# uname -a
Linux imx95evk 6.6.36-lts-next-gd23d64eea511 #1 SMP PREEMPT Wed Sep 4 08:22:45 UTC 2024 aarch64 GNU/Linux
root@imx95evk:~/eiq-example# ls -la models/
total 24024
drwxr-xr-x 2 root root 4096 Feb 14 11:44 .
drwxr-xr-x 8 root root 4096 Feb 14 11:45 ..
-rw-r--r-- 1 root root 24538272 Feb 14 11:44 facenet_512_int_quantized.tflite
-rw-r--r-- 1 root root 49472 Feb 14 11:44 yoloface_int8.tflite
root@imx95evk:~/eiq-example/face_recognition# python3 main.py --input=/dev/video13 --delegate=/usr/lib/libneutron_delegate.so
INFO: NeutronDelegate delegate: 45 nodes delegated out of 54 nodes with 4 partitions.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
WARNING: De-extracting subgraph 'subgraph_159' because of a failure!
WARNING: The failure occurred with message 'Out of TCM memory!' at line 412 in file '/opt/bamboo/agent-home/xml-data/build-dir/MLTEC-NEUTNPIIMX950-BDEG/neutron-software/neutron-converter/src/NeutronTCMAllocator.cpp' in domain 'Allocator'.
Traceback (most recent call last):
File "/root/eiq-example/face_recognition/main.py", line 30, in <module>
recognizer = Facenet("../models/facenet_512_int_quantized.tflite", args.delegate)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/eiq-example/face_recognition/face_recognition.py", line 20, in __init__
self.interpreter.allocate_tensors()
File "/usr/lib/python3.12/site-packages/tflite_runtime/interpreter.py", line 531, in allocate_tensors
return self._interpreter.AllocateTensors()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Failed to build Neutron graph.
Delegate kernel was not initializedNode number 198 (NeutronDelegate) failed to prepare.Failed to apply the default TensorFlow Lite delegate indexed at 0.
Error: invalid model handle!
Error: invalid model handle!
Error: invalid model handle!
Error: invalid model handle!
Error: invalid model handle!
Error: invalid model handle!
Error: invalid model handle!
Error: invalid model handle!
Error: invalid model handle!

Let me know how this can be resolved.

Regards,

Aekam Parmar

topic Re: Out of TCM memory in imx95 Neutron delegate in i.MX Processors

Out of TCM memory in imx95 Neutron delegate

Re: Out of TCM memory in imx95 Neutron delegate

Re: Out of TCM memory in imx95 Neutron delegate