NXP iMX95 EVK - NPU TensorFlow Lite Model

[Sudharsun]

Hi,

we have ran the eIQ TensorFlow Lite example models on the NPU, but they fail with the following errors:

INFO: Loaded model mobilenet_v1_1.0_224_quant.tflite
INFO: resolved reporter
INFO: EXTERNAL delegate created.
remoteproc remoteproc0: Booting fw image NeutronFirmware.elf
remoteproc remoteproc0: remote processor neutron-rproc is now up
INFO: NeutronDelegate delegate: 29 nodes delegated out of 31 nodes with 1 partition.
INFO: Applied EXTERNAL delegate.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
neutron 4ab00004.imx95-neutron: Invalid ioctl. cmd=1075577096, arg=549719032088
fail to create neutron inference job
Error: component='Neutron Driver', category='internal fault', code=442
ERROR: Node number 31 (NeutronDelegate) failed to invoke.
ERROR: Failed to invoke tflite!

The examples were built using the following package:

# AI/ML
IMAGE_INSTALL:append = " tensorflow-lite-neutron-delegate"

Currently, the remote processor shows as offline:

root@toradex-smarc-imx95-12593622:/sys/class/remoteproc/remoteproc0# cat state
offline

When attempting to start the remote processor manually, I encounter permission errors:

root@toradex-smarc-imx95-12593622:~# echo rproc-neutron-rproc-fw > /sys/class/remoteproc/remoteproc0/firmware
-sh: /sys/class/remoteproc/remoteproc0/firmware: Permission denied

root@toradex-smarc-imx95-12593622:~# echo start > /sys/class/remoteproc/remoteproc0/state
-sh: /sys/class/remoteproc/remoteproc0/state: Permission denied

We are currently working on running eIQ Toolkit TensorFlow Lite examples on the NPU referring to the IMX_ML_User_Guide and are encountering microcode mismatch errors.

Our setup uses the BSP version scarthgap-6.6.36-2.1.0 and the eIQ Toolkit version 1.16.0. We converted the TensorFlow Lite model .tflite into an NPU-optimized model using the eIQ Toolkit, but during execution, the following error occurs:

To troubleshoot, we have tested multiple BSP and eIQ Toolkit version combinations (BSP 6.6.52_2.2.0, 6.12.34_2.1.0 , lf-6.6.3_1.0.0 6.6.23_2.0.0 and 6.6.36_2.1.0 with eIQ 1.16.0, 1.15.1, and 1.14.0). However, any of the microcode versions doesn't match with the expected versions.

Please provide the expected microcode binary (NeutromFirmware.elf) for the execution or advice for any change.

[ekho]

Hi there,

I'm facing and cannot get around the same issue.

I have tested the example code that comes with the image - under example. I get the same problem with the NeutroDelegate not properly delegating the nodes. I tried with our detection model, same thing - as expected.

====== V4L2ENC: 1.26.0 build on May 7 2025 08:46:20. ======
INFO: NeutronDelegate delegate: 0 nodes delegated out of 678 nodes with 0 partitions.

Error in cpuinfo: prctl(PR_SVE_GET_VL) failed
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.

Any attempt to do this:

echo start > /sys/class/remoteproc/remoteproc0/state

Results in permission denied.

I have tried with many images:

• 6.12.20_2.0.0-v1.1: Works, it boots, but the delegate does not get properly allocated.
• 6.6.52_2.2.0-v1.1: Doesn't work - doesn't even boot.
• 6.6.23_2.0.0-v1.1: Doesn't work - doesn't even boot.
• 6.6.1_1.0.0-v1..00: Doesn't work - doesn't even boot.

We are testing, or trying to test, the NXP i.MX 95 that comes with the Variscite DART DM95. The images were downloaded from their site. I also tried with the latest NXP release, but that didn't work either.

Any idea what could be behind this permission denied? If you have a particular, working, image, could you please share the link to download it?

Thanks in advance for your support.

Cheers!

[Sudharsun]

we have updated to the latest BSP release, but the issue still persists. Could you please help us resolve this problem?

[ekho]

The NXP support is really bad and their documentation doesn't offer all we need, not to mention the images that do not come with a Neutron compliant model - because they did not convert it and added to the image.

I reached out to Variscite, the company who provides the board where the NXP iMX 95 is integrated, and they help me. To make it work, even with the NPX examples, you have to conver the model yourself. Follow the instructions here: https://dev.variscite.com/dart-mx95/mx95-yocto-walnascar-6.12.20_2.0.0-v1.2/machine-learning/

I did not do the conversion yet, busy with other stuff.

Hope it helps.

[Sudharsun]

Hi,

    Thank you for your support. I reviewed the link you shared and followed the provided steps, but I’m still encountering the microcode mismatch error. I’ve attached a screenshot for your reference.

Could you please review this and help us resolve the issue?

[ekho]

Hi,

I don't know this error, perhaps someone from NXP should jump in.

I followed the guides as well and managed to convert my custom one stage detector (CNN based). Unfortunately, only 1 node is loaded into the delegate (for the rest, it's still XNNPACK for CPU):

====== V4L2ENC: 1.26.0 build on May  7 2025 08:46:20. ======
INFO: NeutronDelegate delegate: 1 nodes delegated out of 697 nodes with 1 partitions.

INFO: Neutron delegate version: v1.0.0-a5d640e6, zerocp enabled.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.

That's probably because after conversion, only 1 Neutron Graph is created - out of 697. With only 27 operators converted. lol

Conversion statistics:
  Number of operators after import    = 678
  Number of operators after optimize  = 723
    Number of operators converted     = 27
    Number of operators NOT converted = 696
  Number of operators after extract   = 697
    Number of Neutron graphs          = 1
    Number of operators NOT converted = 696
  Operator conversion ratio           = 27 / 723 = 0.0373444
  Operators converted                 = 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,

I have contacted Variscite support again to see if I get any ideas from them. It's a shame NXP does not really reply here.

I hope you find a solution. I will keep digging and if I find anything I will let you know.

Cheers! 

[ekho]

Hi again,

I managed to get way more nodes and operators converted with the latest version (6.12.34). Went from 3% to 62$ operators converted, which is really good for a YOLOv11 based model.

Now I have the version mismatch as well. But that's due to the version on my board. So, I'm trying to update it from 6.12.20 to 6.12.34. Please, check the version you have and the version you used for conversion.

Cheers!

[ekho]

Hi again,

You shoudl convert your model like this:

/opt/nxp/eIQ_Toolkit_v1.17.0/bin/neutron-converter/MCU_SDK_25.09.00+Linux_6.12.34_2.1.0/neutron-converter \
    --input you_model_quant.tflite \
    --output your_model_neutron.tflite \
    --target imx95 \
    --convert-inputs-uint8-to-int8 \
    --convert-outputs-uint8-to-int8

I got YOLOv11 Nano running at 27 FPS. I still have the version mismatch warning, but those are not causing issues. It's not fun to have warning, but can't do much about it now.

I hope it helps.

Cheers!

[Sudharsun]

Hi,

The BSP version was upgraded from 6.12.20 to 6.12.34, and the default example from the eIQ toolkit was executed. The attached screenshot shows the result — the example successfully ran and produced the default image classification output on the NXP i.MX95 EVK. However, none of the 31 nodes were delegated during execution.

I converted my .tflite model using the latest eIQ Toolkit version 17. However, when I run the example with TensorFlow Lite version 17, no image classification results are displayed. I’ve attached a screenshot showing this issue.

Both of the above cases were executed on the NXP i.MX95 EVK.
However, our main objective is to run these examples on the Toradex SMARC i.MX95 EVK.
The BSP version upgrade on the Toradex board is still unstable, resulting in multiple errors when running AI workloads on the NPU, as the Toradex i.MX95 EVK does not yet have a stable BSP release for version 6.12.34.

[Sudharsun]

Hi,

   Could you please help us to resolve this problem ?

[Bio_TICFSL]

Hello,

Based on your error messages, you’re encountering both RemoteProc permission issues and NPU delegate failures. Let me help you resolve these step by step.

1. RemoteProc Permission Issues

The permission errors you’re seeing when trying to manually control the remote processor are common. According to the RemoteProc documentation, you need to ensure the remote processor is in the correct state before making changes.

# cat /sys/class/remoteproc/remoteproc0/state

If you’re getting permission denied errors, try stopping the remote processor first:

# echo stop > /sys/class/remoteproc/remoteproc0/state

(

Then verify it’s offline:

# cat /sys/class/remoteproc/remoteproc0/state
offline

2. Firmware Loading Process

For loading firmware, the RemoteProc framework looks for *.elf firmware stored inside /lib/firmware by default (1). You can copy your firmware to this location:

$ scp hello_world.elf root@<board-ip>:/lib/firmware
If your firmware is stored elsewhere, you can update the firmware path:

# echo -n new_path > /sys/module/firmware_class/parameters/path

3. Loading and Starting the Remote Processor

To load your firmware, write its name to the firmware file:

# echo <firmware>.elf > /sys/class/remoteproc/remoteproc0/firmware

Important: Make sure to run a *.elf firmware, otherwise it will not work

To start the remote processor with the new firmware:

# echo start > /sys/class/remoteproc/remoteproc0/state

You should see output similar to:

[12594.692880] remoteproc remoteproc0: powering up imx-rproc
[12594.698629] remoteproc remoteproc0: Booting fw image hello_world.elf, size 240736
[12594.706192] remoteproc remoteproc0: no dtb rsrc-table
[12594.761474] remoteproc remoteproc0: remote processor imx-rproc is now up

4. NPU Configuration Requirements

Based on the community discussions, there are several important considerations for NPU setup:

1. Verify NPU Support: Ensure that your hardware supports NPU execution and that the TensorFlow Lite version you are using is compatible with NPU acceleration

2. Install NPU Drivers: Make sure NPU drivers are properly installed and configured

3. Use the Correct TensorFlow Lite Delegate: Ensure you are using the appropriate delegate for NPU execution

5. Troubleshooting Steps

1. Check Hardware Support: Verify your hardware configuration supports the NPU delegate
2. Firmware Verification: Ensure the correct NPU firmware is loaded
3. Driver Status: Check if the neutron driver is properly loaded and accessible
4. Delegate Configuration: Verify the TensorFlow Lite delegate is correctly configured for your specific hardware

6. Alternative Approach

If you continue experiencing issues, consider testing with CPU-only inference first to isolate whether the problem is specifically with NPU acceleration. You can disable hardware acceleration temporarily with USE_HW_ACCELERATED_INFERENCE=0

Regards

[Sudharsun]

Hi,

   We have tried the below commands as per you suggestion still faced the errors like permission denied. We are trying the command in root and sudo also but still facing the permission denied issue. Have added the Screenshots for you reference. 

1.echo stop > /sys/class/remoteproc/remoteproc0/state

2.cat /sys/class/remoteproc/remoteproc0/state

3.ls /lib/firmware

4.echo NeutronFirmware.elf > /sys/class/remoteproc/remoteproc0/firmware

[Bio_TICFSL]

Hi. 

That is weird, so please make sure you have the latest BSP, erase and burn the linux again in your board, I can not said anymore since it works nicely right here.

Regards