topic Re: Output error running eiq_genai_flow with Neutron in Generative AI & LLMs

Output error running eiq_genai_flow with Neutron

herveld — Mon, 29 Sep 2025 09:27:43 GMT

Hello,

We are getting garbled LLM output when using eiq_genai_flow with Neutron (please see output log below)

root@imx95-var-dart:/home/dm-eiq-genai-flow-demonstrator/eiq_genai_flow# ./eiq_genai_flow --use-neutron --output-mode text
Target Device: i.MX95
[NeutronEP:Allocator] CMA userspace memory 1536 MB
Neutron: kernel buffer vaddr 0xfffe37600000, paddr 0xffff80000000, align_addr 0xffff80000000, size 0x80000000 alignment 0x0
high 0xffff, low 0x80000000, offset 0x0
[NeutronEP:Allocator] 0 0xfffe37600000 512 MB
[NeutronEP:Allocator] 1 0xfffe57600000 512 MB
[NeutronEP:Allocator] 2 0xfffe77600000 512 MB
2025-03-06 18:56:15.903541867 [W:onnxruntime:, session_state.cc:1168 VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the prefer
red execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to im
prove perf.
2025-03-06 18:56:15.903631201 [W:onnxruntime:, session_state.cc:1170 VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-min
imal build will show node assignments.
LLM used: danube-onnx

Please type your question:
who are you ?
utatMAGESgewwiequantfrag morteasiNSermńskiCMangleshalten JahrhundertTRAN pró身atinPCMorem südanes tijdfolg relativeifontанTEMPńskaneGTH когда s
iскоutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatuta
tutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutatutat[...]

Neutron kernel buffer address is different from the screenshot on github : https://github.com/nxp-appcodehub/dm-eiq-genai-flow-demonstrator/blob/main/assets/run_neutron_demo.png

Here is the dmesg output related to Neutron:

root@imx95-var-dart:~/eiq_genai_flow# dmesg | grep -i neutron
[    0.000000] OF: reserved mem: initialized node neutron_memory@100000000, compatible id shared-dma-pool
[    0.000000] OF: reserved mem: 0x0000000100000000..0x00000001ffffffff (4194304 KiB) map reusable neutron_memory@100000000
[    2.313294] remoteproc remoteproc0: neutron-rproc is available
[    4.986961] neutron 4ab00004.imx95-neutron: Adding to iommu group 8
[    4.993755] neutron 4ab00004.imx95-neutron: assigned reserved memory node neutron_memory@100000000
[    5.003367] neutron 4ab00004.imx95-neutron: created neutron device, name=neutron0

Is there a special configuration to use eiq_genai_flow with Neutron ?

Regards,

Herve

Re: Output error running eiq_genai_flow with Neutron

Pierre_M — Mon, 29 Sep 2025 14:41:58 GMT

Hi Hervé,

Thank you for reaching out!

Based on your description, it’s likely that your i.MX95 SoC is not a revision 2.0 (B0). You can confirm this by running the following command:

cat /sys/devices/soc0/revision

If the output indicates a 1.x revision (Ax), no worries — you simply need to use the corresponding v1.x Neutron firmware. You’ll find it in the package, and it should replace the existing /lib/firmware/NeutronFwllm.elf file included in the L6.12.20 BSP.

Please let me know if this resolves the issue or if you need further assistance.

Thanks and best regards,
Pierre

Re: Output error running eiq_genai_flow with Neutron

herveld — Mon, 29 Sep 2025 14:51:53 GMT

Hi Pierre,

The i.MX95 revision we are using is B0.

The output of :

cat /sys/devices/soc0/revision

is 2.0

Best regards,

Hervé

Re: Output error running eiq_genai_flow with Neutron

Pierre_M — Mon, 29 Sep 2025 15:25:18 GMT

Hi Hervé,

Please can you share the BSP version you are using?

uname -a

Also please can you share the md5sum of the Neutron FWs?

md5sum /lib/firmware/Neutron*

Thanks and best regards,

Pierre

Re: Output error running eiq_genai_flow with Neutron

herveld — Mon, 29 Sep 2025 16:22:20 GMT

Hi Pierre,

Following are the information about our working environment.

Best regards,

Hervé

root@imx95-var-dart:~# uname -a
Linux imx95-var-dart 6.12.20-var-lts-next-g4bffdb611345 #1 SMP PREEMPT Sat Aug 9 08:08:38 UTC 2025 aarch64 GNU/Linux

 root@imx95-var-dart:~# md5sum /lib/firmware/Neutron*
a4893808e8927b56be8461cdf7678927 /lib/firmware/NeutronFirmware.elf
eb22f5ebd105e780cad87162f4a069a8 /lib/firmware/NeutronFwllm.elf

Re: Output error running eiq_genai_flow with Neutron

Pierre_M — Mon, 29 Sep 2025 16:42:30 GMT

Hi Hervé,
Thanks for your input. After checking, we can confirm that there is indeed an issue with Neutron in this release. We're currently investigating and working on a solution, and we'll get back to you as soon as possible.
Apologies for the inconvenience, and thank you for your patience.
Best regards,
Pierre

Re: Output error running eiq_genai_flow with Neutron

Pierre_M — Tue, 30 Sep 2025 10:22:05 GMT

Hi Hervé,

Unfortunately, the current package cannot run Neutron on the i.MX95 B0 due to an incompatible Neutron firmware included in the BSP. However, you can still use the package in CPU mode without any issues.

We’re actively working on a new release planned for the end of 2025, which will support more platforms and introduce additional features. Neutron support will be reinstated in that version — stay tuned!

Thanks again, and best regards,
Pierre

Re: Output error running eiq_genai_flow with Neutron

herveld — Tue, 30 Sep 2025 10:40:14 GMT

Hi Pierre,

Thank you for your input regarding Neutron. We will wait for a BSP patch.

We also have an Alsa issue when enabling ASR on eiq_genai_flow. I posted a message earlier about it on the list. Could you please look into it ?

Thank you and best regards,

Herve