topic Re: Does eIQ NPU on MCXN947 support transformer based model? in MCX Microcontrollers https://community.nxp.com/t5/MCX-Microcontrollers/Does-eIQ-NPU-on-MCXN947-support-transformer-based-model/m-p/2153225#M3768 <P>Hi&nbsp;<a href="https://community.nxp.com/t5/user/viewprofilepage/user-id/246062">@TomC818</a>&nbsp;</P> <P><SPAN data-slate-fragment="JTVCJTdCJTIydHlwZSUyMiUzQSUyMnBhcmFncmFwaCUyMiUyQyUyMmNoaWxkcmVuJTIyJTNBJTVCJTdCJTIyaWQlMjIlM0ElMjJWQW85QWptWWRCJTIyJTJDJTIycGFyYUlkeCUyMiUzQTIlMkMlMjJzcmMlMjIlM0ElMjIlMjBGdWxsJTIwJUUyJTgwJTlDTExNJTIwb24lMjBOUFUlRTIlODAlOUQlMjBpc24lRTIlODAlOTl0JTIwYSUyMHN1cHBvcnRlZCUyMHBhdGglMjB5ZXQlMjBvbiUyME1DWC4lMjAlMjIlMkMlMjJkc3QlMjIlM0ElMjJNQ1glRTQlQjglOEElRTUlQjAlOUElRTQlQjglOEQlRTYlOTQlQUYlRTYlOEMlODElRTUlQUUlOEMlRTYlOTUlQjQlRTclOUElODQlRTIlODAlOUNOUFUlRTQlQjglOEElRTclOUElODRMTE0lRTIlODAlOUQlRTglQjclQUYlRTUlQkUlODQlRTMlODAlODIlMjIlMkMlMjJtZXRhZGF0YSUyMiUzQSUyMiUyMiUyQyUyMm1hdGNoZXMlMjIlM0FudWxsJTJDJTIybWV0YURhdGElMjIlM0ElNUIlNUQlMkMlMjJ0ZXh0JTIyJTNBJTIyJTIwRnVsbCUyMCVFMiU4MCU5Q0xMTSUyMG9uJTIwTlBVJUUyJTgwJTlEJTIwaXNuJUUyJTgwJTk5dCUyMGElMjBzdXBwb3J0ZWQlMjBwYXRoJTIweWV0JTIwb24lMjBNQ1guJTIwJTIyJTdEJTVEJTdEJTVE"> Full “LLM on NPU” isn’t a supported path yet on MCX. </SPAN></P> <P><SPAN data-slate-fragment="JTVCJTdCJTIydHlwZSUyMiUzQSUyMnBhcmFncmFwaCUyMiUyQyUyMmNoaWxkcmVuJTIyJTNBJTVCJTdCJTIyaWQlMjIlM0ElMjJWQW85QWptWWRCJTIyJTJDJTIycGFyYUlkeCUyMiUzQTIlMkMlMjJzcmMlMjIlM0ElMjIlMjBGdWxsJTIwJUUyJTgwJTlDTExNJTIwb24lMjBOUFUlRTIlODAlOUQlMjBpc24lRTIlODAlOTl0JTIwYSUyMHN1cHBvcnRlZCUyMHBhdGglMjB5ZXQlMjBvbiUyME1DWC4lMjAlMjIlMkMlMjJkc3QlMjIlM0ElMjJNQ1glRTQlQjglOEElRTUlQjAlOUElRTQlQjglOEQlRTYlOTQlQUYlRTYlOEMlODElRTUlQUUlOEMlRTYlOTUlQjQlRTclOUElODQlRTIlODAlOUNOUFUlRTQlQjglOEElRTclOUElODRMTE0lRTIlODAlOUQlRTglQjclQUYlRTUlQkUlODQlRTMlODAlODIlMjIlMkMlMjJtZXRhZGF0YSUyMiUzQSUyMiUyMiUyQyUyMm1hdGNoZXMlMjIlM0FudWxsJTJDJTIybWV0YURhdGElMjIlM0ElNUIlNUQlMkMlMjJ0ZXh0JTIyJTNBJTIyJTIwRnVsbCUyMCVFMiU4MCU5Q0xMTSUyMG9uJTIwTlBVJUUyJTgwJTlEJTIwaXNuJUUyJTgwJTk5dCUyMGElMjBzdXBwb3J0ZWQlMjBwYXRoJTIweWV0JTIwb24lMjBNQ1guJTIwJTIyJTdEJTVEJTdEJTVE">LLMs rely on dynamic sequences, KV caches, etc.—many are outside the current eIQ Neutron TFLite op set.</SPAN></P> <P><SPAN data-slate-fragment="JTVCJTdCJTIydHlwZSUyMiUzQSUyMnBhcmFncmFwaCUyMiUyQyUyMmNoaWxkcmVuJTIyJTNBJTVCJTdCJTIyaWQlMjIlM0ElMjJWQW85QWptWWRCJTIyJTJDJTIycGFyYUlkeCUyMiUzQTIlMkMlMjJzcmMlMjIlM0ElMjIlMjBGdWxsJTIwJUUyJTgwJTlDTExNJTIwb24lMjBOUFUlRTIlODAlOUQlMjBpc24lRTIlODAlOTl0JTIwYSUyMHN1cHBvcnRlZCUyMHBhdGglMjB5ZXQlMjBvbiUyME1DWC4lMjAlMjIlMkMlMjJkc3QlMjIlM0ElMjJNQ1glRTQlQjglOEElRTUlQjAlOUElRTQlQjglOEQlRTYlOTQlQUYlRTYlOEMlODElRTUlQUUlOEMlRTYlOTUlQjQlRTclOUElODQlRTIlODAlOUNOUFUlRTQlQjglOEElRTclOUElODRMTE0lRTIlODAlOUQlRTglQjclQUYlRTUlQkUlODQlRTMlODAlODIlMjIlMkMlMjJtZXRhZGF0YSUyMiUzQSUyMiUyMiUyQyUyMm1hdGNoZXMlMjIlM0FudWxsJTJDJTIybWV0YURhdGElMjIlM0ElNUIlNUQlMkMlMjJ0ZXh0JTIyJTNBJTIyJTIwRnVsbCUyMCVFMiU4MCU5Q0xMTSUyMG9uJTIwTlBVJUUyJTgwJTlEJTIwaXNuJUUyJTgwJTk5dCUyMGElMjBzdXBwb3J0ZWQlMjBwYXRoJTIweWV0JTIwb24lMjBNQ1guJTIwJTIyJTdEJTVEJTdEJTVE">BR</SPAN></P> <P><SPAN data-slate-fragment="JTVCJTdCJTIydHlwZSUyMiUzQSUyMnBhcmFncmFwaCUyMiUyQyUyMmNoaWxkcmVuJTIyJTNBJTVCJTdCJTIyaWQlMjIlM0ElMjJWQW85QWptWWRCJTIyJTJDJTIycGFyYUlkeCUyMiUzQTIlMkMlMjJzcmMlMjIlM0ElMjIlMjBGdWxsJTIwJUUyJTgwJTlDTExNJTIwb24lMjBOUFUlRTIlODAlOUQlMjBpc24lRTIlODAlOTl0JTIwYSUyMHN1cHBvcnRlZCUyMHBhdGglMjB5ZXQlMjBvbiUyME1DWC4lMjAlMjIlMkMlMjJkc3QlMjIlM0ElMjJNQ1glRTQlQjglOEElRTUlQjAlOUElRTQlQjglOEQlRTYlOTQlQUYlRTYlOEMlODElRTUlQUUlOEMlRTYlOTUlQjQlRTclOUElODQlRTIlODAlOUNOUFUlRTQlQjglOEElRTclOUElODRMTE0lRTIlODAlOUQlRTglQjclQUYlRTUlQkUlODQlRTMlODAlODIlMjIlMkMlMjJtZXRhZGF0YSUyMiUzQSUyMiUyMiUyQyUyMm1hdGNoZXMlMjIlM0FudWxsJTJDJTIybWV0YURhdGElMjIlM0ElNUIlNUQlMkMlMjJ0ZXh0JTIyJTNBJTIyJTIwRnVsbCUyMCVFMiU4MCU5Q0xMTSUyMG9uJTIwTlBVJUUyJTgwJTlEJTIwaXNuJUUyJTgwJTk5dCUyMGElMjBzdXBwb3J0ZWQlMjBwYXRoJTIweWV0JTIwb24lMjBNQ1guJTIwJTIyJTdEJTVEJTdEJTVE">Harry</SPAN></P> Mon, 18 Aug 2025 03:50:22 GMT Harry_Zhang 2025-08-18T03:50:22Z Does eIQ NPU on MCXN947 support transformer based model? https://community.nxp.com/t5/MCX-Microcontrollers/Does-eIQ-NPU-on-MCXN947-support-transformer-based-model/m-p/2152255#M3755 <P>Hello,<BR />As mentioned in the title, can the NPU in the mcu support transformer based model? From the official website, it was claimed that the NPU can support transformer models.</P><P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="TomC818_0-1755162965843.png" style="width: 400px;"><img src="https://community.nxp.com/t5/image/serverpage/image-id/352424i485E078360472DE6/image-size/medium?v=v2&amp;px=400" role="button" title="TomC818_0-1755162965843.png" alt="TomC818_0-1755162965843.png" /></span><BR />Assuming with compression, quantization and external RAM through flexspi, a small transformer based model can be fitted on board. Can the NPU actually be used to accelerate the inference?&nbsp;<BR /><BR />An example would be&nbsp;<A href="https://github.com/maxbbraun/llama4micro" target="_blank">GitHub - maxbbraun/llama4micro: A "large" language model running on a microcontroller</A><BR />Instead of running on the CPU, can the model be ran on the NPU?&nbsp;<BR />How can such a use case be adapted to MCXN?</P><P>&nbsp;</P> Thu, 14 Aug 2025 09:24:19 GMT https://community.nxp.com/t5/MCX-Microcontrollers/Does-eIQ-NPU-on-MCXN947-support-transformer-based-model/m-p/2152255#M3755 TomC818 2025-08-14T09:24:19Z Re: Does eIQ NPU on MCXN947 support transformer based model? https://community.nxp.com/t5/MCX-Microcontrollers/Does-eIQ-NPU-on-MCXN947-support-transformer-based-model/m-p/2153225#M3768 <P>Hi&nbsp;<a href="https://community.nxp.com/t5/user/viewprofilepage/user-id/246062">@TomC818</a>&nbsp;</P> <P><SPAN data-slate-fragment="JTVCJTdCJTIydHlwZSUyMiUzQSUyMnBhcmFncmFwaCUyMiUyQyUyMmNoaWxkcmVuJTIyJTNBJTVCJTdCJTIyaWQlMjIlM0ElMjJWQW85QWptWWRCJTIyJTJDJTIycGFyYUlkeCUyMiUzQTIlMkMlMjJzcmMlMjIlM0ElMjIlMjBGdWxsJTIwJUUyJTgwJTlDTExNJTIwb24lMjBOUFUlRTIlODAlOUQlMjBpc24lRTIlODAlOTl0JTIwYSUyMHN1cHBvcnRlZCUyMHBhdGglMjB5ZXQlMjBvbiUyME1DWC4lMjAlMjIlMkMlMjJkc3QlMjIlM0ElMjJNQ1glRTQlQjglOEElRTUlQjAlOUElRTQlQjglOEQlRTYlOTQlQUYlRTYlOEMlODElRTUlQUUlOEMlRTYlOTUlQjQlRTclOUElODQlRTIlODAlOUNOUFUlRTQlQjglOEElRTclOUElODRMTE0lRTIlODAlOUQlRTglQjclQUYlRTUlQkUlODQlRTMlODAlODIlMjIlMkMlMjJtZXRhZGF0YSUyMiUzQSUyMiUyMiUyQyUyMm1hdGNoZXMlMjIlM0FudWxsJTJDJTIybWV0YURhdGElMjIlM0ElNUIlNUQlMkMlMjJ0ZXh0JTIyJTNBJTIyJTIwRnVsbCUyMCVFMiU4MCU5Q0xMTSUyMG9uJTIwTlBVJUUyJTgwJTlEJTIwaXNuJUUyJTgwJTk5dCUyMGElMjBzdXBwb3J0ZWQlMjBwYXRoJTIweWV0JTIwb24lMjBNQ1guJTIwJTIyJTdEJTVEJTdEJTVE"> Full “LLM on NPU” isn’t a supported path yet on MCX. </SPAN></P> <P><SPAN data-slate-fragment="JTVCJTdCJTIydHlwZSUyMiUzQSUyMnBhcmFncmFwaCUyMiUyQyUyMmNoaWxkcmVuJTIyJTNBJTVCJTdCJTIyaWQlMjIlM0ElMjJWQW85QWptWWRCJTIyJTJDJTIycGFyYUlkeCUyMiUzQTIlMkMlMjJzcmMlMjIlM0ElMjIlMjBGdWxsJTIwJUUyJTgwJTlDTExNJTIwb24lMjBOUFUlRTIlODAlOUQlMjBpc24lRTIlODAlOTl0JTIwYSUyMHN1cHBvcnRlZCUyMHBhdGglMjB5ZXQlMjBvbiUyME1DWC4lMjAlMjIlMkMlMjJkc3QlMjIlM0ElMjJNQ1glRTQlQjglOEElRTUlQjAlOUElRTQlQjglOEQlRTYlOTQlQUYlRTYlOEMlODElRTUlQUUlOEMlRTYlOTUlQjQlRTclOUElODQlRTIlODAlOUNOUFUlRTQlQjglOEElRTclOUElODRMTE0lRTIlODAlOUQlRTglQjclQUYlRTUlQkUlODQlRTMlODAlODIlMjIlMkMlMjJtZXRhZGF0YSUyMiUzQSUyMiUyMiUyQyUyMm1hdGNoZXMlMjIlM0FudWxsJTJDJTIybWV0YURhdGElMjIlM0ElNUIlNUQlMkMlMjJ0ZXh0JTIyJTNBJTIyJTIwRnVsbCUyMCVFMiU4MCU5Q0xMTSUyMG9uJTIwTlBVJUUyJTgwJTlEJTIwaXNuJUUyJTgwJTk5dCUyMGElMjBzdXBwb3J0ZWQlMjBwYXRoJTIweWV0JTIwb24lMjBNQ1guJTIwJTIyJTdEJTVEJTdEJTVE">LLMs rely on dynamic sequences, KV caches, etc.—many are outside the current eIQ Neutron TFLite op set.</SPAN></P> <P><SPAN data-slate-fragment="JTVCJTdCJTIydHlwZSUyMiUzQSUyMnBhcmFncmFwaCUyMiUyQyUyMmNoaWxkcmVuJTIyJTNBJTVCJTdCJTIyaWQlMjIlM0ElMjJWQW85QWptWWRCJTIyJTJDJTIycGFyYUlkeCUyMiUzQTIlMkMlMjJzcmMlMjIlM0ElMjIlMjBGdWxsJTIwJUUyJTgwJTlDTExNJTIwb24lMjBOUFUlRTIlODAlOUQlMjBpc24lRTIlODAlOTl0JTIwYSUyMHN1cHBvcnRlZCUyMHBhdGglMjB5ZXQlMjBvbiUyME1DWC4lMjAlMjIlMkMlMjJkc3QlMjIlM0ElMjJNQ1glRTQlQjglOEElRTUlQjAlOUElRTQlQjglOEQlRTYlOTQlQUYlRTYlOEMlODElRTUlQUUlOEMlRTYlOTUlQjQlRTclOUElODQlRTIlODAlOUNOUFUlRTQlQjglOEElRTclOUElODRMTE0lRTIlODAlOUQlRTglQjclQUYlRTUlQkUlODQlRTMlODAlODIlMjIlMkMlMjJtZXRhZGF0YSUyMiUzQSUyMiUyMiUyQyUyMm1hdGNoZXMlMjIlM0FudWxsJTJDJTIybWV0YURhdGElMjIlM0ElNUIlNUQlMkMlMjJ0ZXh0JTIyJTNBJTIyJTIwRnVsbCUyMCVFMiU4MCU5Q0xMTSUyMG9uJTIwTlBVJUUyJTgwJTlEJTIwaXNuJUUyJTgwJTk5dCUyMGElMjBzdXBwb3J0ZWQlMjBwYXRoJTIweWV0JTIwb24lMjBNQ1guJTIwJTIyJTdEJTVEJTdEJTVE">BR</SPAN></P> <P><SPAN data-slate-fragment="JTVCJTdCJTIydHlwZSUyMiUzQSUyMnBhcmFncmFwaCUyMiUyQyUyMmNoaWxkcmVuJTIyJTNBJTVCJTdCJTIyaWQlMjIlM0ElMjJWQW85QWptWWRCJTIyJTJDJTIycGFyYUlkeCUyMiUzQTIlMkMlMjJzcmMlMjIlM0ElMjIlMjBGdWxsJTIwJUUyJTgwJTlDTExNJTIwb24lMjBOUFUlRTIlODAlOUQlMjBpc24lRTIlODAlOTl0JTIwYSUyMHN1cHBvcnRlZCUyMHBhdGglMjB5ZXQlMjBvbiUyME1DWC4lMjAlMjIlMkMlMjJkc3QlMjIlM0ElMjJNQ1glRTQlQjglOEElRTUlQjAlOUElRTQlQjglOEQlRTYlOTQlQUYlRTYlOEMlODElRTUlQUUlOEMlRTYlOTUlQjQlRTclOUElODQlRTIlODAlOUNOUFUlRTQlQjglOEElRTclOUElODRMTE0lRTIlODAlOUQlRTglQjclQUYlRTUlQkUlODQlRTMlODAlODIlMjIlMkMlMjJtZXRhZGF0YSUyMiUzQSUyMiUyMiUyQyUyMm1hdGNoZXMlMjIlM0FudWxsJTJDJTIybWV0YURhdGElMjIlM0ElNUIlNUQlMkMlMjJ0ZXh0JTIyJTNBJTIyJTIwRnVsbCUyMCVFMiU4MCU5Q0xMTSUyMG9uJTIwTlBVJUUyJTgwJTlEJTIwaXNuJUUyJTgwJTk5dCUyMGElMjBzdXBwb3J0ZWQlMjBwYXRoJTIweWV0JTIwb24lMjBNQ1guJTIwJTIyJTdEJTVEJTdEJTVE">Harry</SPAN></P> Mon, 18 Aug 2025 03:50:22 GMT https://community.nxp.com/t5/MCX-Microcontrollers/Does-eIQ-NPU-on-MCXN947-support-transformer-based-model/m-p/2153225#M3768 Harry_Zhang 2025-08-18T03:50:22Z