Does eIQ Neutron support Vision Transformer (RF-DETR / DETR) conversion for NPU acceleration on i.MX

[vijayranaACL]

Goal

• Deploy RF-DETR Nano (vision transformer + DETR head) on NXP FRDM-IMX95
• Run inference with eIQ Neutron NPU via libneutron_delegate.so
• Need confirmation: Does eIQ support vision-transformer-based models for NPU acceleration?

My Board Details

• Board: NXP FRDM-IMX95 (imx95-15x15-lpddr4x-frdm)
• Kernel: Linux 6.18.2-1.0.0-gf49f45233f7b (aarch64, Feb 2026)
• CPU: 6× ARM Cortex-A55 (500–1800 MHz)
• NPU: eIQ Neutron-S (libneutron_delegate.so)
• Guide used: FRDM i.MX95 Getting Started – NPU section

Model & Pipeline Tried

• Model: RF-DETR Nano (DINOv2 ViT backbone + DETR decoder)
• Input: 384×384, COCO 80 classes
• Flow:
  1. PyTorch → ONNX (opset 18)
  2. ONNX → TFLite via onnx2tf (-fdosm for SavedModel)
  3. TFLite dynamic-range quant (float32 I/O) — CPU inference works
  4. Host: neutron-converter --input model.tflite --output model_neutron.tflite --target imx95
  5. Board: TFLite + Neutron delegate

What Works

• ONNX export succeeds
• TFLite conversion succeeds (~30 MB model)
• CPU inference is correct — detections match PyTorch (dog/person/car on test image)
• Model runs on FRDM with and without delegate

What Fails / Blocked

• No NPU graph created after conversion
  • Inspecting converted .tflite: no neutronGraph / NeutronOperator markers
  • Neutron delegate appears to offload 0 nodes → full CPU fallback
• Full INT8 quantization fails on transformer ops (RANGE, DIV, etc.)
• Used dynamic-range quant instead (weights quantized, float32 I/O)

Why I Think It’s Unsupported (Need Confirmation)

• RF-DETR is transformer-heavy: attention, BatchMatMul, Softmax, LayerNorm, Gather, etc.
• Neutron docs list mainly CNN ops (Conv2D, DepthwiseConv2D, Pooling, Add…)
  • Neutron Converter
  • Supported operators
• Similar split-model approach needed on Rockchip NPU (backbone on NPU, decoder on CPU):
  rfdetr-on-rockchip-npu

My Specific Questions

1. Does eIQ Neutron on i.MX95 support end-to-end vision transformer models (ViT / DETR / RF-DETR)?
2. If not, what is the recommended split (e.g. DINOv2 backbone on NPU, DETR head on CPU)?
3. Are BatchMatMul / Multi-Head Attention supported on Neutron-S for i.MX95 in current SDK?
4. Which eIQ Toolkit version matches BSP kernel 6.18.2-1.0.0 for FRDM-IMX95?
5. Should we use NXP tflite-profiler + tflite-quantizer instead of onnx2tf dynamic-range quant for better NPU compatibility?
   • Model Deployment Quick Start
6. Any reference example for transformer or DETR-style models on i.MX95 Neutron

Commands Used (for reproducibility)

Host (convert):

neutron-converter \

--input rf_detr_nano_full_int8.tflite \

--output rf_detr_nano_neutron_imx95.tflite \

--target imx95

Board (inference):

python3 redetr_test_neutron_imx95.py \

--model rf_detr_nano_neutron_imx95.tflite \

--image dog.jpg \

--delegate /usr/lib/libneutron_delegate.so

Expected issue: converted model has no neutronGraph → no NPU acceleration.

Please confirm:

• Is vision-transformer NPU acceleration supported on i.MX95 today?
• If not, what is the official workaround (model split, different export path, or future roadmap)?

[Oswalag]

Hello,

Q1- Neutron N3.2 introduces dedicated hardware features to efficiently support modern GenAI workloads including Language Models (LLMs), Vision Transformers (ViTs), and transformer-derived operators." It further notes: "Vision Transformers do not require dynamic shapes, and thus integrate naturally into the existing Neutron converter pipeline. Their support is primarily enabled through the extended operator set and improved quantization options (still under development).

Q2- Yes, unsupported ops fall back to CPU automatically

Q3- All tensor quantization must be INT8, not float32. BatchMatMul on Neutron only operates on quantized INT8 tensors

Q4- Upgrade to the latest BSP + SDK 3.2.2 for best transformer operator coverage

Q5- It is correct, that is the recommended path to follow

Q6- There are no public end-to-end DETR examples yet. 

Regards.