Conv2D on i.MX 8M Plus VX delegate produces incorrect output when either spatial kernel dim >= 16 Summary On i.MX 8M Plus EVK, Conv2D ops where either spatial kernel dimension (H or W) is >= 16 produce incorrect output under the VX delegate (libvx_delegate.so). The model loads, the Conv2D is bound to an OPENVX kernel by vsi_nn_kernel_selector, error_during_init/prepare/invoke are all 0, and inference latency is normal, but the int8 output is saturated to a single value (int8 = 127) at every position across diverse inputs. The boundary is sharp between K=15 and K=16 along either spatial axis. This matches the threshold of the documented stride > 15 limit (https://community.nxp.com/t5/i-MX-Processors/Conv2D-not-working-with-stride-16-for-NPU-Kernel-crashes/td-p/1754217). Two questions: Is this kernel-dim ≥ 16 cliff a known limitation, and if so could it be added to the i.MX Machine Learning User's Guide (UG10166) Conv2D constraints section? Should vsi_nn_op_conv2d::op_check (or the VX delegate's partition logic) reject this case at graph-compile time so the op falls back to CPU? The current "accept, run, return constant int8=127, no diagnostic" behaviour is unsafe — a deployed model with a large kernel appears to load and run normally. Environment Component Value Board i.MX 8M Plus EVK BSP NXP i.MX Release Distro 6.18-whinlatter (VERSION_ID=6.18-whinlatter) Kernel Linux 6.18.2-1.0.0-gf49f45233f7b SMP PREEMPT (aarch64) libvx_delegate.so md5 2f88ec0871d18298bfa357ddaaea4d6d libGAL.so md5 af4806f617b23363b3be69c4dad2dc05 libOpenVX.so{,.1,.1.3.0} md5 92f85c32746d4d0b38800e21503d13d0 (all three identical) imx-gpu-viv package 1:6.4.11.p4.4-aarch64-r0 tim-vx package 1.2.2-r0 OVxlib (runtime-reported) OVXLIB_VERSION==1.2.14 TFLite runtime TFLite 2.19.0 (/usr/bin/tensorflow-lite-2.19.0/examples/benchmark_model) Reproducer The attached conv_1x17_broken.tflite (2,136 bytes) is a single-Conv2D artifact: kernel (1, 17), input 1×1×1000×5 int8, output 1×1×1000×8 int8, per-channel int8 weight quantization, padding=SAME, stride=(1,1), no fused activation. # On the i.MX 8M Plus board:
python3 reproduce.py conv_1x17_broken.tflite # exits 1: BROKEN, saturated to int8=127
python3 reproduce.py conv_1x15_torch_control.tflite # exits 0: OK (K<16, same pipeline) reproduce.py (attached, depends only on tflite_runtime and numpy) feeds 6 seeded random int8 inputs through plain and VX-delegated interpreters and reports mean|Δ|, max|Δ|, and the count of unique int8 output values from the VX path. Kernel-size sweep Single-Conv2D unit-test models, INT8 PTQ. mean|Δ| board VX vs board CPU on the same int8 input bytes; uniq_VX = unique int8 output values across 6 inputs (out of 256). kernel (H, W) mean |Δ| uniq_VX result (1, 11) 0.03 256 OK (1, 15) 0.02 256 OK (1, 16) 134.9 1 BROKEN (constant int8=127) (1, 17) 136.9 1 BROKEN (constant int8=127) (1, 23) 130.5 1 BROKEN (constant int8=127) (15, 1) 0.11 256 OK (17, 1) 136.9 1 BROKEN (constant int8=127) (23, 1) 129.3 1 BROKEN (constant int8=127) (3, 3), (5, 5), (7, 7) < 0.2 256 OK (15, 15) (area 225) 0.41 256 OK (3, 23) 123.4 1 BROKEN (constant int8=127) (5, 15) 0.24 256 OK The threshold applies independently to H and W. In every broken case the VX output saturates to int8 = 127 at every position — saturation to the positive int8 extreme (rather than to output_zp or 0) suggests either multiplier-shift overflow before the final clamp, or a fixed value being written into the output tile in place of the MAC result. The issue doesn't seem related to kernel area (e.g., 15x15 works fine) The threshold is necessary but not sufficient — some int8 weight value patterns at K ≥ 16 trigger the bug, others don't. We've done byte-level isolation experiments (grafting individual TFLite tensor fields between broken and non-broken artifacts) narrowing the value-pattern trigger to specific tensor fields. Happy to share the isolation results and tooling if useful. Verbose log excerpt (VSI_NN_LOG_LEVEL=5, broken case) INFO: Vx delegate: error_during_init set to 0.
INFO: Vx delegate: error_during_prepare set to 0.
INFO: Vx delegate: error_during_invoke set to 0.
I [vsi_nn_CreateGraph:1327] OVXLIB_VERSION==1.2.14
D [setup_node:535] Setup node id[3] uid[30000] op[DATACONVERT]
D [setup_node:535] Setup node id[0] uid[1] op[PERMUTE]
D [setup_node:535] Setup node id[1] uid[2] op[CONV2D]
D [setup_node:535] Setup node id[2] uid[3] op[PERMUTE]
D [setup_node:535] Setup node id[4] uid[30001] op[DATACONVERT]
D [vsi_nn_kernel_selector:1286] Instance OPENVX node with kernel "conv2d" Full log attached as vsi_nn_log_level_5_conv_1x17.txt. Attachments Bundled as bug_report_artifacts.tar.gz conv_1x17_broken.tflite — 2,136-byte broken artifact. conv_1x15_torch_control.tflite — 2,056-byte sub-threshold control, same pipeline. reproduce.py — board-side reproducer (tflite_runtime + numpy only). vsi_nn_log_level_5_conv_1x17.txt — full verbose log. Additional artifacts available on request: byte-level isolation tooling, a structurally-identical working (1, 17) artifact for delta analysis, full sufficient-condition graft results. i.MX 8M | i.MX 8M Mini | i.MX 8M Nano Linux Suspected Software Defect Re: Conv2D on i.MX 8M Plus VX delegate produces incorrect output when either spatial kernel dim > Hi @themis_stewart Thanks for your information, i am checking with internal team about your two questions. Best Regards, Zhiming Re: Conv2D on i.MX 8M Plus VX delegate produces incorrect output when either spatial kernel dim > Hello @themis_stewart Please try the following patch based on L6.18.2 to fallback the op to the CPU if the kernel size is > 16 Best Regards, Zhiming Re: Conv2D on i.MX 8M Plus VX delegate produces incorrect output when either spatial kernel dim > Thanks very much for the quick turnaround on this — we've applied and tested the guard patch and can confirm it resolves the issue on our side. We built the patched libvx_delegate.so from the lf-6.18.2_1.0.0 delegate source with your op_map.cc change and validated it on our i.MX 8M Plus EVK against the minimal reproducers from the original report. With the patch: The affected INT8 Conv2D (spatial kernel ≥ 16) is now cleanly rejected by the delegate and falls back to TFLite CPU, with the log message making the fallback explicit — no more silent saturation to int8 = 127. Output matches the CPU reference exactly. Our production models (kernels ≤ 15) are unaffected — still fully delegated to the NPU, so the guard doesn't over-trigger. The CPU-fallback path is performing well for us in practice, though the fallback is naturally slower than a model able to run fully on the NPU. We're happy to share detailed before/after benchmarks privately if they'd be useful for your regression coverage. Thanks again — much appreciated.
查看全文
