GC7000UL general-compute inference path on i.MX8M Plus and TIM-VX/VSINPU appears NPU-only Board/BSP: i.MX8M Plus, aarch64 Galcore version 6.4.11.p2.745085 ONNX Runtime with VSINPUExecutionProvider (statically linked against libtim-vx.so) Vivante OpenCL ICD present and functional (Vivante.icd → libVivanteOpenCL.so) Goal: Run ResNet50 inference benchmarks (MLPerf loadgen harness) on the GC7000UL 3D GPU core specifically, as a comparison point against existing NPU (VIP8000Nano) and CPU benchmark results already collected. What's confirmed working: clGetPlatformIDs/clGetDeviceIDs via the Vivante OpenCL ICD cleanly enumerates two independent devices under one platform: Device 0: GC7000UL.6204.0000 Device 1: VIP8000Nano-S+I.8002.0000 Both report CL_DEVICE_TYPE_ACCELERATOR, no errors, confirmed via a minimal C test program linked against libOpenCL.so → libGAL.so. What's blocking GPU dispatch via ORT: ort.get_available_providers() returns only ['VSINPUExecutionProvider', 'CPUExecutionProvider'] — no OpenCL-based EP. VSINPUExecutionProvider is statically linked to libtim-vx.so (OVXLIB/vsi_nn_* API). Symbol/string dump of both libtim-vx.so and libGAL.so shows no DEVICE_INDEX/DEVICE_ID-style env var or config surface — only behavior toggles (VIV_VX_ENABLE_SHADER, VSI_NN_ENABLE_*, etc). libGAL.so does export gcoHAL_SetDeviceIndex/gcoHAL_GetCurrentDeviceIndex at the raw HAL layer, but there's no visible plumbing from OVXLIB/TIM-VX down to that call which is suggesting the graph compiler used by VSINPU may be hardcoded to target the NPU core only, regardless of device index. Specific question: Does TIM-VX / OVXLIB on this BSP (galcore 6.4.11.p2) support compiling and dispatching a graph to the GC7000UL as a general-compute target, or is the graph compiler NPU-only by design in this build, and how can I verify if it is possible to run it that way ? If GPU-target graph compilation is supported upstream in TIM-VX but not enabled in this NXP-shipped build, is there a build flag / SDK component that exposes it? If there is no supported path through TIM-VX/ORT, is there an NXP-recommended way to run generic inference on the GC7000UL directly (e.g. via the OpenCL/OpenVX layer, since that portion of the stack is confirmed functional) , a sample app, SDK component, or reference implementation we should be building against instead? as a currently a student, and trying to work on this implementation and running an ORT on TOP of the GPU, is there any way, or any other way to be able use the GPU for inference ? Thank you very much IMX8MPLUS #GC7000UL Re: GC7000UL general-compute inference path on i.MX8M Plus and TIM-VX/VSINPU appears NPU-only HI @WaleedO,
Thank you for contacting NXP Support.
To run inference on the GPU, you should use the GPU delegate, which enables supported operations to be accelerated by the GPU instead of running entirely on the CPU.
I recommend reviewing our Machine Learning User Guide to better understand the available execution backends, delegate configuration, supported frameworks, and example applications. The guide also includes step-by-step examples that can help you validate that the GPU delegate is being loaded correctly and that your model is executing as expected.
If you encounter any issues during setup or execution, please share the model, BSP version, and the commands you are using, and I will be happy to assist further.
Best regards, Alejandro Garcia Re: GC7000UL general-compute inference path on i.MX8M Plus and TIM-VX/VSINPU appears NPU-only Hello @Chavira Nice to meet you. After checking the documentation, the GPU delegate and the OpenCL path is used within i.MX 95/952 GPU (Arm Mali G310). I am curently working on The IMX8MPLUS . the imx8m Plus have this stack: VX delegate ==> TIM-VX ==> GPU/NPU (unified driver) ==> I.MX 8 series NPU and GPU (GC7000,GC7000L, GC7000UL). as per the documentation. Currently, I am working with ONNX and ORT. When I run the execution, It is per default running on the NPU. Is there any way to use the OpenCL to work ont the IMX8MPLUS GPU ? or if there is any manual override, or technique that I can implement, so that I can manually set the compilation toward either NPU or/And GPU ? Thank Your very much for your reply. Kind regards, IMX8MPLUS #TIM-VX #VX-delegate Re: GC7000UL general-compute inference path on i.MX8M Plus and TIM-VX/VSINPU appears NPU-only Hello, Yes there is still a possible path, but I would not try to force VSINPU Execution Provider to use the GC7000UL. Your results strongly suggest that the current NXP TIM-VX/OVXLIB build is wired primarily for the VIP8000 NPU. gcoHAL_SetDeviceIndex() in libGAL.so alone doesn't mean TIM-VX can redirect NN graphs to the GPU. First test TIM-VX/OpenVX directly, outside ONNX Runtime, with a small model. Check whether your specific OVXLIB exposes a GC7000UL/GPU NN target. Compare your NXP TIM-VX build with upstream TIM-VX, especially its multi-device/platform support. If TIM-VX cannot compile for GC7000UL, use the Vivante OpenCL/OpenVX stack directly for the GPU benchmark. For your MLPerf project, you don't necessarily need ORT for the GPU path. You can make a separate GC7000UL backend and connect it to the same LoadGen harness used for CPU/NPU. So I would structure it as: MLPerf LoadGen → ResNet50 GPU backend → OpenCL/OpenVX → GC7000UL rather than: MLPerf → ORT → VSINPU → somehow force GPU Best Regrad, fesaji
記事全体を表示
