2022-06-28 20:43:12.921710951 [I:onnxruntime:, inference_session.cc:1329 Initialize] Session successfully initialized. 2022-06-28 20:43:12.979434216 [I:onnxruntime:, sequential_executor.cc:157 Execute] Begin execution E [_init_tensor:461]Create vx tensor fail. E [_create_tensor:540]Create vx tensor fail. python3: ovxlib_delegate.cpp:839: int nnrt::OvxlibDelegate::addTensor(vsi_nn_graph_t*, vsi_nn_tensor_attr_t*, size_t, const void*, bool): Assertion `false' failed.
For information, I am using the 5.10.72_2.2.0 image and building ORT from source from https://source.codeaurora.org/external/imx/onnxruntime-imx/tree/?h=lf-5.10.72_2.2.0
The onnx graph being tested is the INT8 model from https://github.com/onnx/models/blob/main/vision/classification/mobilenet/model/mobilenetv2-12-int8.o...
Does ONNXRuntime actually work with INT8 and the VSI_NPU? The model in the guide (https://github.com/onnx/models/blob/main/vision/classification/mobilenet/model/mobilenetv2-7.onnx) is a FP32 model, which runs on CPU and will complete but defer back to the CPU if vsi_npu is selected.
Solved! Go to Solution.
Hi,
According reply from R&D, VSI NPU execution provider in ONNX support for quantized models is very limited (I would say none). And the Modelrunner only uses ONNX APIs too.
There is currently an ongoing migration from NNRT and nn-imx modules to TIM-VX and so as a part of this migration we expect the support for quantized ops. No estimates or dates for the delivery, it is a long-term plan roughly targeting the end of 2022.So, I advise ctms not to use onnxruntime.
Regards
Hi,
According reply from R&D, VSI NPU execution provider in ONNX support for quantized models is very limited (I would say none). And the Modelrunner only uses ONNX APIs too.
There is currently an ongoing migration from NNRT and nn-imx modules to TIM-VX and so as a part of this migration we expect the support for quantized ops. No estimates or dates for the delivery, it is a long-term plan roughly targeting the end of 2022.So, I advise ctms not to use onnxruntime.
Regards
Can you please provide an update on support for NPU execution of onnx models?
Thanks,
Matt