2389684_en-US

[#1]

Why results from NPU tflite model and tflite model are different?

I have quantized classification model. I convert to NPU tflite model with command 

./neutron-converter \
--input QAT.tflite \
--output QAT_NPU.tflite \
--target imxrt700 \
--dump-header-file-output \
--dump-header-file-input \
--use-sequencer

After that, I use 2 generated model header files for NPU and CPU.

I use the sample tflm_cifar10_cm33_core0, modified for our models. I use the sample image_data.h (resized image to model input size). But the final results of 2 models (on CPU and NPU modes) are different:

- In almost cases, the predicted class is same with similar probability (not exactlty match by values)

- In some cases, the predicted classes in 2 modes are different ==> Do you have any comment for this problem? Sorry I can not share my model.

Re: Why results from NPU tflite model and tflite model are different?

I tried to verify this problem with the sample tflm_cifar10_cm33_core0. But in this sample, there is only NPU tflite model, I did not see the other one (CPU tflite model). I want to compare predicted results with different images to see whether this problem is happened with model pretrained by NXP.

If you have CPU tflite model (correspond NPU tflite model tflm_cifar10_cm33_core0), please share with me.

I am curious about whether conversion from tflite model to NPU tflite model results in difference of inference's results.

Thank you.

Re: Why results from NPU tflite model and tflite model are different?

@mayliu1 Hi could you help me about this problem?

Sorry, I feel that the number of NXP's supporters in i.MX RT is small and questions are sometimes missed. Before, I worked with MIMXRT1060 and N947, I got response very quickly.

Re: Why results from NPU tflite model and tflite model are different?

Hi @nnxxpp,

It is expected that after the model conversion process, you see slight differences on the output values, due to the fact that the Neutron Converter restructures the model into NeutronGraph nodes for NPU execution, rather than executing the original graph on an operator basis like it would be done on a CPU-based TFLM.

That said, if the outputs are too different, resulting in miss predicted classes on too many occasions, it would be important to check things like: The neutron convertor version and neutron libraries version used on runtime to ensure matching SW, memory configuration used for the NPU, as well as inspecting the converted nodes to ensure the whole model was correctly converted rather than only partially.

BR,
Edwin.

Re: Why results from NPU tflite model and tflite model are different?

Hi @nnxxpp ,

Thank you for sharing your feedback.

Your case is currently being followed by my colleague, Edwin, who is actively working on it. We would appreciate your patience while the investigation continues. Edwin will continue to follow up on this matter and keep you informed of any progress.

Thank you for your understanding.

Best regards,

May

Re: Why results from NPU tflite model and tflite model are different?

@mayliu1 

Oh, I am very happy to hear that from you. Thank you so much for supporting. I will wait good news from you.