NPU bad detection with Yolov5 - i.MX8MP

cancel
Showing results for 
Search instead for 
Did you mean: 

NPU bad detection with Yolov5 - i.MX8MP

709 Views
simoberny
Contributor II

Hi, 

I'm quite struggling for some time now trying to get NPU detection to work with a C++ program. The same code on the CPU gets optimal results, but using VX delegate the detections are completely wrong. The code seems to run smoothly and inference shows good timing (yolov5s model with 448x448 input ~ 70ms). 

Right now I'm trying with Yolov5 (uint8 quantized), but I have tried with different pre-trained models obtaining the same behavior, good detection on CPU, and random detection on NPU. 

To obtain the model I used the export from yolov5 repo: 

 python export.py --weights yolov5s.pt  --imgsz 448 --include tflite --int8

I've also tried TFlite hub models like SSD and mobilenet, that have already been converted to uint8. 

 

In the attachment the piece of code I am using for the inference and the converted yolov5n model. 

What could it be the cause? 

 

Thanks,

Best regards

0 Kudos
11 Replies

487 Views
sams4
Contributor I

@simoberny : I am facing the same issue . For me the detections are showing correctly when printed on console. The issue is with the bounding box co-ordinates. Co-ordinates of detected objects are random and some are negative as well. Can you or NXP support help on this matter?

Thanks!!!

0 Kudos

452 Views
simoberny
Contributor II

Which version of BSP? 

So you have correct labels and predictions, but wrong bounding boxes? 

In my case, everything seems wrong. The results seem totally random. 

 

From the Variscite customer helpdesk, they say that the model should be rebuilt to be NPU-compatible. They sent me an optimized small mobile-net SSD model and the detections are perfect. But is actually a pain in the ass to train. 

At least I know for sure that the problem is entirely related to the model itself. For now, I'll use CPU with a smaller YOLO model, In the hope of finding a way to use it with the NPU.

 

Bests

 

0 Kudos

346 Views
sams4
Contributor I

@simoberny : The BSP is upgraded to 5.15.32_2.0.0 version and it worked.

Detections and bounding boxes started appearing.

Thanks!!!

0 Kudos

683 Views
Bio_TICFSL
NXP TechSupport
NXP TechSupport

Hello,

Attached you will find some benchmark on vx delegate and MX8MPlus, also it is an appnote on object detection.

Hope this helps

160 Views
hy982530
Contributor I

Hello, @Bio_TICFSL 

My version of BSP is 5.15.71.

I execute yolov5s-32fp-256.tflite on npu according to your teaching. My program is python and the .tflite has correct result on cpu. However, on npu, I get correct labels and predictions, but wrong bounding boxes. 

in addition, some error appear when I use VX delegate. Can you tell me what's wrong?

hy982530_0-1679544743981.png

Thank you, 

Best regards 

 

0 Kudos

113 Views
hy982530
Contributor I

I added cl_viv_vx_ext.h in version 5.15.71, and the error message in the picture disappeared.

However, I got the same result: correct labels and predictions, but wrong bounding boxes.

0 Kudos

117 Views
hy982530
Contributor I

I roll back the version of BSP to 5.15.32 and the bounding boxes are correct!

I want to confirm whether version 5.15.71 is the cause of the incorrect results. Has anyone encountered the same problem?

Thanks

0 Kudos

680 Views
simoberny
Contributor II

Thanks for the response and the documentation. 

The guide actually describes what I already did.

For the sake of scruple, I followed all the steps and recreated a new model. But the situation remains the same, on CPU it works perfectly, instead on NPU I have no result except random detection with really low confidence. I tried both with INT8 quantized and FLOAT.

I am on Yocto 5.15.52-2.1.0 which uses Tensorflow 2.5.0 as default. I'm now trying to compile a newer version. 

Another strange behavior is that when I use the VX delegate I can't gently close the application, because Segmentation Fault occurs. VX delegate is compiled to the last version with git official repo. 

Thanks

0 Kudos

621 Views
simoberny
Contributor II

I wanted to clarify that the version I'm working on is 5.10.52.

Also, the yolov5_decode python script used in the guide is not accessible

0 Kudos

580 Views
Bio_TICFSL
NXP TechSupport
NXP TechSupport

Hi,

 

At least, You have to change to 5.15.71 BSP.

Regards

0 Kudos

453 Views
simoberny
Contributor II

Unfortunately, this is not an option for me since I'm using a Basler camera and for that BSP version, there is still no driver available. Actually with the Variscite board, right now, the BSP is only up to 5.15.60. 

Could I ask what's the difference in the 5.15.70 version, that enables the use of NPU with Yolo models? 

I will try in the future, but for now, I'm forced to use the CPU with a smaller model to have a decent time.

Thank you, 

Best regards 

 

 

0 Kudos