Possible problem with resultant eIQ-Portal's Augmentation-Tool generated images

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Possible problem with resultant eIQ-Portal's Augmentation-Tool generated images

2,428 Views
jonmg
Contributor I

Hello,

I am using the eIQ Portal to import my own dataset and train an object detection model.

Within my dataset there are some images with more than one object being labled. See attached the CapturaX_labelled images. I noticed however, that the new sample images generated and presented by the Augmentation tool contains just one label despite containing several target objects on it. 

For example an input image containing 4 labelled pistols will be augmented (being flipped horizontally as seen in the attached pictures) but only one pistol is shown as being labelled. 

Please let me know if this is just a visualisation/presentation problem or if in contrary the augmented resultant picture contains really a single object labelled? In this case some of the augmented images would be decreasing the quality of the dataset instead of improving it. 

 

0 Kudos
10 Replies

2,386 Views
david_piskula
NXP Employee
NXP Employee

Hello @jonmg,

I tried reproducing your issue with a random dataset and I haven't been able to reproduce it. Additionally, looking at the screenshots you provided, I believe you are using an outdated version of eIQ Portal. Can you please download the latest version of the tool and let me know if you can still reproduce your issue? (If so, please provide your dataset or at least some sample of it and tell me the exact steps to reproduce it.)

https://www.nxp.com/design/software/development-software/eiq-ml-development-environment/eiq-toolkit-...

Thanks,

David

0 Kudos

2,380 Views
jonmg
Contributor I

Hello David,

You are right, I was using an older version of the eIQ Portal (the eIQ Toolkit v1.0.5) . With this tool I achieved to train a default eIQ available object detection model with my open source dataset but without augmentation.  

When I moved to the newest version (1.1.8), the visualisation improved. I was however not able to train any eIQ default model with the same open source dataset. I could not train the model neither with nor without the augmentation data activated. 

I am using the following dataset for an object detection model training with two object classes (pistol and knife):

https://github.com/ari-dasci/OD-WeaponDetection/tree/master/Pistol%20detection

https://github.com/ari-dasci/OD-WeaponDetection/tree/master/Knife_detection

When I try to train the model I am not able to achieve even one Epoch. The loss goes to infinit almost and the accuracy goes to 0. I attach some images showing the effect.

 

Best regards,

Jon 

0 Kudos

2,346 Views
david_piskula
NXP Employee
NXP Employee

Thank you for sharing the project files over email. I am now able to reproduce your behavior. I will investigate together with the development team and get back to you when I have more information.

By the way, when you said you were able to train the model in the v1.0.5 without augmentation, what accuracy did you achieve?

0 Kudos

2,341 Views
jonmg
Contributor I

My intention in this moment is to learn about the eIQ Tool and not so much about achieving a very good model so I stopped after 10 Epochs achieving a 0.83% accuracy as seen in the image bellow. I think you should be able to open my training status as I also sent you the model training checkpoints. 

jonmg_0-1637247652409.png

 

0 Kudos

2,337 Views
david_piskula
NXP Employee
NXP Employee

I see, thank you. I just wanted to confirm that the dataset can be used to train a model with a somewhat useful accuracy. The accuracy rises very slowly in the v1.0.5 so I wanted to know if it actually keeps getting better over a long time period or not. Seems like it works fine, so that's good.

0 Kudos

2,331 Views
jonmg
Contributor I

Try using the Adam optimiser instead of the SGD. This shouls remarkably increase the learning speed.

0 Kudos

2,375 Views
david_piskula
NXP Employee
NXP Employee

I see, would you be able to provide me with an eIQ Portal project file that contains the dataset? It would help me to reproduce your issue faster. Thanks.

0 Kudos

2,368 Views
jonmg
Contributor I

The project containing the dataset is much bigger than the maximum 25MB size to be attached. How can we do it?

0 Kudos

2,362 Views
david_piskula
NXP Employee
NXP Employee

Could you try google drive or maybe something like https://wetransfer.com/ since the project isn't confidential? You can either send the link to david.piskula@nxp.com or click the 3 dots next to the Transfer button and choose Get a transfer link and post it here.

0 Kudos

2,388 Views
david_piskula
NXP Employee
NXP Employee

Hello @jonmg,

thank you for reporting this problem. Can you please let me know which version of the tool you are using? (You can find the version number in "Help" -> "About eIQ Portal" )

This definitely doesn't sound like the expected behavior and I will need to report it to the development team. I will let you know what they respond as soon as possible.

Regards,

David

0 Kudos