Possible problem with resultant eIQ-Portal's Augmentation-Tool generated images

[jonmg]

Hello,

I am using the eIQ Portal to import my own dataset and train an object detection model.

Within my dataset there are some images with more than one object being labled. See attached the CapturaX_labelled images. I noticed however, that the new sample images generated and presented by the Augmentation tool contains just one label despite containing several target objects on it. 

For example an input image containing 4 labelled pistols will be augmented (being flipped horizontally as seen in the attached pictures) but only one pistol is shown as being labelled. 

Please let me know if this is just a visualisation/presentation problem or if in contrary the augmented resultant picture contains really a single object labelled? In this case some of the augmented images would be decreasing the quality of the dataset instead of improving it. 

[david_piskula]

Hello @jonmg,

I tried reproducing your issue with a random dataset and I haven't been able to reproduce it. Additionally, looking at the screenshots you provided, I believe you are using an outdated version of eIQ Portal. Can you please download the latest version of the tool and let me know if you can still reproduce your issue? (If so, please provide your dataset or at least some sample of it and tell me the exact steps to reproduce it.)

https://www.nxp.com/design/software/development-software/eiq-ml-development-environment/eiq-toolkit-...

Thanks,

David

[jonmg]

Hello David,

You are right, I was using an older version of the eIQ Portal (the eIQ Toolkit v1.0.5) . With this tool I achieved to train a default eIQ available object detection model with my open source dataset but without augmentation.  

When I moved to the newest version (1.1.8), the visualisation improved. I was however not able to train any eIQ default model with the same open source dataset. I could not train the model neither with nor without the augmentation data activated. 

I am using the following dataset for an object detection model training with two object classes (pistol and knife):

- https://github.com/ari-dasci/OD-WeaponDetection/tree/master/Pistol%20detection

- https://github.com/ari-dasci/OD-WeaponDetection/tree/master/Knife_detection

When I try to train the model I am not able to achieve even one Epoch. The loss goes to infinit almost and the accuracy goes to 0. I attach some images showing the effect.

Best regards,

Jon 

[david_piskula]

Thank you for sharing the project files over email. I am now able to reproduce your behavior. I will investigate together with the development team and get back to you when I have more information.

By the way, when you said you were able to train the model in the v1.0.5 without augmentation, what accuracy did you achieve?

[jonmg]

My intention in this moment is to learn about the eIQ Tool and not so much about achieving a very good model so I stopped after 10 Epochs achieving a 0.83% accuracy as seen in the image bellow. I think you should be able to open my training status as I also sent you the model training checkpoints. 

[david_piskula]

I see, thank you. I just wanted to confirm that the dataset can be used to train a model with a somewhat useful accuracy. The accuracy rises very slowly in the v1.0.5 so I wanted to know if it actually keeps getting better over a long time period or not. Seems like it works fine, so that's good.

[jonmg]

Try using the Adam optimiser instead of the SGD. This shouls remarkably increase the learning speed.

[david_piskula]

I see, would you be able to provide me with an eIQ Portal project file that contains the dataset? It would help me to reproduce your issue faster. Thanks.

[jonmg]

The project containing the dataset is much bigger than the maximum 25MB size to be attached. How can we do it?

[david_piskula]

Could you try google drive or maybe something like https://wetransfer.com/ since the project isn't confidential? You can either send the link to david.piskula@nxp.com or click the 3 dots next to the Transfer button and choose Get a transfer link and post it here.

[david_piskula]

Hello @jonmg,

thank you for reporting this problem. Can you please let me know which version of the tool you are using? (You can find the version number in "Help" -> "About eIQ Portal" )

This definitely doesn't sound like the expected behavior and I will need to report it to the development team. I will let you know what they respond as soon as possible.

Regards,

David