ML-based monitor

[khinhtethtetaung25]

Hi all!

I am using Application Software Pack - ML-base System State Monitor on LPCXpresso55S69 for classifying different states of the fan (ON, OFF, CLOG, FRICTION). The prediction accuracy percentage on confusion matrix  for all state is over 95% on validation dataset. However, when tested the model on real-time dataset, the model cannot differentiate very clearly on ON and CLOG state. The result always swings between 2 states. From the Jupyter-notebook, I noticed that the model trained by using Ax,Ay,Az data.  May I know training the model with addition of Bx,By,Bz data would resolve the problem? Would adding 2 fans make any improvement on this inaccuracy? 

Thank you so much!

#LPCXpresso55s69 #ml_state_monitor_cm33 @https://community.nxp.com/t5/eIQ-Machine-Learning-Software/Application-Software-Pack-ML-based-System-State-Monitor/ta-p/1413290

[Alice_Yang]

Hello @khinhtethtetaung25 

In normal, more Bx,By,Bz data would be helpful, you can test.

Additional, please confirm real-time dataset and validation dataset come from the same fan and same collection conditions. And use another  test dataset  to determine the generalization ability of the model.

BR

Alice

[khinhtethtetaung25]

Hi @Alice_Yang 

Thank you so much for the reply. I modified the model in Jupyter-notebook to include Bx, By, Bz data and deploy the tensorflow model on the MCUXpresso IDE. The result on real-time test data got even worse. It cannot recognize all four states. How do I change the source code on the IDE to properly interface with the new model and process the real-time data? 

Best regards

[Alice_Yang]

Hello @khinhtethtetaung25 

About your this ML question, I have help to ask AE team for help, they give some suggestions as below:

"

 1. If we use the Bx,By,Bz data for training, then the MCU project need also these datas, that means, we need to change the source code, especially the data feed to the model(The model's input data). And make that the datas include the Bx, By, Bz

    2. when tested the model on real-time dataset, the model cannot differentiate very clearly on ON and CLOG state?

        About this, please have a try, a) the preprocessing way is same on PC and MCU. b) how you get the training/Val data for training, from the MCU real-time or not? c) The model is quantized, right? so that can have a try the fp32 model(directly from the training-stage, and do not by quantize). For the quantization may cause the accuracy down.

"

BR

Alice

[khinhtethtetaung25]

Hi @Alice_Yang 

Thank you so much for the reply. I have checked that the preprocessing method is the same on PC and MCU, and I have also tried using the fp32 model. However, the results have not improved.

As a novice in MCU programming, could you please let me know where in the source code I should make changes to include Bx, By, and Bz data?

Best regards,

Khin

[Alice_Yang]

Hello @khinhtethtetaung25 

I have talked with our SE again about this issue, give you some sugestions:

1)  First of all, run the demo under software package,  

https://www.nxp.com/design/software/embedded-software/application-software-packs/application-softwar...  

check whether can work well.

2) If the demo can not work, what about your hardware? It is custom board? 

3) For state ON and CLOG,  It's really not easy to distinguish, try to make big difference for them. for the FAN-Clogged, make it more sealed. 

From AN:

"•FAN-ON – the fan is turned on and running in normal conditions
•FAN-Clogged – the fan is turned on and the airflow is obstructed

ON while for the Clogged class a thin piece of cardboard was used to block the airflow "

4)If change input, need retraining model.

BR

Alice