eIQ Machine Learning SoftwareのトピックRunning ARMNN CAFFE Binary on IMX8 board with GPU Acceleration https://community.nxp.com/t5/eIQ-Machine-Learning-Software/Running-ARMNN-CAFFE-Binary-on-IMX8-board-with-GPU-Acceleration/m-p/1277578#M365 <P>Hi Support Team,</P><P>Right now trying to run the ARMNN CaffeTest on iMX8 Board. I tried to run Inception-BNbatchsize1.<BR />caffemodel with GPU Acceleration but the performance is not as expected. We are following the<BR />i.MX_Machine_Learning_User's_Guide for running the ARMNN Tests.<BR />We are running the Arm NN Caffe binary as below :</P><P><BR /><STRONG>With CPU :</STRONG><BR />-----------</P><P>root@imx8qxpmek:/usr/bin/ArmnnTests# CaffeInception_BN-Armnn --compute=CpuAcc --data-dir=data --model-dir=models<BR />ArmNN v20190801<BR />= Prediction values for test #0<BR />Top(1) prediction is 3694 with value: 0.940841<BR />Top(2) prediction is 3197 with value: 0.000752777<BR />Top(3) prediction is 2411 with value: 0.000105063<BR />Top(4) prediction is 1972 with value: 4.33676e-05<BR />Top(5) prediction is 1081 with value: 4.27122e-05<BR />Total time for 1 test cases: 0.831 seconds<BR />Average time per test case: 831.004 ms<BR />Overall accuracy: 1.000</P><P><STRONG>With GPU Backend:</STRONG><BR /><STRONG>----------------------------</STRONG></P><P>root@imx8qxpmek:/usr/bin/ArmnnTests# CaffeInception_BN-Armnn --compute=VsiNpu --data-dir=data --model-dir=models<BR />ArmNN v20190801<BR />W [query_hardware_caps:53]Unsupported evis version<BR />Warn-Start NN execution<BR />Warn-Start NN execution<BR />= Prediction values for test #0<BR />Top(1) prediction is 3694 with value: 0.899821<BR />Top(2) prediction is 3197 with value: 0.000630801<BR />Top(3) prediction is 2411 with value: 0.000116962<BR />Top(4) prediction is 1081 with value: 4.72108e-05<BR />Top(5) prediction is 567 with value: 1.84702e-05<BR />Total time for 1 test cases: 2.073 seconds<BR />Average time per test case: 2073.426 ms<BR />Overall accuracy: 1.000</P><P>In the above example with GPU acceleration ,we can see the Total time for the test cases taken is more<BR />than the one computed with CPU.. and we can see this error when the demo is run with GPU acceleration:</P><P><STRONG>W [query_hardware_caps:53]Unsupported evis version</STRONG><BR /><STRONG>Warn-Start NN execution</STRONG><BR /><STRONG>Warn-Start NN execution</STRONG><BR /><BR /><STRONG>and Total Time taken is more when run with GPU acceleration compared to CPU?</STRONG></P><P><BR />Can anyone please guide me what could be the issue?</P><P>Thanks..</P> Mon, 17 May 2021 11:37:40 GMT Divya17 2021-05-17T11:37:40Z Running ARMNN CAFFE Binary on IMX8 board with GPU Acceleration https://community.nxp.com/t5/eIQ-Machine-Learning-Software/Running-ARMNN-CAFFE-Binary-on-IMX8-board-with-GPU-Acceleration/m-p/1277578#M365 <P>Hi Support Team,</P><P>Right now trying to run the ARMNN CaffeTest on iMX8 Board. I tried to run Inception-BNbatchsize1.<BR />caffemodel with GPU Acceleration but the performance is not as expected. We are following the<BR />i.MX_Machine_Learning_User's_Guide for running the ARMNN Tests.<BR />We are running the Arm NN Caffe binary as below :</P><P><BR /><STRONG>With CPU :</STRONG><BR />-----------</P><P>root@imx8qxpmek:/usr/bin/ArmnnTests# CaffeInception_BN-Armnn --compute=CpuAcc --data-dir=data --model-dir=models<BR />ArmNN v20190801<BR />= Prediction values for test #0<BR />Top(1) prediction is 3694 with value: 0.940841<BR />Top(2) prediction is 3197 with value: 0.000752777<BR />Top(3) prediction is 2411 with value: 0.000105063<BR />Top(4) prediction is 1972 with value: 4.33676e-05<BR />Top(5) prediction is 1081 with value: 4.27122e-05<BR />Total time for 1 test cases: 0.831 seconds<BR />Average time per test case: 831.004 ms<BR />Overall accuracy: 1.000</P><P><STRONG>With GPU Backend:</STRONG><BR /><STRONG>----------------------------</STRONG></P><P>root@imx8qxpmek:/usr/bin/ArmnnTests# CaffeInception_BN-Armnn --compute=VsiNpu --data-dir=data --model-dir=models<BR />ArmNN v20190801<BR />W [query_hardware_caps:53]Unsupported evis version<BR />Warn-Start NN execution<BR />Warn-Start NN execution<BR />= Prediction values for test #0<BR />Top(1) prediction is 3694 with value: 0.899821<BR />Top(2) prediction is 3197 with value: 0.000630801<BR />Top(3) prediction is 2411 with value: 0.000116962<BR />Top(4) prediction is 1081 with value: 4.72108e-05<BR />Top(5) prediction is 567 with value: 1.84702e-05<BR />Total time for 1 test cases: 2.073 seconds<BR />Average time per test case: 2073.426 ms<BR />Overall accuracy: 1.000</P><P>In the above example with GPU acceleration ,we can see the Total time for the test cases taken is more<BR />than the one computed with CPU.. and we can see this error when the demo is run with GPU acceleration:</P><P><STRONG>W [query_hardware_caps:53]Unsupported evis version</STRONG><BR /><STRONG>Warn-Start NN execution</STRONG><BR /><STRONG>Warn-Start NN execution</STRONG><BR /><BR /><STRONG>and Total Time taken is more when run with GPU acceleration compared to CPU?</STRONG></P><P><BR />Can anyone please guide me what could be the issue?</P><P>Thanks..</P> Mon, 17 May 2021 11:37:40 GMT https://community.nxp.com/t5/eIQ-Machine-Learning-Software/Running-ARMNN-CAFFE-Binary-on-IMX8-board-with-GPU-Acceleration/m-p/1277578#M365 Divya17 2021-05-17T11:37:40Z