eIQ Sample Apps - Handwritten Digit Recognition

Document created by Diego Dorta Employee on Jun 21, 2019Last modified by Markus Levy on Nov 11, 2019
Version 11Show Document
  • View in full screen mode

This Lab 2 explains how to get started with MNIST Handwritten Digit application demo on i.MX8 board using eIQ ML Software Development Environment.

Get the source code available on code aurora:


MNIST Handwritten Digits

The MNIST is a large database of handwritten digits commonly used for training various image processing systems. This section provides a comparison of Caffe and TensorFlow models for Handwritten Digit Recognition. The data set used for these applications is from Yann Lecun. This is an MNIST data set sample:


Figure 7. MNIST data set


Setting Up the Board

Step 1 - Create the following folder and grant it permission as it follows:

root@imx8mmevk:~# mkdir -p /opt/mnist
root@imx8mmevk:~# chmod 777 /opt/mnist


Step 2 - To easily deploy the demos to the board, get the board's IP address using ifconfig command, then set the IMX_INET_ADDR environment variable as follows:

$ export IMX_INET_ADDR=<imx_ip>


Setting Up the Host

Step 1 - Obtain the eIQ toolchain (3.2.9.Generating the Toolchain) from NXP eIQ(TM) Machine Learning Enablement.


Step 2 - Install the toolchain:

$ chmod +x <toolchain>.sh
$ ./<toolchain>.sh

This provides all the needed setup for building ARM64 applications on a x86 machine.


Step 3 - Download the application from eIQ Sample Apps.


Step 4 - Get the models and dataset. The following command-lines create the needed folder structure for the demos and retrieves the mnist dataset, and the Caffe and TensorFlow models:

$ mkdir -p bin data model
$ wget -qN https://github.com/ARM-software/ML-examples/raw/master/armnn-mnist/data/t10k-images-idx3-ubyte -P data/
$ wget -qN https://github.com/ARM-software/ML-examples/raw/master/armnn-mnist/data/t10k-labels-idx1-ubyte -P data/
$ wget -qN https://github.com/ARM-software/ML-examples/raw/master/armnn-mnist/model/lenet_iter_9000.caffemodel -P model/
$ wget -qN https://github.com/ARM-software/ML-examples/raw/master/armnn-mnist/model/simple_mnist_tf.pb -P model/
$ wget -qN https://github.com/ARM-software/ML-examples/raw/master/armnn-mnist/model/simple_mnist_tf.prototxt -P model/
$ wget -qN https://github.com/ARM-software/Tool-Solutions/raw/master/ml-tool-examples/mnist-draw/model/optimized_mnist_tf.pb -P model/


Step 5 - Compile the source code using the eIQ toolchain:

$ source /opt/fsl-imx-xwayland/<version>/environment-setup-aarch64-poky-linux
$ ${CXX} -Wall -Wextra -O3 -std=c++14 caffe_inference.cpp -o caffe_inference -larmnn -larmnnCaffeParser $ ${CXX} -Wall -Wextra -O3 -std=c++14 tensorflow_inference.cpp -o tensorflow_inference -larmnn -larmnnTfParser


Step 6 - Deploy the built files to the board:

$ scp -r caffe_inference tensorflow_inference data/ model/ root@${IMX_INET_ADDR}:/opt/mnist


Inference Comparison Applications

Step 1 - At user space, enter the mnist folder which holds the demo files:


This is how the mnist folder structure should look like:

├── caffe_inference
├── tensorflow_inference
├── data
│├── t10k-images-idx3-ubyte
│└── t10k-labels-idx1-ubyte
├── model
│├── lenet_iter_9000.caffemodel
│├── optimized_mnist_tf.pb
│├── simple_mnist_tf.pb
│└── simple_mnist_tf.prototxt


Step 2 - Run the applications:

NOTE: For running these applications, please provide the wanted number of predictions, which can vary from 0 to 9999 since the dataset has 10K images.


1 - Handwritten Digit Recognition using Caffe

root@imx8mmevk:/opt/mnist# ./caffe_inference 10
[0] Caffe >> Actual: 7 Predict: 7 Time: 0.0336484s
[1] Caffe >> Actual: 2 Predict: 2 Time: 0.028399s
[2] Caffe >> Actual: 1 Predict: 1 Time: 0.0283713s
[3] Caffe >> Actual: 0 Predict: 0 Time: 0.0284133s
[4] Caffe >> Actual: 4 Predict: 4 Time: 0.0280637s
[5] Caffe >> Actual: 1 Predict: 1 Time: 0.0281574s
[6] Caffe >> Actual: 4 Predict: 4 Time: 0.0285136s
[7] Caffe >> Actual: 9 Predict: 9 Time: 0.0283779s
[8] Caffe >> Actual: 5 Predict: 5 Time: 0.0283902s
[9] Caffe >> Actual: 9 Predict: 9 Time: 0.0283282s
Total Time: 0.296081s
Sucessfull: 10 Failed: 0


2 - Handwritten Digit Recognition using TensorFlow

root@imx8mmevk:/opt/mnist# ./tensorflow_inference 10
[0] Tensor >> Actual: 7 Predict: 7 Time: 0.00670075s
[1] Tensor >> Actual: 2 Predict: 2 Time: 0.00377025s
[2] Tensor >> Actual: 1 Predict: 1 Time: 0.0036785s
[3] Tensor >> Actual: 0 Predict: 0 Time: 0.0036815s
[4] Tensor >> Actual: 4 Predict: 4 Time: 0.00372875s
[5] Tensor >> Actual: 1 Predict: 1 Time: 0.003669s
[6] Tensor >> Actual: 4 Predict: 4 Time: 0.00367825s
[7] Tensor >> Actual: 9 Predict: 9 Time: 0.0036955s
[8] Tensor >> Actual: 5 Predict: 6 Time: 0.00367488s FAILED
[9] Tensor >> Actual: 9 Predict: 9 Time: 0.0036025s
Total Time: 0.0414569s Sucessfull: 10 Failed: 1

NOTE: The argument 10 refers to the number of predictions for each test.


These tests run the inference on the input MNIST dataset images (Actual), showing the inference results (Predict) and how long it took to complete the prediction. The input images for this test are in the binary form and can be found at the t10k-images-idx3-ubyte.gz package from Yann Lecun.

By the output results, it is possible to notice that the Caffe model is slower than TensorFlow, however, it is also more accurate than the latter. Change the argument to compare further results between the two models.


Go to the eIQ Sample Apps - Object Recognition using OpenCV DNN.