This Lab 3 explains how to get started with OpenCV DNN applications demos on i.MX8 board using eIQ™ ML Software Development Environment.
Get the source code available on code aurora:
OpenCV Inference
The OpenCV offers a unitary solution for both neural network inference (DNN module) and classic machine learning algorithms (ML module). Moreover, it includes many computer vision functions, making it easier to build complex machine learning applications in a short amount of time and without having dependencies on other libraries.
The OpenCV DNN model is basically an inference engine. It does not aim to provide any model training capabilities. For training, one should use dedicated solutions, such as machine learning frameworks. The inference engine from OpenCV supports a wide set of input model formats: TensorFlow, Caffe, Torch/PyTorch.
Comparison with Arm NN
Arm NN is a library deeply focused on neural networks. It offers acceleration for Arm Neon, while Vivante GPUs are not currently supported. Arm NN does not support classical non-neural machine learning algorithms.
OpenCV is a more complex library focused on computer vision. Besides image and vision specific algorithms, it offers support for neural network machine learning, but also for traditional non-neural machine learning algorithms. OpenCV is the best choice in case your application needs a neural network inference engine, but also other computer vision functionalities.
Setting Up the Board
Step 1 - Create the following folders and grant them permissions as it follows:
root@imx8mmevk:# mkdir -p /opt/opencv/model
root@imx8mmevk:# mkdir -p /opt/opencv/media
root@imx8mmevk:# chmod 777 /opt/opencvStep 2 - To easily deploy the demos to the board, get the boards IP address using ifconfig command, then set the IMX_INET_ADDR environment variable as it follows:
$ export IMX_INET_ADDR=<imx_ip>Step 3 - In the target device, export the required variables:
root@imx8mmevk:~# export LD_LIBRARY_PATH=/usr/local/lib
root@imx8mmevk:~# export PYTHONPATH=/usr/local/lib/python3.5/site-packages/
Setting Up the Host
Step 1 - Download the application from eIQ Sample Apps.
Step 2 - Get the models and dataset. The following command-line creates the needed folder structure for the demos and retrieves all needed data and model files for the demo:
$ mkdir -p model
$ wget -qN https://github.com/diegohdorta/models/raw/master/caffe/MobileNetSSD_deploy.caffemodel -P model/
$ wget -qN https://github.com/diegohdorta/models/raw/master/caffe/MobileNetSSD_deploy.prototxt -P model/
Step 3 - Deploy the built files to the board:
$ scp -r src/* model/ media/ root@${IMX_INET_ADDR}:/opt/opencv
OpenCV DNN Applications
This application was based on:
1 - OpenCV DNN example: File-Based
The folder structure must be equal to:
├── file.py
├── camera.py
├── media
└── ...
├── model
│├── MobileNetSSD_deploy.caffemodel
│└── MobileNetSSD_deploy.prototxtThis example runs a single picture for example, but you pass as many pictures as you want and save them inside media/ folder. The application tries to recognize all the objects in the picture.
Step 1 - For copying new images to the media/ folder:
root@imx8mmevk:/opt/opencv/media# cp <path_to_image> .
Step 2 - Run the example image:
root@imx8mmevk:/opt/opencv# ./file.pyNOTE: If GPU is available, the example shows: [INFO:0] Initialize OpenCL runtime
This demo runs the inference using a Caffe model to recognize a few type of objects for all the images inside the media/ folder. It includes labels for each recognized object in the input images. The processed images are available in the media-labeled/ folder. See before and after labeling:
Step 3 - Display the labeled image with the following line:
root@imx8mmevk:/opt/opencv/media-labeled# gst-launch-1.0 filesrc location=<image> ! jpegdec ! imagefreeze ! autovideosink2 - OpenCV DNN example: MIPI Camera
This example is the same as above, except that it uses a camera input. It enables the MIPI camera and runs an inference on each captured frame, then displays it in a window interface in real time:
root@imx8mmevk:/opt/opencv# ./camera.py3 - OpenCV DNN example: MIPI Camera improved
This example differs from the above due the additional support of GStreamer applied to it. Using the Leaky Bucket algorithm idea, the GStreamer pipeline enables the camera to continue performing its own thread (bucket overflow when full), even if the frame was not processed by the inference thread (bucket water capacity).
As a result of this Leaky Bucket algorithm, this demo has smooth camera video at the expense of having some frames dropped in the inference process.
root@imx8mmevk:/opt/opencv# ./camera_improved.py
