eIQ Sample Apps - Face Recognition using TF Lite

[#1]

This Lab 4 explains how to get started with TensorFlow Lite application demo on i.MX8 board using Inference Engines for eIQ Software.

• eIQ Sample Apps - Overview
• eIQ Sample Apps - Introduction

Get the source code available on code aurora:

TensorFlow Lite MobileFaceNets MIPI/USB Camera

Face Detection Using OpenCV

This application demo uses Haar Feature-based Cascade Classifiers for real time face detection. The pre-trained Haar Feature-based Cascade Classifiers for face, named as XML, is already contained in OpenCV. The XML file for face is stored in the opencv/data/haarcascades/ folder as well as code aurora. Read Face Detection using Haar Cascades for more details.

TensorFlow Lite implementation for MobileFaceNets 

The MobileFaceNets is re-trained on a host PC with a smaller batch size and input size to get higher performance. The trained model is loaded as a source file in this demo.

Setting Up the Board

Step 1 - Download the demo from eIQ Sample Apps and put it in /opt/tflite folder. Then enter the src folder:

root@imx8mmevk:~# cd /opt/tflite/examples-tflite/face_recognition/src/

root@imx8mmevk:/opt/tflite/examples-tflite/face_recognition/src#

This folder should include these files:

.
├── face_detect_helpers.cpp
├── face_detect_helpers.h
├── face_detect_helpers_impl.h
├── face_recognition.cpp
├── face_recognition.h
├── haarcascade_frontalface_alt.xml
├── Makefile
├── mfn.h
├── profiling.h
└── ThreadPool.h

Step 2 - Compile the source code on the board:

root@imx8mmevk:/opt/tflite/examples-tflite/face_recognition/src# make

Step 3 - Run the demo:

root@imx8mmevk:/opt/tflite/examples-tflite/face_recognition/src# ./FaceRecognition -c 0 -h 0.85

NOTE: -c is used to specify the camera index. '0' means the MIPI/USB camera is mounted on /dev/video0. -h is a threshold for the prediction score.

Step 4 - Add a new person to the face data set. 

When the demo is running, it will detect one biggest face at real time. Once the face is detected, you can click keyboards on the right of GUI to input the new person's name. Then, click 'Add new person' to add the face to data set. 

In brief,

1. Detect face.

2. Input new person's name.

3. Click 'Add new person'.

NOTE: Once new faces are added, it will create a folder named 'data' in current directory. If you want to remove the new face from the data set, just delete it in 'data'.

[Rodrigue]

HI Diego,

This still refer to an old Tensorflow version, from r1.13, Tensorflow Lite has been migrated out of Tensorflow Contrib

face_recognition.cpp:52:10: fatal error: tensorflow/contrib/lite/kernels/register.h: No such file or directory
   52 | #include "tensorflow/contrib/lite/kernels/register.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
In file included from face_detect_helpers.h:23,
                 from face_detect_helpers.cpp:13:
face_detect_helpers_impl.h:30:10: fatal error: tensorflow/contrib/lite/builtin_op_data.h: No such file or directory
   30 | #include "tensorflow/contrib/lite/builtin_op_data.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

best regards,

Rod

[DevinJiao]

Hi Rod,

In new TF-Lite version, these headers can be found at tensorflow/lite.

You can replace path of these headers in face_recognition.cpp and face_detect_helpers_impl.h by the following:

face_recognition.cpp

#include "tensorflow/lite/kernels/register.h"#include "tensorflow/lite/model.h"#include "tensorflow/lite/optional_debug_tools.h"#include "tensorflow/lite/string_util.h"

 face_detect_helpers_impl.h

#include "tensorflow/lite/builtin_op_data.h"
 #include "tensorflow/lite/interpreter.h" 
#include "tensorflow/lite/kernels/register.h" 
#include "tensorflow/lite/string_util.h"
Then link the corresponding tensorflow-lite.a to build the app.

BR，
Devin

[manizillion]

Hi,

I am trying to run this face recognition app with tf lite building on the imx8mqevk board, facing this error .. what is the solution

imx8mqevk:/opt/face_recognition/src# make
g++ -o FaceRecognition -I /usr/include/tensorflow/contrib/lite/tools/make/downloads/flatbuffers/include/ face_recognition.cpp face_detect_helpers.cpp -ltensorflow-lite -lpthread -ldl -g -lopencv_objdetect -lopencv_highgui -lopencv_imgproc -lopencv_core -lopencv_videoio -lopencv_imgcodecs
In file included from face_recognition.cpp:59:
face_detect_helpers.h:18:10: fatal error: opencv2/objdetect.hpp: No such file or directory
#include "opencv2/objdetect.hpp"
^~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
In file included from face_detect_helpers.cpp:13:
face_detect_helpers.h:18:10: fatal error: opencv2/objdetect.hpp: No such file or directory
#include "opencv2/objdetect.hpp"
^~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

[DevinJiao]

Hi Manivannan,

OpenCV is required for this App. The errors are caused by no OpenCV header files in your rootfs. Can you double check if OpenCV was installed in your SDcard Image?

[manizillion]

Hi Devin,
Thanks for the reply , i have added the following in local conf , its  there in the image, what could be missing.

#Configure the OpenCV package:
IMAGE_INSTALL_append = "opencv python-opencv"

PACKAGECONFIG_remove_pn-opencv_mx8 = "python3"
PACKAGECONFIG_append_pn-opencv_mx8 = " dnn jasper openmp test neon python2 qt5 gtk "

#Add CMake for SDK’s cross-compile:

TOOLCHAIN_HOST_TASK_append += " nativesdk-cmake nativesdk-make"

IMAGE_INSTALL_append = " \
flatbuffers \
arm-compute-library \
tensorflow-lite \
armnn \
armnn-onnx \
"

PREFERRED_VERSION_opencv = "4.0.1%"
PREFERRED_VERSION_tensorflow-lite = "1.12.0%"

[DevinJiao]

Hi Manivannan, 

From your local conf, the OpenCV should be included in the image. You can use Linux command "find" to get the OpenCV header files path on your target, then add the path to the include path and compile the app again.

For example,

find /usr/ -name opencv2

Then,

export CPLUS_INCLUDE_PATH=/usr/<your opencv2 path>

make

Alternatively, you can also build all of the eiq sample apps in Yocto project, following Deploying the eIQ Sample Apps to an i.MX8 board.

[jheaffey]

Hi,

The link to "TensorFlow Lite MobileFaceNets MIPI/USB Camera" at the top of the page is not accessible now that NXP moved from Code Auorora to github. I can't find the C++ facial recognition examples in github, only the python ones.

May I please ask where are the C++ examples?