Face recognition
Actually, face recognition technology is used in many scenes in our daily life, for instance, when taking pictures with the mobile phone, the camera software will automatically recognize the faces in the lens and focus, scan face for real-name verification when registering the App and scan face for pay, etc.
The basic steps of face recognition are shown in the below figure. Firstly, the camera captures image data, then through preprocessing such as noise elimination and image format conversion, the image data will be transmitted to the processor for face detection and recognition calculations. After recognizing the face successful, continue to do the follow-up operations.
Fig1 The basic steps of face recognition
i.MX RT106F MCU based solution for face recognition
The below figure is the block diagram of i.MX RT106F MCU-based solution for face recognition provided by the NXP. Comparing with the general processor (CPU) solution, it has comparative advantages in cost and power consumption. Further, the PCB size will be smaller too and the MCU usually can boot up within a few hundred milliseconds even with RTOS, versus to the boot-up speed of the processor (CPU) equipped with a Linux system that is about 10 seconds, it will give customers a better user experience.
Fig2 i.MX RT106F MCU based solution for face recognition
Of course, the i.MX RT106F MCU-based solution face recognition solution is not intended to replace the solution based on the processor (CPU). As aforementioned, face recognition technology has a lot of application cases, and it will definitely be used in more fields in the future, so the MCU-based face recognition solution provides customers and the market with another choice.
i.MX RT106F MCU
The i.MX RT106F face recognition crossover processor is an EdgeReady™ solution-specific variant of the i.MX RT1060 family of crossover processors, targeting face recognition applications. It features NXP’s advanced implementation of the Arm Cortex®-M7 core, which operates at speeds up to 600 MHz to provide high CPU performance and the best real-time response. i.MX RT106F based solutions enable system designers to easily and inexpensively add face recognition capabilities to a wide variety of smart appliances, smart homes, smart retail, and smart industrial devices. The i.MX RT106F is licensed to run the OASIS Lite library for face recognition (as the below figure shows) which include:
- Face detection
- Anti-spoofing
- Face tracking
- Face alignment
- Glass detection
- Face recognition
- Confidence measure
- Face recognition quantified results, etc
Fig3 OASIS Recognition Software Pipeline
sln_viznas_iot_elock_oobe
The sln_viznas_iot_elock_oobe project is the application on the SLN-VIZNAS-IOT (as the below figure shows, regarding the Bootstrap and Bootloader in the software flowchart, I will introduce them in the future). The following development work is based on the sln_viznas_iot_elock_oobe project, however, I need to sketch the basic workflow of it prior to starting real development work.
Fig4 SLN-VIZNAS-IOT software flowchart
sln_viznas_iot_elock_oobe's workflow flow
- In the Camera_Start() function, the task (Camera_Init_Task) completes the initialization of the RGB and IR cameras, then creates a task (Camera_Task);
- In the Display_Start() function, after the task (Display_Init_Task) completes the initialization of the display medium (USB or LCD), it immediately creates the task (Display_Task) and sends the message queue s_DisplayReqMsg.id = QMSG_DISPLAY_FRAME_REQ to the task (Camera_Task), then the pDispData will point to the s_BufferLcd[0] array for storing the image data to be displayed;
- In the Oasis_Start() function, firstly, OASISLT_init() completes the initialization of the OAISIT library, then creates a task (Oasis_Task) to send the message queues gFaceDetReqMsg.id = QMSG_FACEREC_FRAME_REQ and gFaceInfoMsg.id = QMSG_FACEREC_INFO_UPDATE to the task (Camera_Task) to make the pDetIR and pDetRGB point to the face block diagram captured by the RGB and IR cameras, and update the content pointed by infoMsgIn.
- After the camera is initialized, the RGB camera works at first. After the image data is captured, an interrupt is triggered and the callback function Camera_Callback() sends the message queue DQMsg.id = QMSG_CAMERA_DQ to the task (Camera_Task), and DQIndex++;
- CAMERA_RECEIVER_GetFullBuffer() extracts the image data captured by the RGB camera, and sends the message queue DPxpMsg.id = QMSG_PXP_DISPLAY to the task (PXP_Task) created in the APP_PXP_Start() function and EQIndex++, meanwhile switch the camera from RGB to IR. After the APP_PXPStartCamera2Display() function in the task (PXP_Task) completes processing, it sends the message queue s_DResMsg.id = QMSG_PXP_DISPLAY to the task (Camera_Task), and the task (Camera_Task) sends the message queue DresMsg.id = QMSG_DISPLAY_FRAME_RES to the task (Display_Task) after receiving the above message queue. The task (Display_Task) completes display, then it sends the message queue s_DisplayReqMsg.id = QMSG_DISPLAY_FRAME_REQ to the task (Camera_Task) to make pDispData point to the s_BufferLcd[1] array;
- After the IR camera completes capturing work, CAMERA_RECEIVER_GetFullBuffer() extracts the image data and sends the message queue DPxpMsg.id = QMSG_PXP_DISPLAY to the (PXP_Task) task created in the APP_PXP_Start() function, continue to execute EQIndex++ and switch to RGB camera again, and repeat the steps 5. Finally, send the message queue FPxpMsg.id = QMSG_PXP_FACEREC to the task (PXP_Task) and set irReady = true. After the task (PXP_Task) receives the above message queue, it calls APP_PXPStartCamera2DetBuf() and after completes the processing, sends the message queue s_FResMsg.id = QMSG_PXP_FACEREC to the task (Camera_Task);
- CAMERA_RECEIVER_GetFullBuffer() extracts the image data collected by the RGB camera, repeat step 5, when (pDetRGB && irReady) condition is met, send the message queue FPxpMsg.id = QMSG_PXP_FACEREC to the task (PXP_Task) and set irReady = false, pDetRGB = NULL, pDetIR = NULL. After the task (PXP_Task) receives the above message queue, it calls APP_PXPStartCamera2DetBuf() and after completes the processing, sends the message queue s_FResMsg.id = QMSG_PXP_FACEREC to the task (Camera_Task). At this time, the (!pDetIR && !pDetRGB) condition is met and the Queue message FResMsg.id = QMSG_FACEREC_FRAME_RES is sent to the task (Oasis_Task), run OASISLT_run_extend to perform face recognition calculation, and send the message queue gFaceDetReqMsg.id = QMSG_FACEREC_FRAME_REQ to the task (Camera_Task) to make the pDetIR and pDetRGB point to the face block diagram captured by the RGB and IR cameras again.
- keep repeat steps 6 and 7;
Fig5 sln_viznas_iot_elock_oobe's workflow flow
Smart Coffee machine
Fig 6 is the workflow of the smart coffee machine that I want to develop for, as there is no LCD board on hand,
in the below development process, I will select Win10's camera (as the below figure shows) to output the captured image, further, take advantage of the Shell command to simulate the LCD's touch feature to interact with the board.
Fig6 workflow of the smart coffee machine
Fig7 Camera
Code modification
- In the commondef.h, add a new member variable 'uint16_t coffee_taste' in Union FeatureItem to stand for the favorite coffee taste;
typedef union
{
struct
{
/*put char/unsigned char together to avoid padding*/
unsigned char magic;
char name[FEATUREDATA_NAME_MAX_LEN];
int index;
// this id identify a feature uniquely,we should use it as a handler for feature add/del/update/rename
uint16_t id;
uint16_t pad;
// Add a new component
uint16_t coffee_taste;
/*put feature in the last so, we can take it as dynamic, size limitation:
* (FEATUREDATA_FLASH_PAGE_SIZE * 2 - 1 - FEATUREDATA_NAME_MAX_LEN - 4 - 4 -2)/4*/
float feature[0];
};
unsigned char raw[FEATUREDATA_FLASH_PAGE_SIZE * 2];
} FeatureItem; // 1kB
- In featuredb.h, add two member functions into class FeatureDB:
set_taste()andget_taste(), and add the definition of the above two member functions in featuredb.cpp;
class FeatureDB
{
public:
FeatureDB();
~FeatureDB();
int add_feature(uint16_t id, const std::string name, float *feature);
int update_feature(uint16_t id, const std::string name, float *feature);
int del_feature(uint16_t id, std::string name);
int del_feature(const std::string name);
int del_feature_all();
std::vector<std::string> get_names();
int get_name(uint16_t id, std::string &name);
std::vector<uint16_t> get_ids();
int ren_name(const std::string oldname, const std::string newname);
int feature_count();
int get_free(int &index);
int database_save(int count);
int get_feature(uint16_t id, float *feature);
void set_autosave(bool auto_save);
bool get_autosave();
//Add two customize member functions
int set_taste(const std::string username, uint16_t taste_number);
int get_taste(const std::string username);
private:
bool auto_save;
int load_feature();
int erase_feature(int index);
int save_feature(int index = 0);
int reassign_feature();
int get_free_mapmagic();
int get_remain_map();
};
int FeatureDB::set_taste(const std::string username, uint16_t taste_number)
{
int index = FEATUREDATA_MAX_COUNT;
for (int i = 0; i < FEATUREDATA_MAX_COUNT; i++)
{
if (s_FeatureData.item[i].magic == FEATUREDATA_MAGIC_VALID)
{
if (!strcmp(username.c_str(), s_FeatureData.item[i].name))
{
index = i;
}
}
}
if (index != FEATUREDATA_MAX_COUNT)
{
s_FeatureData.item[index].coffee_taste = taste_number;
return 0;
}
else
{
return -1;
}
}
int FeatureDB::get_taste(const std::string username)
{
int index = FEATUREDATA_MAX_COUNT;
int taste_number;
for (int i = 0; i < FEATUREDATA_MAX_COUNT; i++)
{
if (s_FeatureData.item[i].magic == FEATUREDATA_MAGIC_VALID)
{
if (!strcmp(username.c_str(), s_FeatureData.item[i].name))
{
index = i;
}
}
}
if (index != FEATUREDATA_MAX_COUNT)
{
taste_number = s_FeatureData.item[index].coffee_taste;
return taste_number;
}
else
{
return -1;
}
}
- In database.h, add the declarations of
DB_Set_Taste()andDB_Get_Taste()functions, and in database.cpp, add the related codes of the above two functions. These two functions are equivalent to encapsulating the newly added member functions set_taste() and get_taste() of the FeatureDB class;
int DB_Del(uint16_t id, std::string name);
int DB_Del(string name);
int DB_DelAll();
int DB_Ren(const std::string oldname, const std::string newname);
int DB_GetFree(int &index);
int DB_GetNames(std::vector<std::string> *names);
int DB_Count(int *count);
int DB_Save(int count);
int DB_GetFeature(uint16_t id, float *feature);
int DB_Add(uint16_t id, float *feature);
int DB_Add(uint16_t id, std::string name, float *feature);
int DB_Update(uint16_t id, float *feature);
int DB_GetIDs(std::vector<uint16_t> &ids);
int DB_GetName(uint16_t id, std::string &names);
int DB_GenID(uint16_t *id);
int DB_SetAutoSave(bool auto_save);
// Add two customize functions
int DB_Set_Taste(const std::string username, const uint16_t taste);
int DB_Get_Taste(const std::string username);
int DB_Set_Taste(const std::string username, const uint16_t taste)
{
int ret = DB_MGMT_FAILED;
ret = DB_Lock();
if (DB_MGMT_OK == ret)
{
ret = s_DB->set_taste(username, taste);
DB_UnLock();
}
return ret;
}
int DB_Get_Taste(const std::string username)
{
int ret = DB_MGMT_FAILED;
ret = DB_Lock();
if (DB_MGMT_OK == ret)
{
ret = s_DB->get_taste(username);
DB_UnLock();
}
return ret;
}- In sln_api.h, add the declarations of the functions
VIZN_SetTaste(),VIZN_GetTaste()andVIZN_Is_Rec_User(), and add the codes of the above three functions in sln_api.cpp. The VIZN_SetTaste() and VIZN_GetTaste() functions are equivalent to the encapsulation of the DB_Set_Taste() and DB_Get_Taste() functions. Why is it so complicated? To follow the code layering mechanism of the elock_oobe project and reduce the difficulty of code implementation through code layered encapsulation.
/**
* @brief Set user's favorite coffee taste.
*
* @Param clientHandle The client handler which required this action
* @Param userName Pointer to a buffer which contains the name of the new user.
* @Param taste Coffee taste
*/
vizn_api_status_t VIZN_SetTaste(VIZN_api_client_t *clientHandle, char *UserName, cfg_Coffee_taste taste);
/**
* @brief Set user's favorite coffee taste.
*
* @Param clientHandle The client handler which required this action
* @Param userName Pointer to a buffer which contains the name of the new user.
* @Param taste Pointer to the Coffee taste
*/
vizn_api_status_t VIZN_GetTaste(VIZN_api_client_t *clientHandle, char *UserName, int *taste);
vizn_api_status_t VIZN_Is_Rec_User(VIZN_api_client_t *clientHandle, char *UserName);
~~~~~~~~~
vizn_api_status_t VIZN_SetTaste(VIZN_api_client_t *clientHandle, char *UserName, cfg_Coffee_taste taste)
{
int32_t status;
if (!IsValidUserName(UserName))
{
return kStatus_API_Layer_RenameUser_InvalidUserName;
}
status = DB_Set_Taste(std::string(UserName), (uint16_t)taste);
if (status == 0)
{
return kStatus_API_Layer_Success;
}
else if (status == -1)
{
return kStatus_API_Layer_SetTaste_Failed;
}
}
vizn_api_status_t VIZN_GetTaste(VIZN_api_client_t *clientHandle, char *UserName, int *taste)
{
int32_t status;
if (!IsValidUserName(UserName))
{
return kStatus_API_Layer_RenameUser_InvalidUserName;
}
*taste = DB_Get_Taste(std::string(UserName));
if (*taste != -1)
{
return kStatus_API_Layer_Success;
}
else
{
return kStatus_API_Layer_GetTaste_Failed;
}
}
vizn_api_status_t VIZN_Is_Rec_User(VIZN_api_client_t *clientHandle, char *UserName)
{
if (!IsValidUserName(UserName))
{
return kStatus_API_Layer_RenameUser_InvalidUserName;
}
return kStatus_API_Layer_Success;
}- In sln_api_init.cpp, declare the variable:
std::string Current_User = ""; which is used to store the name corresponding to the face after recognition, and add the processing functionCoffee_Rec()after successful face recognition in the structure variable ops2;
std::string Current_User = " ";
//Add customize function
int Coffee_Rec(VIZN_api_client_t *pClient, face_info_t face_info);
client_operations_t ops2 = {
.detect = NULL,
.recognize = Coffee_Rec,//NULL,
.enrolment = NULL,
};
//Add customize function
int Coffee_Rec(VIZN_api_client_t *pClient, face_info_t face_info)
{
Current_User = face_info.name;
return 1;
}- In sln_timers.h, increase MS_SYSTEM_LOCKED to extend the locked status time to 25 seconds;
~~~~~~~~
#define MS_SYSTEM_LOCKED 25000 //2000 // MS in which the board is in a locked state after a reg/rec.
~~~~~~~~- In sln_cli.cpp, add three Shell commands: order, set_taste, get_taste to stand for the operations of brewing coffee, setting coffee taste, and checking coffee taste;
SHELL_COMMAND_DEFINE(set_taste, (char *)"\r\n\"set_taste username <0|1|2|3|~>\": set user's favorite taste\r\n"
"0 - Cappuccino\r\n"
"1 - Black Coffee\r\n"
"2 - Coffee latte\r\n"
"3 - Flat White\r\n"
"4 - Cortado\r\n"
"5 - Mocha\r\n"
"6 - Con Panna\r\n"
"7 - Lungo\r\n"
"8 - Ristretto\r\n"
"9 - Others \r\n", FFI_CLI_SetTasteCommand, SHELL_IGNORE_PARAMETER_COUNT);
SHELL_COMMAND_DEFINE(get_taste, (char *)"\r\n\"get_taste username\": return user's favorite taste \r\n", FFI_CLI_GetTasteCommand, SHELL_IGNORE_PARAMETER_COUNT);
SHELL_COMMAND_DEFINE(order, (char *)"\r\n\"order <0|1|2|3|~>\": order a favorite taste \r\n", FFI_CLI_OrderCommand, SHELL_IGNORE_PARAMETER_COUNT);
~~~~~~
static shell_status_t FFI_CLI_SetTasteCommand(shell_handle_t shellContextHandle,
int32_t argc,
char **argv)
{
if (argc != 3)
{
SHELL_Printf(shellContextHandle, "Wrong parameters\r\n");
return kStatus_SHELL_Error;
}
return UsbShell_QueueSendFromISR(shellContextHandle, argc, argv, SHELL_EV_FFI_CLI_SET_TASTE);
}
static shell_status_t FFI_CLI_GetTasteCommand(shell_handle_t shellContextHandle,
int32_t argc,
char **argv)
{
if (argc != 2)
{
SHELL_Printf(shellContextHandle, "Wrong parameters\r\n");
return kStatus_SHELL_Error;
}
return UsbShell_QueueSendFromISR(shellContextHandle, argc, argv, SHELL_EV_FFI_CLI_GET_TASTE);
}
shell_status_t FFI_CLI_OrderCommand(shell_handle_t shellContextHandle,
int32_t argc,
char **argv)
{
if (argc > 2)
{
SHELL_Printf(shellContextHandle, "Wrong parameters\r\n");
return kStatus_SHELL_Error;
}
return UsbShell_QueueSendFromISR(shellContextHandle, argc, argv, SHELL_EV_FFI_CLI_ORDER);
}
~~~~~~
shell_status_t RegisterFFICmds(shell_handle_t shellContextHandle)
{
SHELL_RegisterCommand(shellContextHandle, SHELL_COMMAND(list));
SHELL_RegisterCommand(shellContextHandle, SHELL_COMMAND(add));
SHELL_RegisterCommand(shellContextHandle, SHELL_COMMAND(del));
SHELL_RegisterCommand(shellContextHandle, SHELL_COMMAND(rename));
SHELL_RegisterCommand(shellContextHandle, SHELL_COMMAND(verbose));
SHELL_RegisterCommand(shellContextHandle, SHELL_COMMAND(camera));
SHELL_RegisterCommand(shellContextHandle, SHELL_COMMAND(version));
SHELL_RegisterCommand(shellContextHandle, SHELL_COMMAND(save));
SHELL_RegisterCommand(shellContextHandle, SHELL_COMMAND(updateotw));
SHELL_RegisterCommand(shellContextHandle, SHELL_COMMAND(reset));
SHELL_RegisterCommand(shellContextHandle, SHELL_COMMAND(emotion));
SHELL_RegisterCommand(shellContextHandle, SHELL_COMMAND(liveness));
SHELL_RegisterCommand(shellContextHandle, SHELL_COMMAND(detection));
SHELL_RegisterCommand(shellContextHandle, SHELL_COMMAND(display));
SHELL_RegisterCommand(shellContextHandle, SHELL_COMMAND(wifi));
SHELL_RegisterCommand(shellContextHandle, SHELL_COMMAND(app_type));
SHELL_RegisterCommand(shellContextHandle, SHELL_COMMAND(low_power));
// Add three Shell commands
SHELL_RegisterCommand(shellContextHandle, SHELL_COMMAND(order));
SHELL_RegisterCommand(shellContextHandle, SHELL_COMMAND(set_taste));
SHELL_RegisterCommand(shellContextHandle, SHELL_COMMAND(get_taste));
return kStatus_SHELL_Success;
}- In sln_cli.cpp, it needs to add corresponding codes for handle order, set_taste, get_taste instructions in task UsbShell_CmdProcess_Task
else if (queueMsg.shellCommand == SHELL_EV_FFI_CLI_SET_TASTE)
{
int coffee_taste = atoi(queueMsg.argv[2]);
if (coffee_taste >= Cappuccino && coffee_taste <= Others)
{
status = VIZN_SetTaste(&VIZN_API_CLIENT(Shell),(char *)queueMsg.argv[1], (cfg_Coffee_taste)coffee_taste);
if (status == kStatus_API_Layer_Success)
{
SHELL_Printf(shellContextHandle, "User: %s like coffee taste: %s \r\n", queueMsg.argv[1], Coffee_type[coffee_taste]);
}
else
{
SHELL_Printf(shellContextHandle, "Cannot set coffee taste\r\n");
}
}
else
{
SHELL_Printf(shellContextHandle, "Unsupported coffee taste\r\n");
}
}
else if (queueMsg.shellCommand == SHELL_EV_FFI_CLI_GET_TASTE)
{
int get_taste_num = 0;
status = VIZN_GetTaste(&VIZN_API_CLIENT(Shell),(char *)queueMsg.argv[1], &get_taste_num);
if (status == kStatus_API_Layer_Success)
{
SHELL_Printf(shellContextHandle, "User: %s like coffee taste: %s \r\n", queueMsg.argv[1], Coffee_type[(cfg_Coffee_taste)(get_taste_num)]);
}
else
{
SHELL_Printf(shellContextHandle, "Cannot get coffee taste\r\n");
}
}
else if (queueMsg.shellCommand == SHELL_EV_FFI_CLI_ORDER)
{
status = VIZN_Is_Rec_User(&VIZN_API_CLIENT(Shell),(char *)Current_User.c_str());
if (status == kStatus_API_Layer_Success)
{
if (queueMsg.argc == 1)
{
int get_taste_num = 0;
status = VIZN_GetTaste(&VIZN_API_CLIENT(Shell),(char*)Current_User.c_str(), &get_taste_num);
if (status == kStatus_API_Layer_Success)
{
SHELL_Printf(shellContextHandle, "User: %s order the a cup of %s \r\n", Current_User.c_str(), Coffee_type[(cfg_Coffee_taste)(get_taste_num)]);
}
else
{
SHELL_Printf(shellContextHandle, "Sorry, please order again, Current user is %s\r\n",Current_User.c_str());
}
}
else if(queueMsg.argc == 2)
{
int coffee_taste = atoi(queueMsg.argv[1]);
if (coffee_taste >= Cappuccino && coffee_taste <= Others)
{
status = VIZN_SetTaste(&VIZN_API_CLIENT(Shell),(char*)Current_User.c_str(), (cfg_Coffee_taste)coffee_taste);
if (status == kStatus_API_Layer_Success)
{
SHELL_Printf(shellContextHandle, "User: %s order a cup of %s \r\n", Current_User.c_str(), Coffee_type[coffee_taste]);
}
else
{
SHELL_Printf(shellContextHandle, "Cannot set coffee taste, Current user is %s\r\n",Current_User.c_str());
}
}
else
{
SHELL_Printf(shellContextHandle, "Unsupported coffee taste\r\n");
}
}
}
}- Use the cafe logo of《Friends》to replace the original Welcome_home picture, use the BmpCvt tool to convert the picture into the corresponding array, and add it to welcomehome_320x122.h.
static const unsigned short Coffee_shop_320_122[] = {
0x59E6, 0x6227, 0x6247, 0x59C5, 0x59C5, 0x59A5, 0x4103, 0x6A67, 0x6A47, 0x6227, 0x6A47, 0x6A68, 0x7268, 0x6A67, 0x6A67, 0x6A47, 0x72A9, 0x6A68, 0x7268, 0x6A48, 0x5A06, 0x6A88, 0x6A68, 0x6247, 0x6A47, 0x7289, 0x7289, 0x6A47, 0x6A47, 0x6A47, 0x6227,
0x6A68, 0x6206, 0x6A47, 0x5A26, 0x6247, 0x6227, 0x6A27, 0x4924, 0x836D, 0x5207, 0x7BAC, 0x5247, 0x83ED, 0x4A47, 0x2923, 0x7B8C, 0x49E5, 0x49E5, 0x4A05, 0x28C1, 0x5226, 0x6267, 0x6A87, 0x72E9, 0x6267, 0x6AA9, 0x5A27, 0x6AA9, 0x6AA9, 0x5A47,
0x6A88, 0x5A06, 0x5A47, 0x6AA9, 0x5A47, 0x62A9, 0x5206, 0x6288, 0x6268, 0x5A47, 0x5A27, 0x5A47, 0x5A27, 0x49E6, 0x4A07, 0x4A07, 0x5A89, 0x49C6, 0x5A48, 0x5A28, 0x5A47, 0x5226, 0x49E6, 0x49C6, 0x41A6, 0x5208, 0x2082, 0x52A8, 0x6B6B, 0x39A5,
0x39A5, 0x3964, 0x49E7, 0x3104, 0x49C7, 0x3945, 0x41A6, 0x28A2, 0x2061, 0x3965, 0x28E3, 0x1881, 0x3944, 0x3103, 0x3103, 0x3903, 0x4145, 0x51A6, 0x51C6, 0x4985, 0x51E6, 0x51E6, 0x61E7, 0x6A48, 0x6A28, 0x6A28, 0x6A27, 0x61E6, 0x6207, 0x6A68,
0x59E7, 0x4185, 0x51E6, 0x51A6, 0x6228, 0x5A07, 0x6228, 0x5A08, 0x4184, 0x41A5, 0x4164, 0x3944, 0x3944, 0x736B, 0x83ED, 0x41A5, 0x83ED, 0x6288, 0x8BAB, 0x836A, 0x6287, 0x6B2A, 0x5267, 0x83CD, 0x5A68, 0x5228, 0x3986, 0x3985, 0x7B0A, 0x6A67,
0x7267, 0x832B, 0x49A5, 0x6206, 0x8AC9, 0x72A8, 0x82C9, 0x82E9, 0x8309, 0x6A46, 0x8B2B, 0x3860, 0x8329, 0x6A67, 0x7288, 0x7268, 0x61E6, 0x7267, 0x6A67, 0x59C5, 0x51A4, 0x6A46, 0x7AA8, 0x6A26, 0x7287, 0x7AA8, 0x72A8, 0x72A9, 0x51C5, 0x5A27,
0x5A27, 0x3923, 0x
~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~
0x7B8C, 0x734B, 0x6B0A, 0x83CD, 0x83ED, 0x8C0E, 0x7B8C, 0x7B6C, 0x20C2, 0x5227, 0x83ED, 0x6AE9, 0x734B, 0x62A9, 0x7B6B, 0x7B8C, 0x62E9, 0x7BAC, 0x7B6B, 0x732A, 0x940D, 0x83AC, 0x732A, 0x7309, 0x8BCC, 0x7309, 0x8BCD, 0x83AC, 0x7B6B, 0x940D,
0x3943, 0x942E, 0x7B6B, 0x734A, 0x7B8B, 0x62C8, 0x7B8B, 0x7B6A, 0x7BAB, 0x732A, 0x7B6B, 0x7B6B, 0x83CC, 0x6B09, 0x6AA9, 0x6AE9, 0x7B6B, 0x7B8B, 0x83AC, 0x734B, 0x6AC9, 0x6B0A, 0x734B, 0x734A, 0x62A8, 0x732A, 0x8C0E, 0x8BCD, 0x944F, 0x734B,
0x7B8B, 0x732A, 0x942E, 0x8BCD, 0x83AD, 0x732B, 0x6B0A, 0x6AEA, 0x62C9, 0x9C90, 0x28C2, 0x8BEE, 0x93EE, 0x8BCD, 0x4183, 0x838B, 0x7B6A, 0x6287, 0x8BCB
};Programming the new project
After saving the modified code and recompile the sln_viznas_iot_elock_oobe project (as shown in the figure below), then connect the MCU-LINK to J6 on the SLN-VIZNAS-IOT, just like Fig9 shows.
Fig8 Recompile code
Fig9 MCU-LINK
(Note: it needs to reselect the Flash driver, as the below figure shows.)
Fig10 Flash driver
After that, it's able to program the code project to the on-board Hyperflash.
Test & Summary
When the new code project boot-up, please refer to Get Started with the SLN-VIZNAS-IOT to use the serial terminal to test the newly added three Shell commands: orders, set_taste, and get_taste. Once a face is successfully recognized, the cafe logo will appear up (as shown in Fig11).
Fig11 Cafe logo
Definitely, this smart coffee machine seems like a 'toy' demo, and there is a lot of work to improve it.
Below is the list of my future work plans,
- Use the LCD panel instead of USB to display;
- Connect an external amplifier to enable voice prompt feature;
- Enable the Wifi feature to connect to the App;
- Use the GUI library to enhance UI experience;
- Add a voice recognition feature to control;
And I'll be glad to hear any comments from you.
