Original: Stephen Evanczuk@DigiKey Dejie Electronics
Face recognition has been widely recognized in verifying access rights of smartphones, but has not been widely used in other fields, although this technology is extremely efficient and easy to use. In addition to technical challenges, developers must address user concerns about the reliability and privacy of traditional face recognition methods in implementing reliable, low-cost machine learning solutions, which rely on the fraudulent cloud connection .
This article will first discuss the difficulties in secure authentication, and then introduce software and hardware solutions that NXP Semiconductors can solve these problems. Finally, we will explain how developers who have no previous experience in machine learning methods should use this solution to quickly implement offline anti-spoofing face recognition in smart products.
Challenge of secure authentication of smart products
In solving people's concerns about the growing security issues of smart products, developers find themselves having few sustainable options to reliably perform user authentication, enabling fast and secure access. Traditional approaches rely on multi-factor authentication methods, which rely on some combination of classic three authentication factors: "What you know" such as passwords; "What you have", such as a physical key or key card; and "What you are", usually biometric factors such as fingerprints or iris. In this way, a strong authentication door lock may require the user to enter a password, use a key card, and further provide fingerprints to unlock. In fact, this strict requirement is cumbersome or not practical at all for consumers who need to use smartphones or other common devices frequently and conveniently. The use of
The use of face recognition greatly simplifies the identity authentication of smartphone users, but some of the advantages that smartphones have may not be available to every device. In addition to the strong processing power of high-end smartphones, connecting is always online is the basic requirement for providing users with a series of complex services they expect.
For many products that require secure authentication, the underlying operating platform usually provides more moderate computing resources and limited connectivity. Face recognition services provided by head cloud service providers shift processing load to the cloud, but require reliable connections to ensure minimal response latency, and this requirement may be beyond the capabilities of the platform. Likewise or users are more concerned about transmitting their photos on public networks for processing and potentially storing them in the cloud, which can cause significant privacy concerns.
uses NXP Semiconductors' i.MX RT106F processor and related software. Developers can now implement offline face recognition, directly solving these concerns.
Hardware and software for anti-spoofing offline face recognition
As a member of the NXP i.MX RT1060 Crossover Microcontroller (MCU) family, the NXP i.MX RT106F series is specially designed to support the easy integration of offline face recognition into smart home devices, consumer appliances, security devices and industrial equipment. It is based on the Arm® Cortex®-M7 processor core. This industrial-grade MIMXRT106FCVL5B processor has a running frequency of 528 megahertz (MHz), while commercial-grade processors such as MIMXRT106FDVL6A and MIMXRT106FDVL6B have a running frequency of 600MHz.
In addition to supporting multiple external memory interfaces, the i.MX RT106F processor also includes 1 megabyte (Mb) on-chip random access memory (RAM), where 512 kilobyte (Kb) is configured as universal RAM and 512Kb can be configured as universal RAM or tightly coupled memory (TCM) for instructions (I-TCM) or data (D-TCM).In addition to on-chip power management, these processors offer a wide range of integrated functions for graphics, security, system control, and support for analog and digital interfaces generally required for consumer devices, industrial human-machine interfaces (HMI) and motor control (Figure 1).
Image source: NXP
Figure 1: NXP Semiconductor's i.MX RT106F processor combines a full set of functional blocks required to support face recognition for consumer, industrial and secure products.
Although similar to other i.MX RT1060 family members, the i.MX RT106F processor is bundled with NXP's Oasis Lite face recognition software runtime license. The Oasis Lite runtime environment aims to speed up the inference of this class of processors, performing face detection, recognition, and even limited emotion classification using neural network (NN) inference models running on the inference engine and MiniCV (simplified version of the open source OpenCV Computer Vision library). The inference engine is built on the NXP NN library and the Arm Cortex Microcontroller System Interface Standard NN (CMSIS-NN) library (Figure 2).
Image source: NXP
Figure 2: The NXP Oasis Lite runtime library includes an Oasis Lite kernel
and an NXP inference engine built based on NXP and Arm neural network libraries.
inference model resides on the i.MX RT106F platform, so face detection and recognition are performed locally, unlike other solutions that rely on cloud resources to run machine learning algorithms. Thanks to the offline face recognition feature, smart product designers can ensure private and secure authentication in low bandwidth or unstable Internet connections. In addition, using this combination of software and hardware, the authentication execution speed is very fast, and the processor takes less than 800 milliseconds (ms) to wake up from a low-power standby state to complete face recognition.
Oasis Lite runtime is used with the i.MX RT106F processor, simplifying the implementation of offline face recognition for smart products. Of course, the processor and runtime environment are only part of the required system solution. In addition to requiring a more complete set of system components, an effective authentication solution also requires imaging capabilities to mitigate a security threat called a presentation attack. These attacks attempt to trick face recognition authentication by using photos. For developers who want to quickly deploy face-based authentication in their products, the NXP SLN-VIZNAS-IOT development kit and related software provide a ready-to-use platform for offline anti-spoofing face recognition evaluation, prototyping and development.
Complete face recognition security system solution
Like most advanced processors, the i.MX RT106F processor only requires some extra components to provide an effective computing platform. The NXP SLN-VIZNAS-IOT kit completes the design by integrating the i.MX RT106F with other devices and is a complete hardware platform (Figure 3).
Image source: NXP
Figure 3: NXP SLN-VIZNAS-IOT suite includes a connection module that provides a powerful networked system platform required to run authentication software.
The kit's connection module board combines the NXP MIMXRT106FDVL6A i.MX RT106F processor, NXP A71CH security element and two connection options: NXP's MKW41Z512VHT4 Kinetis KW41Z Low Energy Bluetooth (BLE) System on Chip (SoC) and Murata Electronics LBEE5KL1DX-883 Wi-Fi/Bluetooth module.
In order to supplement the processor's on-chip memory, the connection module has added Winbond ElectronicsW9825G6JB 56 megabits (Mb) synchronous dynamic RAM (SDRAM), Integrated Silicon Solution. (ISSI) IS26KL256S-DABLI00 56 Mb NOR flash memory and ISSI's IS25LP256D 256 Mb 4-channel serial peripheral interface (SPI) devices.
Finally, the module adds an Torex Semiconductor XCL214B333DR step-down converter to complement the internal power management function of the i.MX RT106F processor for connecting other devices on the module board.
connection module instead is installed on the vision application board, which combines Murata Electronics IRA-S210ST01 Passive infrared (PIR) sensor, motion sensor, battery charger, audio support, light emitting diode (LED), buttons and interface connectors (Figure 4).
Image source: NXP
Figure 4: In the NXP SLN-VIZNAS-IOT suite, the connection module (left) is connected to the vision application board to provide a hardware foundation for face recognition.
In addition to this system platform, the face recognition system design obviously also requires a suitable camera sensor to capture the user's facial images. However, as mentioned earlier, concerns about presenting attacks require additional imaging capabilities.
Decomposition presentation attack
For years, researchers have been exploring different presentation attack detection (PAD) methods aimed at resolving attempts such as using hidden fingerprints or face images to deceive biometric-based authentication systems. Although the details are far beyond the scope of this article, in general, the PAD method uses in-depth analysis of the quality and characteristics of collected biometric data, as well as a "live" detection method designed to determine whether biometric data is collected from living people. Based on many different approaches, deep neural network (DNN) models play an important role not only in face recognition, but also in identifying attempts to spoof systems. Nevertheless, the imaging system for capturing the user's face can provide additional live detection support.
For the SLN-VIZNAS-IOT kit, NXP includes a camera module, which contains a pair of On Semiconductor MT9M114 image sensors. Here, one camera is equipped with a red, green, blue (RGB) filter and the other is equipped with an infrared (IR) filter. The RGB camera is connected to the vision application board through the camera interface, which can generate normal visible light images, while the images captured by the infrared camera are different for living people. Using this live detection method and its internal face recognition capabilities, the SLN-VIZNAS-IOT kit realizes offline, anti-spoofing face recognition capabilities in a package with a size of approximately 30×40 mm (mm) (Figure 5).
Image source: NXP
Figure 5: NXP SLN-VIZNAS-IOT hardware suite integrates a dual camera system for live detection (top) and vision application board (bottom) with the connection module, providing a plug-and-play offline face recognition solution with anti-spoofing capabilities.
SLN-VIZNAS-IOT kit Getting Started with using
NXP SLN-VIZNAS-IOT kit has a built-in face recognition model and can be used at any time. The developer plugs in a USB cable, touches the buttons on the kit, and uses the pre-installed "elock" app and the companion mobile app for simple manual face registration (Figure 6, left). After registration, when the suite authenticates the registered face, the mobile APP will display the "welcome home" message and the "unlocked" tag (Figure 6, right).
Image source: NXP
Figure 6: The NXP SLN-VIZNAS-IOT hardware suite is available out of the box, and use the supporting APP to register faces (left) and identify the registered faces (right).
The Oasis Lite face recognition software of this kit can process up to 3,000 RGB face models from its database, with a recognition accuracy of up to 99.6%, an infrared face maximum of 100, and an anti-spoofing accuracy of up to 96.5%. As mentioned earlier, NXP's hardware/software solution can complete face detection, image alignment, quality inspection, liveness detection and recognition in less than one second in the range of 0.2 meters to 1.0 meters. In fact, the system supports an alternate "light" inference model that can execute this same sequence in less than 0.5 seconds, but the maximum number of supported databases is small, namely 1000 RGB faces and 50 infrared faces.
Build a custom face recognition application
NXP SLN-VIZNAS-IOT suite just needs to be used as it is, allowing developers to quickly evaluate, prototypify and develop facial recognition applications. When creating a custom hardware solution, the kit is available as a complete reference design and provides a complete schematic and detailed bill of materials (BOM). In software development, programmer can use the NXP MCUXpresso integrated development environment (IDE) with FreeRTOS support and configuration tools. For face recognition applications, developers only need to use NXP's online MCUXpresso SDK Builder and use NXP's VIZNAS SDK to complete the software development environment configuration, which includes the NXP Oasis Lite machine learning vision engine (Figure 7).
Image source: NXP
Figure 7: NXP provides a comprehensive software environment that executes NXP Oasis Lite runtime library and utility middleware on the FreeRTOS operating system.
This package contains the complete source code of the operating environment, as well as the elock sample application mentioned above. NXP does not provide source code for its proprietary Oasis Lite engine or model. Instead, developers can use the Oasis Lite runtime library through the provided application programming interface (API). The API includes an intuitive set of function calls to perform supported operations. In addition, developers can use a set of C definitions and structures provided to specify various parameters, including image size, memory allocation, callback , and enable functions used by the system when starting the Oasis Lite runtime environment (see Listing 1 for details).
typedef struct {
//max input image height, width and channel, min_face: minimum face can be detected
int height;
int height;
int width;
//only valid for RGB images; for IR image, always GREY888 formatml2
OASISLTImageFormat_t img_format;
//min_face should not smaller than 40
int min_face;
/*memory pool pointer, this memory pool should only be used by OASIS LIB*/
char* mem_pool;
/*memory pool size*/
int size;
/*output parameter,indicate authenticated or not*/
int auth;
/*callback functions provided by caller*/
InfCallbacks_t cbs;
/*what functions should be enabled in OASIS LIB*/
uint8_t enable_flags;
/*only valid when OASIS_ENABLE_EMO is activated*/
OASISLTEmoMode_t emo_mode;
/*false accept rate*/
/*model class */
OASISLTModelClass_t mod_class;
} OASISLTInitPara_t;
Listing 1: Developers can modify software execution parameters by modifying the content of the structure, as shown above is the code for initialization of the Oasis Lite runtime. (Code source: NXP)
elock sample application code shows the key design patterns used for the following operations: start Oasis as a running task under FreeRTOS, initialize the environment, and enter the normal operation stage. During the run phase, the runtime environment operates on each frame of the image and executes the provided callback function related to each event defined in the environment (see Listing 2 for details).
typedef enum {
/*indicate the start of face detection, user can update frame data if it is needed.
* all parameter in callback parameter is invalid.*/
OASISLT_EVT_DET_START,
/*The end of face detection.
*if a face is found, pfaceBox(OASISLTCbPara_t) indicated the rect(left,top,right,bottom point value)
*info and landmark value of the face.
*if no face is found,pfaceBox is NULL, following event will not be triggered for current frame.
*other parameter in callback parameter is invalid */
OASISLT_Evt_DET_COMPLETE,
/*Face quality check is done before face recognition*/
OASISLT_EVT_QUALITY_CHK_START,
OASISLT_EVT_QUALITY_CHK_COMPLETE,
/*Start of face recognition*/
OASISLT_EVT_REC_START,
/*The end of face recognition.
* when face feature in current frame is gotten, GetRegisteredFaces callback will be called to get all
* faces feature registered and OASIS lib will try to search this face in registered faces, if this face
* is matched, a valid face ID will be set in callback parameter faceID and corresponding simularity(indicate
* how confidence for the match) also will be set.
* if no face match, a invalid(INVALID_FACE_ID) will be set.*/
OASISLT_EVT_REC_COMPLETE,
/*start of emotion recognition*/
/*End of emotion recognition, emoID indicates which emotion current face is.*/
OASISLT_EVT_EMO_REC_COMPLETE,
/*if user set a registration flag in a call of OASISLT_run and a face is detected, this two events will be notified
* for auto registration mode, only new face(not recognized) is added(call AddNewFace callback function)
* for manufacturing registration mode, face will be added forcely.
* for both cases, face ID of new added face will be set in callback function */
OASISLT_EVT_REG_START,
/*when registration start, for each valid frame is handled, this event will be triggered and indicate
* registration process is going forward a little.
* */
OASISLT_EVT_REG_IN_PROGRESS,
OASISLT_EVT_REG_COMPLETE,
} OASISLTEvt_t;
Listing 2: The Oasis Lite runtime recognizes a series of events and records it as an enumeration set in the Oasis Lite runtime header file. (Code source: NXP)
This sample application can provide developers with step-by-step debug messages describing the results related to each event handled by the event handle (EvtHandler).For example, after the quality inspection is completed (OASISLT_EVT_QUALITY_CHK_COMPLETE), the system will print out the debugging information describing the result. After the face recognition is completed (OASISLT_EVT_REC_COMPLETE), the system will call up the user ID and name of the recognized face from its database and print out this information (see Listing 3 for details). Original: Stephen Evanczuk@DigiKey Dejie Electronics Face recognition has been widely recognized in verifying access rights of smartphones, but has not been widely used in other fields, although this technology is extremely efficient and easy to use. In addition to technical challenges, developers must address user concerns about the reliability and privacy of traditional face recognition methods in implementing reliable, low-cost machine learning solutions, which rely on the fraudulent cloud connection . This article will first discuss the difficulties in secure authentication, and then introduce software and hardware solutions that NXP Semiconductors can solve these problems. Finally, we will explain how developers who have no previous experience in machine learning methods should use this solution to quickly implement offline anti-spoofing face recognition in smart products. In solving people's concerns about the growing security issues of smart products, developers find themselves having few sustainable options to reliably perform user authentication, enabling fast and secure access. Traditional approaches rely on multi-factor authentication methods, which rely on some combination of classic three authentication factors: "What you know" such as passwords; "What you have", such as a physical key or key card; and "What you are", usually biometric factors such as fingerprints or iris. In this way, a strong authentication door lock may require the user to enter a password, use a key card, and further provide fingerprints to unlock. In fact, this strict requirement is cumbersome or not practical at all for consumers who need to use smartphones or other common devices frequently and conveniently. The use of The use of face recognition greatly simplifies the identity authentication of smartphone users, but some of the advantages that smartphones have may not be available to every device. In addition to the strong processing power of high-end smartphones, connecting is always online is the basic requirement for providing users with a series of complex services they expect. Challenge of secure authentication of smart products
For many products that require secure authentication, the underlying operating platform usually provides more moderate computing resources and limited connectivity. Face recognition services provided by head cloud service providers shift processing load to the cloud, but require reliable connections to ensure minimal response latency, and this requirement may be beyond the capabilities of the platform. Likewise or users are more concerned about transmitting their photos on public networks for processing and potentially storing them in the cloud, which can cause significant privacy concerns.
uses NXP Semiconductors' i.MX RT106F processor and related software. Developers can now implement offline face recognition, directly solving these concerns.
Hardware and software for anti-spoofing offline face recognition
As a member of the NXP i.MX RT1060 Crossover Microcontroller (MCU) family, the NXP i.MX RT106F series is specially designed to support the easy integration of offline face recognition into smart home devices, consumer appliances, security devices and industrial equipment. It is based on the Arm® Cortex®-M7 processor core. This industrial-grade MIMXRT106FCVL5B processor has a running frequency of 528 megahertz (MHz), while commercial-grade processors such as MIMXRT106FDVL6A and MIMXRT106FDVL6B have a running frequency of 600MHz.
In addition to supporting multiple external memory interfaces, the i.MX RT106F processor also includes 1 megabyte (Mb) on-chip random access memory (RAM), where 512 kilobyte (Kb) is configured as universal RAM and 512Kb can be configured as universal RAM or tightly coupled memory (TCM) for instructions (I-TCM) or data (D-TCM).In addition to on-chip power management, these processors offer a wide range of integrated functions for graphics, security, system control, and support for analog and digital interfaces generally required for consumer devices, industrial human-machine interfaces (HMI) and motor control (Figure 1).
Image source: NXP
Figure 1: NXP Semiconductor's i.MX RT106F processor combines a full set of functional blocks required to support face recognition for consumer, industrial and secure products.
Although similar to other i.MX RT1060 family members, the i.MX RT106F processor is bundled with NXP's Oasis Lite face recognition software runtime license. The Oasis Lite runtime environment aims to speed up the inference of this class of processors, performing face detection, recognition, and even limited emotion classification using neural network (NN) inference models running on the inference engine and MiniCV (simplified version of the open source OpenCV Computer Vision library). The inference engine is built on the NXP NN library and the Arm Cortex Microcontroller System Interface Standard NN (CMSIS-NN) library (Figure 2).
Image source: NXP
Figure 2: The NXP Oasis Lite runtime library includes an Oasis Lite kernel
and an NXP inference engine built based on NXP and Arm neural network libraries.
inference model resides on the i.MX RT106F platform, so face detection and recognition are performed locally, unlike other solutions that rely on cloud resources to run machine learning algorithms. Thanks to the offline face recognition feature, smart product designers can ensure private and secure authentication in low bandwidth or unstable Internet connections. In addition, using this combination of software and hardware, the authentication execution speed is very fast, and the processor takes less than 800 milliseconds (ms) to wake up from a low-power standby state to complete face recognition.
Oasis Lite runtime is used with the i.MX RT106F processor, simplifying the implementation of offline face recognition for smart products. Of course, the processor and runtime environment are only part of the required system solution. In addition to requiring a more complete set of system components, an effective authentication solution also requires imaging capabilities to mitigate a security threat called a presentation attack. These attacks attempt to trick face recognition authentication by using photos. For developers who want to quickly deploy face-based authentication in their products, the NXP SLN-VIZNAS-IOT development kit and related software provide a ready-to-use platform for offline anti-spoofing face recognition evaluation, prototyping and development.
Complete face recognition security system solution
Like most advanced processors, the i.MX RT106F processor only requires some extra components to provide an effective computing platform. The NXP SLN-VIZNAS-IOT kit completes the design by integrating the i.MX RT106F with other devices and is a complete hardware platform (Figure 3).
Image source: NXP
Figure 3: NXP SLN-VIZNAS-IOT suite includes a connection module that provides a powerful networked system platform required to run authentication software.
The kit's connection module board combines the NXP MIMXRT106FDVL6A i.MX RT106F processor, NXP A71CH security element and two connection options: NXP's MKW41Z512VHT4 Kinetis KW41Z Low Energy Bluetooth (BLE) System on Chip (SoC) and Murata Electronics LBEE5KL1DX-883 Wi-Fi/Bluetooth module.
In order to supplement the processor's on-chip memory, the connection module has added Winbond ElectronicsW9825G6JB 56 megabits (Mb) synchronous dynamic RAM (SDRAM), Integrated Silicon Solution. (ISSI) IS26KL256S-DABLI00 56 Mb NOR flash memory and ISSI's IS25LP256D 256 Mb 4-channel serial peripheral interface (SPI) devices.
Finally, the module adds an Torex Semiconductor XCL214B333DR step-down converter to complement the internal power management function of the i.MX RT106F processor for connecting other devices on the module board.
connection module instead is installed on the vision application board, which combines Murata Electronics IRA-S210ST01 Passive infrared (PIR) sensor, motion sensor, battery charger, audio support, light emitting diode (LED), buttons and interface connectors (Figure 4).
Image source: NXP
Figure 4: In the NXP SLN-VIZNAS-IOT suite, the connection module (left) is connected to the vision application board to provide a hardware foundation for face recognition.
In addition to this system platform, the face recognition system design obviously also requires a suitable camera sensor to capture the user's facial images. However, as mentioned earlier, concerns about presenting attacks require additional imaging capabilities.
Decomposition presentation attack
For years, researchers have been exploring different presentation attack detection (PAD) methods aimed at resolving attempts such as using hidden fingerprints or face images to deceive biometric-based authentication systems. Although the details are far beyond the scope of this article, in general, the PAD method uses in-depth analysis of the quality and characteristics of collected biometric data, as well as a "live" detection method designed to determine whether biometric data is collected from living people. Based on many different approaches, deep neural network (DNN) models play an important role not only in face recognition, but also in identifying attempts to spoof systems. Nevertheless, the imaging system for capturing the user's face can provide additional live detection support.
For the SLN-VIZNAS-IOT kit, NXP includes a camera module, which contains a pair of On Semiconductor MT9M114 image sensors. Here, one camera is equipped with a red, green, blue (RGB) filter and the other is equipped with an infrared (IR) filter. The RGB camera is connected to the vision application board through the camera interface, which can generate normal visible light images, while the images captured by the infrared camera are different for living people. Using this live detection method and its internal face recognition capabilities, the SLN-VIZNAS-IOT kit realizes offline, anti-spoofing face recognition capabilities in a package with a size of approximately 30×40 mm (mm) (Figure 5).
Image source: NXP
Figure 5: NXP SLN-VIZNAS-IOT hardware suite integrates a dual camera system for live detection (top) and vision application board (bottom) with the connection module, providing a plug-and-play offline face recognition solution with anti-spoofing capabilities.
SLN-VIZNAS-IOT kit Getting Started with using
NXP SLN-VIZNAS-IOT kit has a built-in face recognition model and can be used at any time. The developer plugs in a USB cable, touches the buttons on the kit, and uses the pre-installed "elock" app and the companion mobile app for simple manual face registration (Figure 6, left). After registration, when the suite authenticates the registered face, the mobile APP will display the "welcome home" message and the "unlocked" tag (Figure 6, right).
Image source: NXP
Figure 6: The NXP SLN-VIZNAS-IOT hardware suite is available out of the box, and use the supporting APP to register faces (left) and identify the registered faces (right).
The Oasis Lite face recognition software of this kit can process up to 3,000 RGB face models from its database, with a recognition accuracy of up to 99.6%, an infrared face maximum of 100, and an anti-spoofing accuracy of up to 96.5%. As mentioned earlier, NXP's hardware/software solution can complete face detection, image alignment, quality inspection, liveness detection and recognition in less than one second in the range of 0.2 meters to 1.0 meters. In fact, the system supports an alternate "light" inference model that can execute this same sequence in less than 0.5 seconds, but the maximum number of supported databases is small, namely 1000 RGB faces and 50 infrared faces.
Build a custom face recognition application
NXP SLN-VIZNAS-IOT suite just needs to be used as it is, allowing developers to quickly evaluate, prototypify and develop facial recognition applications. When creating a custom hardware solution, the kit is available as a complete reference design and provides a complete schematic and detailed bill of materials (BOM). In software development, programmer can use the NXP MCUXpresso integrated development environment (IDE) with FreeRTOS support and configuration tools. For face recognition applications, developers only need to use NXP's online MCUXpresso SDK Builder and use NXP's VIZNAS SDK to complete the software development environment configuration, which includes the NXP Oasis Lite machine learning vision engine (Figure 7).
Image source: NXP
Figure 7: NXP provides a comprehensive software environment that executes NXP Oasis Lite runtime library and utility middleware on the FreeRTOS operating system.
This package contains the complete source code of the operating environment, as well as the elock sample application mentioned above. NXP does not provide source code for its proprietary Oasis Lite engine or model. Instead, developers can use the Oasis Lite runtime library through the provided application programming interface (API). The API includes an intuitive set of function calls to perform supported operations. In addition, developers can use a set of C definitions and structures provided to specify various parameters, including image size, memory allocation, callback , and enable functions used by the system when starting the Oasis Lite runtime environment (see Listing 1 for details).
typedef struct {
//max input image height, width and channel, min_face: minimum face can be detected
int height;
int height;
int width;
//only valid for RGB images; for IR image, always GREY888 formatml2
OASISLTImageFormat_t img_format;
//min_face should not smaller than 40
int min_face;
/*memory pool pointer, this memory pool should only be used by OASIS LIB*/
char* mem_pool;
/*memory pool size*/
int size;
/*output parameter,indicate authenticated or not*/
int auth;
/*callback functions provided by caller*/
InfCallbacks_t cbs;
/*what functions should be enabled in OASIS LIB*/
uint8_t enable_flags;
/*only valid when OASIS_ENABLE_EMO is activated*/
OASISLTEmoMode_t emo_mode;
/*false accept rate*/
/*model class */
OASISLTModelClass_t mod_class;
} OASISLTInitPara_t;
Listing 1: Developers can modify software execution parameters by modifying the content of the structure, as shown above is the code for initialization of the Oasis Lite runtime. (Code source: NXP)
elock sample application code shows the key design patterns used for the following operations: start Oasis as a running task under FreeRTOS, initialize the environment, and enter the normal operation stage. During the run phase, the runtime environment operates on each frame of the image and executes the provided callback function related to each event defined in the environment (see Listing 2 for details).
typedef enum {
/*indicate the start of face detection, user can update frame data if it is needed.
* all parameter in callback parameter is invalid.*/
OASISLT_EVT_DET_START,
/*The end of face detection.
*if a face is found, pfaceBox(OASISLTCbPara_t) indicated the rect(left,top,right,bottom point value)
*info and landmark value of the face.
*if no face is found,pfaceBox is NULL, following event will not be triggered for current frame.
*other parameter in callback parameter is invalid */
OASISLT_Evt_DET_COMPLETE,
/*Face quality check is done before face recognition*/
OASISLT_EVT_QUALITY_CHK_START,
OASISLT_EVT_QUALITY_CHK_COMPLETE,
/*Start of face recognition*/
OASISLT_EVT_REC_START,
/*The end of face recognition.
* when face feature in current frame is gotten, GetRegisteredFaces callback will be called to get all
* faces feature registered and OASIS lib will try to search this face in registered faces, if this face
* is matched, a valid face ID will be set in callback parameter faceID and corresponding simularity(indicate
* how confidence for the match) also will be set.
* if no face match, a invalid(INVALID_FACE_ID) will be set.*/
OASISLT_EVT_REC_COMPLETE,
/*start of emotion recognition*/
/*End of emotion recognition, emoID indicates which emotion current face is.*/
OASISLT_EVT_EMO_REC_COMPLETE,
/*if user set a registration flag in a call of OASISLT_run and a face is detected, this two events will be notified
* for auto registration mode, only new face(not recognized) is added(call AddNewFace callback function)
* for manufacturing registration mode, face will be added forcely.
* for both cases, face ID of new added face will be set in callback function */
OASISLT_EVT_REG_START,
/*when registration start, for each valid frame is handled, this event will be triggered and indicate
* registration process is going forward a little.
* */
OASISLT_EVT_REG_IN_PROGRESS,
OASISLT_EVT_REG_COMPLETE,
} OASISLTEvt_t;
Listing 2: The Oasis Lite runtime recognizes a series of events and records it as an enumeration set in the Oasis Lite runtime header file. (Code source: NXP)
This sample application can provide developers with step-by-step debug messages describing the results related to each event handled by the event handle (EvtHandler).For example, after the quality inspection is completed (OASISLT_EVT_QUALITY_CHK_COMPLETE), the system will print out the debugging information describing the result. After the face recognition is completed (OASISLT_EVT_REC_COMPLETE), the system will call up the user ID and name of the recognized face from its database and print out this information (see Listing 3 for details).
static void EvtHandler(ImageFrame_t *frames[], OASISLTEvt_t evt, OASISLTCbPara_t *para, void *user_data)
{
[code redacted for simplification]
case OASISLT_EVT_QUALITY_CHK_COMPLETE:
{
UsbShell_Printf("[OASIS]:quality chk res:%d\r\n", para-qualityResult);
pQMsg-msg.info.irLive = para-reserved[5];
pQMsg-msg.info.front = para-reserved[1];
pQMsg-msg.info.blur = para-reserved[3];
pQMsg-msg.info.rgbLive = para-reserved[8];
if (para-qualityResult == OASIS_QUALITY_RESULT_FACE_OK_WITHOUT_GLASSES ||
para-qualityResult == OASIS_QUALITY_RESULT_FACE_OK_WITH_GLASSES)
{
{
UsbShell_DbgPrintf(VERBOSE_MODE_L2, "[EVT]:ok!\r\n");
else if (OASIS_QUALITY_RESULT_FACE_SIDE_FACE == para-qualityResult)
{
{
UsbShell_DbgPrintf(VERBOSE_MODE_L2, "[EVT]:side face!\r\n");
else if (para-qualityResult == OASIS_QUALITY_RESULT_FACE_TOO_SMALL)
{
{
UsbShell_DbgPrintf(VERBOSE_MODE_L2, "[EVT]:Small Face!\r\n");
else if (para-qualityResult == OASIS_QUALITY_RESULT_FACE_BLUR)
{
{
UsbShell_DbgPrintf(VERBOSE_MODE_L2, "[EVT]: Blurry Face!\r\n");
else if (para-qualityResult == OASIS_QUALITY_RESULT_FAIL_LIVENESS_IR)
{
{
UsbShell_DbgPrintf(VERBOSE_MODE_L2, "[EVT]: IR Fake Face!\r\n");
}
else if (para-qualityResult == OASIS_QUALITY_RESULT_FAIL_LIVENESS_RGB)
{
UsbShell_DbgPrintf(VERBOSE_MODE_L2, "[EVT]: RGB Fake Face!\r\n");
}
}
}
break;
break;
[code redacted for simplification]
case OASISLT_EVT_REC_COMPLETE:
{
int diff;
unsigned id = para-faceID;
unsigned id = para-faceID;
OASISLTRecognizeRes_t recResult = para-recResult;
timeState-rec_comp = Time_Now();
pQMsg-msg.info.rt = timeState-rec_start - timeState-rec_comp;
face_info.rt = pQMsg-msg.info.rt;
#ifdef SHOW_FPS
/*pit timer unit is us*/
timeState-rec_fps++;
diff = abs(timeState-rec_fps_start - timeState-rec_comp);
if (diff 1000000 / PIT_TIMER_UNIT)
{
// update fps
pQMsg-msg.info.recognize_fps = timeState-rec_fps * 1000.0f / diff;
timeState-rec_fps = 0;
timeState-rec_fps_start = timeState-rec_comp;
}
}
#endif
memset(pQMsg-msg.info.name, 0x0, sizeof(pQMsg-msg.info.name));
if (recResult == OASIS_REC_RESULT_KNOWN_FACE)
{
{
std::string name;
UsbShell_DbgPrintf(VERBOSE_MODE_L2, "[OASIS]:face id:%d\r\n", id);
DB_GetName(id, name);
memcpy(pQMsg-msg.info.name, name.c_str(), name.size());
face_info.recognize = true;
face_info.name = std::string(name);
UsbShell_DbgPrintf(VERBOSE_MODE_L2, "[OASIS]:face id:%d name:%s\r\n", id, pQMsg-msg.info.name);
}
else
else
{
{
// face is not recognized, do nothing
UsbShell_DbgPrintf(VERBOSE_MODE_L2, "[OASIS]:face unrecognized\r\n");
face_info.recognize = false;
}
VIZN_RecognizeEvent(gApiHandle, face_info);
}
break;
list 3: As shown in a fragment of an example application provided in the NXP package, an event handler handle can handle events encountered in the face recognition sequence.(Code source: NXP)
In addition to supporting facial recognition processing requirements, NXP SLN-VIZNAS-IOT software also protects the operating environment. To ensure runtime security, the system is designed to verify the integrity and authenticity of each signed image loaded into the system using certificates stored in the SLN-VIZNAS-IOT suite file system. Since this verification sequence starts with a trusted bootloader stored in read-only memory (ROM), this process provides a trust chain for running the application firmware. In addition, since code signing and verification slows down development, this verification process is designed to bypass during software design and debugging. In fact, the SLN-VIZNAS-IOT kit comes pre-installed with signed images, but bypasses code signature verification by default. Developers can easily set options for full code signature verification for production.
In addition to the runtime environment and related application sample code, NXP also provides complete java source code for Android mobile APP. One of the apps called VIZNAS FaceRec Manager provides a simple interface for registering faces and managing users. Another VIZNAS Companion APP allows users to provide Wi-Fi credentials to the suite using an existing Wi-Fi or BLE connection.
conclusion
face recognition provides an effective method for authentication access to smart products, but implementing it usually requires high-performance computing locally, or providing high-bandwidth connections that are always online for fast response. This technology has also been the target of being deceived, with worrying user privacy issues.
As described in this article, NXP Semiconductors’ dedicated processors and software libraries provide an alternative to accurately perform offline face recognition in less than a second without a cloud connection, while resolving fraud attempts.
Finally, if you like this article, please share it with more friends! Remember to like it!
(Code source: NXP)In addition to supporting facial recognition processing requirements, NXP SLN-VIZNAS-IOT software also protects the operating environment. To ensure runtime security, the system is designed to verify the integrity and authenticity of each signed image loaded into the system using certificates stored in the SLN-VIZNAS-IOT suite file system. Since this verification sequence starts with a trusted bootloader stored in read-only memory (ROM), this process provides a trust chain for running the application firmware. In addition, since code signing and verification slows down development, this verification process is designed to bypass during software design and debugging. In fact, the SLN-VIZNAS-IOT kit comes pre-installed with signed images, but bypasses code signature verification by default. Developers can easily set options for full code signature verification for production.
In addition to the runtime environment and related application sample code, NXP also provides complete java source code for Android mobile APP. One of the apps called VIZNAS FaceRec Manager provides a simple interface for registering faces and managing users. Another VIZNAS Companion APP allows users to provide Wi-Fi credentials to the suite using an existing Wi-Fi or BLE connection.
conclusion
face recognition provides an effective method for authentication access to smart products, but implementing it usually requires high-performance computing locally, or providing high-bandwidth connections that are always online for fast response. This technology has also been the target of being deceived, with worrying user privacy issues.
As described in this article, NXP Semiconductors’ dedicated processors and software libraries provide an alternative to accurately perform offline face recognition in less than a second without a cloud connection, while resolving fraud attempts.
Finally, if you like this article, please share it with more friends! Remember to like it!
