manualshive.com logo in svg

NXP Semiconductors SLN-VIZNAS-IOT, User Manual

Share
Download
NXP Semiconductors SLN-VIZNAS-IOT User Manual Download Page 10

10 

SLN-VIZNAS-IOT User Guide, Rev. 1.1, 10/2020 

NXP Semiconductors

 

 

 

2.2.1  Access the Camera 

NOTE: Camera enumeration is currently supported on Windows and Ubuntu. 

With the kit powered on and connected to your computer, the SLN-VIZNAS-IOT kit will 
automatically enumerate as both a serial device and USB camera device. To access the 
kit’s camera, open 

Camera

 if using Windows, or 

Cheese

, if using Ubuntu. In this guide, 

we’ll be using Windows and the Windows camera app. 

 

Figure 5: Windows Camera App 

Summary of Contents for SLN-VIZNAS-IOT

Page 1: ...BUTTONS 22 3 1 1 SW1 Toggle GUI 23 3 1 2 SW2 Manual Deregistration 24 3 1 3 SW3 Toggle IR RGB Output 26 3 1 4 SW4 Manual Registration 27 3 2 SERIAL COMMANDS 29 3 2 1 List Available Commands 29 3 2 2 List All Registered Users 31 3 2 3 Manually Add Users 31 3 2 4 Manually Delete Users 32 3 2 5 Rename Users 32 3 2 6 Verbose Mode 32 3 2 7 Configure IR LED Brightness 35 3 2 8 Configure White LED Bright...

Page 2: ...ess vs UserID 53 3 6 LOW POWER MODE 54 3 7 AUDIO FEEDBACK 55 3 8 DISPLAY CONFIGURATIONS 56 4 TROUBLESHOOTING 58 4 1 REGISTERING A FACE 58 4 1 1 Turn off Liveness Detection 58 4 1 2 Change App Type 58 4 1 3 Debug Using Verbose Mode 58 4 1 4 Ensure Lighting is Sufficient 59 4 1 5 Adjust Face Proximity and Position 60 5 DOCUMENT DETAILS 62 5 1 REFERENCES 62 5 2 ACRONYMS ABBREVIATIONS DEFINITIONS 62 5...

Page 3: ...GURE 34 SUCCESSFUL REGISTRATION 28 FIGURE 35 REGISTRATION FAILED 28 FIGURE 36 WELCOME HOME MESSAGE 29 FIGURE 37 LIST COMMAND 31 FIGURE 38 ADD COOPER COMMAND 31 FIGURE 39 ADD S COMMAND 32 FIGURE 40 DEL COOPER COMMAND 32 FIGURE 41 DEL A COMMAND 32 FIGURE 42 RENAME USER_0 COOPER COMMAND 32 FIGURE 43 VERBOSE 0 32 FIGURE 44 EXAMPLE VERBOSE MODE DEBUG MESSAGE 33 FIGURE 45 ON SCREEN INFO FOR REGISTERING ...

Page 4: ...OMMAND 44 FIGURE 83 APP_TYPE COMMAND 45 FIGURE 84 LOW_POWER ON COMMAND 45 FIGURE 85 FACEREC MANAGER HOME SCREEN 46 FIGURE 86 SMART LOCKS MANAGEMENT PAGE 46 FIGURE 87 BLE CONNECTIONS IN RANGE 47 FIGURE 88 CONFIGURE SMARTLOCK INFORMATION 47 FIGURE 89 MANAGE SMART LOCKS TAB 48 FIGURE 90 CONNECTED LOCKS 48 FIGURE 92 REMOTE REGISTRATION SCREEN 49 FIGURE 91 REMOTE FACE CAPTURE SCREEN 49 FIGURE 93 MANAGE...

Page 5: ...ION TABLE 33 TABLE 8 VERBOSE MODE ABBREVIATION TABLE 33 TABLE 9 EMOTION RECOGNITION CONFIGURATION TABLE 37 TABLE 10 VGA VS QVGA 39 TABLE 11 APP TYPE ARGUMENTS 44 TABLE 12 APP CONFIGURATIONS 52 TABLE 13 REFERENCE DOCUMENTS 62 TABLE 14 ABBREVIATIONS AND DEFINITIONS 63 TABLE 15 REVISION HISTORY 63 ...

Page 6: ... stoves and dishwashers Home comfort devices Thermostats remote temperature sensors and lighting Counter top appliances Microwaves coffee machines rice cookers and blenders Smart industrial devices Power tools ergonomic stations machine access and authorization 1 1 RT106F VISION CROSSOVER PROCESSOR OVERVIEW The i MX RT106F is an EdgeReady member of the i MX RT1060 family of crossover processors ta...

Page 7: ...Figure 1 SLN VIZNAS IOT Additional Peripherals Additionally the SLN VIZNAS IOT kit comes with an IR RGB Dual Camera Adapter like that shown below for use in secure applications Figure 2 Dual Camera Adapter Kit Configuration Depending on the mode of operation devices using the Dual Camera Adapter can emit highly concentrated non visible infrared light which can be hazardous to the human eye Product...

Page 8: ...ency bands in which the equipment operates b The maximum RF power transmitted PN RF Technology a Freq Range b Max Transmitted Power SLN VIZNAS IOT Wi Fi 2412MHz 2472MHz 17 9dBm Table 2 Wi Fi Frequency Power EUROPEAN DECLARATION OF CONFORMITY Simplified DoC per Article 10 9 of the Radio Equipment Directive 2014 53 EU This apparatus namely SLN VIZNAS IOT conforms to the Radio Equipment Directive 201...

Page 9: ...able Figure 3 SLN VIZNAS IOT Box Contents Please check your kit for damage or marks and if seen please contact your NXP representative 2 2 Running the Demo To get started take the USB C cable provided inside the kit and plug the USB A end into your computer and the USB C end into your kit Figure 4 Plugging in SLN VIZNAS IOT Kit Once connected a green LED D1 will light up to indicate the kit is pow...

Page 10: ...dows and Ubuntu With the kit powered on and connected to your computer the SLN VIZNAS IOT kit will automatically enumerate as both a serial device and USB camera device To access the kit s camera open Camera if using Windows or Cheese if using Ubuntu In this guide we ll be using Windows and the Windows camera app Figure 5 Windows Camera App ...

Page 11: ...p NOTE The Windows Camera app has a face finder feature of its own shown by the smaller blue box around the face in the above screenshot If you have multiple cameras connected to your computer you may need to change which camera is being shown In the Windows Camera app this can be done by using the Change Camera button located in the top right hand corner of the app Figure 7 Change Camera Button i...

Page 12: ... will appear at the top of the screen and the bounding box around the user s face will turn from blue to green To register your face stare straight on at the camera while aligning your face inside the bounding box While registration is taking place a Registering message will be shown Figure 9 Registration in Progress NOTE If pressing the button does not produce a Registering message ensure that th...

Page 13: ...celed or timing out a message like the following will be displayed Figure 11 Registration Failed To retry simply press the manual registration button again NOTE If you are having trouble successfully registering a face try adjusting proximity to the camera either closer or further away slowly moving your head left right and up down so that the camera can get a better view of different angles of yo...

Page 14: ...licitly saved into flash using the save command if they should be retained see Save Users Through Resets 2 2 3 Liveness Detection and Anti Spoofing The SLN VIZNAS IOT comes with Liveness Detection and Anti Spoofing turned ON by default meaning that the system can discern between your actual face and a print out phone display picture of your face Figure 13 Phone Display Spoof Attack By requiring a ...

Page 15: ...poof Attack As shown in the screenshots above using neither a phone display nor a printed picture of a face triggers the Welcome Home message In fact if we enable verbose mode see 2 2 5 Enable Verbose Mode while attempting to recognize the printed picture in Figure 14 we can see that the inference engine detects a face but recognizes that the image captured by the IR camera is a fake spoofed face ...

Page 16: ...m To connect to the serial CLI of the kit identify the COM port associated with the kit and connect using the 115200 8 1 N XON XOFF serial settings shown below Figure 16 Serial Connection Settings After connecting to the kit s serial interface you will encounter a blank terminal screen that echoes any characters that you type Use the help command to display a list of all the available serial comma...

Page 17: ...ould be between 0 inactive to 100 version The version information save n Save database into flash memory updateotw Reboot the board and start the OTW firmware update reset Reset the MCU emotion 0 2 4 7 liveness on off detection resolution qvga vga Set detection resolution and reset display output_mode rgb ir display output_device usb riverdi display interface loopback infobar wifi on off Turn the ...

Page 18: ...serial debug output type the serial command verbose 0 verbose 1 verbose 2 or verbose 3 Figure 18 verbose 0 Using the verbose command will enable logging of the debug information indicated in the table below Message Type Importance verbose 0 verbose 1 verbose 2 verbose 3 Critical High X X X Detailed Medium X X Misc Low X On screen Debug Info N A X Table 3 Verbose Mode Configuration Table The follow...

Page 19: ...ion recognition issues An example can be seen in the images below NOTE Without any registered users the on screen debug info will not update until a registration has been triggered This is because the inference engine does not attempt to recognize any faces if they are none currently registered Figure 20 On screen Info for Registering Face In the above example registration is taking place but acco...

Page 20: ...g performed 2 2 6 Optional Enable Low Power Mode For many use cases managing power consumption is critical Fortunately the SLN VIZNAS IOT supports low power features out of the box Low power mode functionality can be enabled or disabled through the use of the serial commands Use the command low_power on or low_power off to enable or disable the low power mode feature supported on the SLN VIZNAS IO...

Page 21: ...IZNAS IOT wakes up your computer s camera app Windows Camera app Cheese etc may need to be restarted before you can begin seeing camera output once again This may also apply to your serial terminal emulator of choice as well PuTTY Tera Term etc Because only select peripherals are enabled during sleep mode the SLN VIZNAS IOT will only wake up in response to a specific trigger s which by default is ...

Page 22: ...W1 Toggle GUI Toggles the GUI on or off SW2 Manual Deregister Triggers the deletion of the next registered face encountered by the kit SW3 Toggle IR RGB Output Toggles whether output from the RGB or IR camera is shown Does not affect which cameras are in use SW4 Manual Register Triggers registration of the next face encountered by the kit Table 5 Push Buttons Functionality 3 1 Push Buttons The SLN...

Page 23: ...igure 24 SW1 The SW1 push button toggles the GUI ON or OFF When the GUI is deactivated only video from the kit s camera will be shown and no other on screen information Figure 25 GUI Toggled Off NOTE The behavior of this button is identical to that of the display interface serial command ...

Page 24: ...ase When this button is pressed an indicator message will be displayed at the top of the screen which lets the user know that the process of deregistering a face is taking place Figure 27 Deregistration in Progress To deregister a face simply stare straight on at the camera while aligning the face inside the bounding box for 1 3 seconds and wait for a message confirming the removal of the user Onc...

Page 25: ...isplayed Simply press the SW2 push button again to retry Figure 29 Remove Failed To cancel a deregistration simply press the deregistration button again or wait for the deregistration process to timeout 5s NOTE The behavior of this button is identical to specifying the name of the user whose face is being captured by the camera as the argument to the del USERNAME command ...

Page 26: ...e see App Configurations both cameras are always active however only the output from one camera can be displayed at a time The SW3 push button can be used to toggle between the output coming from the IR and RGB cameras as shown in the following figure Figure 31 SW3 The behavior of SW3 is identical to that of the display output_mode rgb ir command ...

Page 27: ... appear at the top of the screen and the bounding box around the user s face will turn from blue to green To register your face stare straight on at the camera while aligning your face inside the bounding box While registration is taking place a Registering message will be shown Figure 33 Registration in Progress NOTE If pressing the button does not produce a Registering message ensure that the ba...

Page 28: ...number of registered users will increase by one A username will be assigned to the registered face usernames can be specified through the CLI interface discussed in the following section Figure 34 Successful Registration Should your face fail to register properly either due to the registration being canceled or timing out a message like the following will be displayed Figure 35 Registration Failed...

Page 29: ...structions you are still experiencing trouble ensure that the face being registered is sufficiently well lit for both the IR and RGB cameras by adjusting the pwm values for the IR and white LEDs For more information check out the Troubleshooting section near the end of this guide 3 2 Serial Commands In order to access the full suite of features that the SLN VIZNAS IOT has to offer communication vi...

Page 30: ...s to use the specified intensity camera white_pwm 0 100 Configures the camera adapter s white LEDs to use the specified intensity version Displays version information regarding inference engine save Saves face database in flash memory updateotw Reboots the board and sets up OTW firmware update mode reset Reset the MCU emotion 0 2 4 7 Configures emotion recognition to use the specified mode 0 2 4 o...

Page 31: ...ht 3 Door access heavy Determines the inference engine model to use For descriptions of these models see low_power on off Enables or disables low power mode Table 6 Serial Command Table 3 2 2 List All Registered Users Using the list command will list all the users currently registered into the system SHELL Registered users count 2 Cooper Chris Figure 37 list Command 3 2 3 Manually Add Users While ...

Page 32: ...HELL All users successfully deleted Figure 41 del a Command 3 2 5 Rename Users The rename command can be used to rename a user who is currently registered into the database To do so simply use the command rename oldusername newusername Below is an example of the command being used SHELL rename user_0 Cooper SHELL User user_0 successfully renamed Figure 42 rename user_0 Cooper Command 3 2 6 Verbose...

Page 33: ... face detection and recognition event Figure 44 Example Verbose Mode Debug Message The following table describes the different messages and the associated descriptions Abbreviation Definition dt Time taken to detect face rt Time taken to recognize face sim Predictive accuracy confidence value of face rec face_id Internal face database identifier Table 8 Verbose Mode Abbreviation Table In addition ...

Page 34: ...ront 1 and the RGB liveness check has passed hence rgbLive 1 the IR liveness check has not passed hence irLive 0 This would suggest that there may be an issue with the IR camera likely a lighting issue Figure 46 On screen Info for Recognized Face In the above example the on screen info reflects information about the face being recognized As shown in the screenshot the face has been recognized with...

Page 35: ...orted Figure 48 Camera IR PWM not supported Message 3 2 8 Configure White LED Brightness The dual camera adapter has two white LEDs which can be configured to control their brightness as appropriate for the environment they are being used in To control the brightness these white LEDs use PWM signals which have configurable duty cycles that can be set using serial commands To control the brightness...

Page 36: ... save into flash database Database save success Time taken in ms 0 Figure 52 save Command The save command returns a success message along with the time taken in milliseconds to save all the faces into flash 3 2 11 Enable OTW Over the Wire Update Mode OTW updates and enabling OTW Update Mode are discussed in the SLN VIZNAS IOT Developer s Guide 3 2 12 Reset the SLN VIZNAS IOT Situations can arise ...

Page 37: ...d Success Output Running the emotion command will enable the recognition of the emotions shown in the table below Emotion emotion 2 emotion 4 emotion 7 Neutral X X X Happy X X X Angry X X Surprised X X Sad X Fear X Disgust X Accuracy 92 81 62 Table 9 Emotion Recognition Configuration Table When a user s emotion is recognized the kit will display a message which will indicate the detected emotion I...

Page 38: ...onfigure Liveness Detection Kits equipped with the dual camera adapter come with the ability to perform liveness anti spoofing detection as an added level of security for those using the kit This feature comes enabled by default Using the command liveness off on can be used to toggle this feature off or on For the command to take effect the board will automatically reset itself so make sure you ha...

Page 39: ... just the resolution of the image passed to the inference engine Mode Resolution Detection Recognition Range Detection Speed VGA default 640x480 3m 1 4m 100ms QVGA 320x240 3m 7m 80ms Table 10 VGA vs QVGA A visual example of the difference in max range between the two resolutions can be seen in the following figure Figure 58 VGA vs QVGA Detection Resolution Max Range In order to switch between the ...

Page 40: ...displayed can be configured via the display rgb ir command which will configure the display to output from whichever camera is specified The functionality of this command is identical to that of the SW3 push button SHELL display output_mode ir SHELL display ir frame Figure 61 display output_mode ir Command Figure 62 IR Display Output SHELL display output_mode rgb SHELL display rgb frame Figure 63 ...

Page 41: ...play SHELL display output_device riverdi SHELL Display output mode set to riverdi system will reset Figure 65 display output_device riverdi Command For more information on supported display peripherals see Display Configurations 3 2 18 Toggle GUI The display interface loopback infobar command can be used to toggle the GUI on or off leaving only the face bounding box The behavior of this command is...

Page 42: ...Fi driver is connecting wait Figure 69 Wi Fi driver is connecting Message If the Wi Fi is already turned on and connected you will receive the following message SHELL wifi on SHELL Wi Fi driver is on Figure 70 Wi Fi driver is on Message Finally if the Wi Fi was already on and had attempted to connect but failed because the credentials given were bad etc you will receive the following message SHELL...

Page 43: ... If no credentials are currently in use the output of the command will look like the following SHELL wifi credentials SHELL Wi Fi Credentials not found Figure 76 Wi Fi Credentials not found Message NOTE While useful during development exposing sensitive password information is not recommended and it is advised that this command be disabled 3 2 22 Configure Wi Fi Credentials The wifi credentials SS...

Page 44: ... 0 0 SHELL wifi ip IPv4 Address 0 0 0 0 Figure 80 wifi ip Command Unsuccessful 3 2 24 Erase Wi Fi Credentials In order to completely erase the existing Wi Fi credentials use the wifi erase command SHELL wifi erase SHELL Wi Fi credentials removed Figure 81 wifi erase Command NOTE Once issued the Wi Fi credentials will be wiped from flash and cannot be retrieved 3 2 25 Configure Application Type The...

Page 45: ...either CLI commands or the on board push buttons as described in earlier sections but it also supports remote registration via BLE using an Android phone tablet application The companion remote registration Android app also known as the FaceRec Manager application sports a relatively simple interface where a user is able to register faces remotely and manage the users stored in the SLN VIZNAS IOT ...

Page 46: ...ble to add kits they own by simply pressing a button to scan for remote devices that are within Bluetooth range of the phone tablet being used Figure 85 Smart Locks Management Page Once scanning has begun any devices in the area will automatically be detected and show up in the app as a potential connection Simply press Connect next to the device you want to add in order to register the VIZNAS kit...

Page 47: ...ection Figure 87 Configure SmartLock Information Finally your registered locks will show up under the Manage Smart Locks tab inside the Android app and the kit your phone tablet is currently connected to will display a BLE icon indicating a connection is currently established with that SLN VIZNAS IOT kit Additionally users can connect disconnect from a device using the BLE icon in the upper right ...

Page 48: ... to at any given time 3 3 3 Manage Users The FaceRec Manager app as the name would imply allows users to manage the users stored on an SLN VIZNAS IOT kit using an Android application This includes the ability to add list and delete users from the SLN VIZNAS IOT s local face database by providing 2 way synchronization This section will give an overview of each of these features ...

Page 49: ...ab Figure 90 Remote Registration Screen To register your face click on the icon of silhouette in upper middle part of the screen Once pressed a screen like the following will be shown and you must enable face capture by using the camera icon at the bottom of the screen Figure 91 Remote Face Capture Screen The face capture screen may provide instructional cues to help you properly align your face e...

Page 50: ... the Manage Users tab in the FaceRec Manager Android application Figure 92 Manage Users Tab From here pulling down from the middle of the page will perform a sync between the device and the application Alternatively individual users registered remotely but not yet synced with the kit can be synced by clicking the sync icon next to a user s name Figure 93 User List Screen ...

Page 51: ...FaceRec Manager App the SLN VIZNAS IOT also comes with a reference application for remote credential provisioning over Wi Fi and BLE call the VIZN Companion App This application allows users to remotely provide Wi Fi credentials using an existing Wi Fi connection or via BLE Similar to the FaceRec Manager App the APK full source code for this Android application can be downloaded from the official ...

Page 52: ...antages 3 5 1 Light vs Heavy As shown in Table 12 App Configurations E Lock Light is roughly 500ms faster at recognizing a face when compared to the E Lock Heavy app type However the heavy configuration is capable of recognizing roughly double the number of faces as the light app type A similar tradeoff between recognition speed and max of faces can be seen in the Door Access Light vs Door Access ...

Page 53: ... where an IR camera has a much easier time Because of this fact the IR camera has less reliance on power hungry illuminator LEDs that an RGB camera would require for the recognition process As a result generally the E Lock app type is recommended in battery powered use cases where continually running power hungry LEDs could prove to be an issue draining a significant portion of the battery life Th...

Page 54: ...llowing diagram Figure 95 Low Power Timeout Flow As shown above when the kit boots and low power mode is enabled two timers are kicked off which are used to determine when the kit should fall asleep One timer is used only when a face is detected but not recognized and the other timer is used when no face is detected at all By default the No_Face_Timer which is used when no face is detected will ca...

Page 55: ...be explicitly saved into flash upon reset when low power mode is disabled NOTE Faces are not automatically saved to flash when low power mode is disabled Once asleep the board can only be awoken by the triggers mentioned in Figure 95 Low Power Timeout Flow the PIR sensor SW3 and USB Power Power Cycling the kit Triggering a wakeup will cause the board to return to normal operation NOTE The PIR sens...

Page 56: ...splay via SPI Currently the only officially supported display is the Riverdi RVT28UEFNWC10 2 8 Capacitive Touch Display More information about this display can be found here Figure 98 Riverdi RVT28UEFNWC10 NOTE The SLN VIZNAS IOT features a SPI interface to connect a Serial display from Riverdi which is supported by our Software Package However the MIMXRT106F also embeds a parallel display control...

Page 57: ...de Rev 1 1 10 2020 NXP Semiconductors To configure the display in use simply specify either usb or riverdi as the argument to the display output_device command see 3 2 17 Configure Display Output Device Requires Riverdi Display ...

Page 58: ...etection off still does not allow your face to be able to be registered follow the steps in Change App Type to help determine why detection recognition is failing 4 1 2 Change App Type As discussed in App Configurations the E Lock app type makes use of the IR camera for detection recognition while the Door Access app type makes use of the RGB camera for detection recognition To help determine if t...

Page 59: ...ion Failed Verbose Mode Output 4 1 4 Ensure Lighting is Sufficient One of the most common issues when registering a face is insufficient lighting for the IR and or RGB camera Shown below are a few examples of insufficient lighting for the RGB and IR cameras Figure 100 Bad RGB Lighting ...

Page 60: ...ightness 3 2 6and Configure White LED Brightness until a lighting setting which works for your environment is found 4 1 5 Adjust Face Proximity and Position Because face recognition makes use of face identifiers to determine the face being looked at it is important that these identifiers can be seen by both cameras Oftentimes an improper face angle and or the proximity to the camera can cause regi...

Page 61: ...ng to register a face the face should ideally take up a majority of the bounding box be centered inside the bounding box and staring straight on at the camera To help ensure the proper face angle is used it can be helpful to slowly move the face from left to right and up to down ...

Page 62: ...r guide SLN VIZNAS IOT DG pdf SLN VIZNAS IOT Developer Guide https www nxp com mcu vision2 SLN VIZNAS IOT Home Page Table 13 Reference Documents 5 2 Acronyms Abbreviations Definitions Acronym Meaning Definition FTDI Future Technology Devices International GUI Graphic User Interface IOT Internet of Things IVT Instruction Vector Table JTAG Joint Test Action Group MANF Manufacturer MCU Microcontrolle...

Page 63: ...opment Kit UART Universal asynchronous receiver transmitter Table 14 Abbreviations and Definitions 5 3 Revision History Date Version Details of Change Author Reviewers 10 28 20 Production 1 1 Various Corrections Cooper Carnahan NXP 9 29 20 Production 1 0 Initial Version Cooper Carnahan NXP Table 15 Revision History ...

Page 64: ... data sheets and or specifications can and do vary in different applications and actual performance may vary over time All operating parameters including typicals must be validated for each customer application by customer s technical experts NXP does not convey any license under its patent rights nor the rights of others NXP sells products pursuant to standard terms and conditions of sale which c...

Reviews:

No comments

Brands by name

Popular brands

Load more brands