manualshive.com logo in svg

NXP Semiconductors MCU-SMHMI-UG, User Manual

Share
Download
NXP Semiconductors MCU-SMHMI-UG User Manual Download Page 17

NXP Semiconductors

MCU-SMHMI-UG

Smart HMI User Guide

Figure 16. Available coffee types

When using voice commands, a confirmation tone plays, saying "Americano",

"Cappuccino", "Espresso", or "Café Latte", and a green LED indicates that the command

is successfully recognized.
When the onscreen coffee type touch controls are used or a "Coffee Type" voice

command is recognized, the icon indicating the currently selected coffee type updates to

reflect the new selection.

MCU-SMHMI-UG

All information provided in this document is subject to legal disclaimers.

© 2022 NXP B.V. All rights reserved.

User guide

Rev. 0 — 29 September 2022

17 / 51

Summary of Contents for MCU-SMHMI-UG

Page 1: ...words MCU SMHMI UG SLN TLHMI IOT Human Machine Interface HMI Smart Home IoT Abstract This document describes the smart Human Machine Interface HMI solution and its associated out of box features The S...

Page 2: ...er MCU simplifying the overall system design NXP s cross platform dual core framework architecture empowers designers with the flexibility to customize their products making i MX RT117H a versatile fi...

Page 3: ...e firmware update over a wireless connection Factory automation scripts Supported by an MCUXpresso SDK MCUXpresso IDE and configuration tools 1 2 Voice library limitations Voice seeker AFE and Cyberon...

Page 4: ...unctions Many of the onboard sensors and peripherals can be seen in Figure 1 and Figure 2 Figure 1 SLN TLHMI IOT kit callout Figure 2 SLN TLHMI IOT main board callout CAUTION Depending on the mode of...

Page 5: ...Additionally the SDK associated with the SLN TLHMI IOT has software support for both the device driver and Hardware Abstraction Layer HAL driver for the sensor The stock firmware utilizes the camera s...

Page 6: ...boot mode Mass Storage Device MSD update mode The following sections describe each of these different boot modes in more detail 4 1 Boot modes The bootloader supports several different boot up method...

Page 7: ...eceive firmware updates without a debugging probe like a SEGGER J Link Instead MSD uses USB to emulate an MSD interface like the one that is used for USB flash drives 1 This feature can be especially...

Page 8: ...update the firmware image on your kit drag and drop the new binary to the USB drive corresponding to your kit Assuming the binary is properly generated the bootloader automatically updates the firmwar...

Page 9: ...for rapid integration of unique customer applications and requirements The Coffee Machine application presents an example of a potential smart HMI use case Here an HMI device must operate using hands...

Page 10: ...rder by anonymously associating it with their face The next time that face is recognized they can either choose a new coffee selection or automatically use their saved coffee selection The following s...

Page 11: ...or use the wake word supported by your preferred language as described in Section 5 2 6 5 2 3 Home screen You can access the home screen by speaking the wake word Hey NXP or touching the screen while...

Page 12: ...ly align their faces with the camera The box indicates the status of the face recognition pipeline Face Not Detected Face Detected and Face Recognized based on the color of the box Figure 9 Face not d...

Page 13: ...ed On the right half of the screen are the three different selections for modifying the coffee selection the coffee type size and strength MCU SMHMI UG All information provided in this document is sub...

Page 14: ...language settings as well as the language used for the hands free voice commands See Section 5 2 6 for more info regarding supported languages wake words and commands 5 2 3 1 Bounding box colors As pr...

Page 15: ...detected but not registered The bounding box turns green when a face is detected and a coffee selection is associated with it Additionally a red X for removing the recognized face is displayed in the...

Page 16: ...ent coffee preference settings including the type of coffee The coffee machine application has four default coffee types Americano Cappuccino Espresso and Caf Latte Each coffee type can be selected us...

Page 17: ...ndicates that the command is successfully recognized When the onscreen coffee type touch controls are used or a Coffee Type voice command is recognized the icon indicating the currently selected coffe...

Page 18: ...arge by using the touchscreen controls Figure 18 Selecting coffee size In addition to the touchscreen controls the coffee size can be adjusted using the Small Medium and Large voice commands depending...

Page 19: ...ngth can be adjusted using Soft Mild or Strong voice commands depending on the desired coffee strength 4 Upon successfully recognizing a voice command the slider position updates and a confirmation to...

Page 20: ...transitions to the final screen before re enabling the standby screen While the final screen is being shown an audio prompt plays Save your coffee selection Confirm or Cancel for either of the follow...

Page 21: ...nd or onscreen touch control This process permanently deletes the face and the coffee preferences associated with the face To delete a face use the Delete User or equivalent voice command in your conf...

Page 22: ...ion that can help with debugging issues recognizing your face tap the face icon next to the NXP logo icon 3 times Figure 23 Face icon Once enabled messages appear at the top of the screen to the right...

Page 23: ...oice command Start is given The brewing screen displays the preferences chosen on the home screen coffee type size and strength An animation of coffee brewing is displayed to the right of these option...

Page 24: ...r is a new unrecognized face the application forgets the user and does not associate any coffee selection with their face 5 2 6 Voice commands The coffee machine application supports over a dozen comm...

Page 25: ...t 1 of 3 selections for coffee strength Mild 1 of 3 selections for coffee strength Strong 1 of 3 selections for coffee strength Start Command to begin brewing the coffee Confirm 1 of 2 responses used...

Page 26: ...type Cappuccino 1 of 4 selections for coffee type Espresso 1 of 4 selections for coffee type Kaffee Latte 1 of 4 selections for coffee type Klein 1 of 3 selections for coffee size shortest brewing an...

Page 27: ...le only while green bounding box and red X in top left corner are present Table 6 Voice commands in Chinese 5 2 7 Shell commands The coffee machine out of box FW provides additional configuration opti...

Page 28: ...e Prints the currently enabled preview mode camera or virtual_face language Prints the currently configured system language wifi ssid SSID Sets the SSID wifi password Password Sets the Password wifi i...

Page 29: ...n tool by associating your face with the floor where you reside 5 3 2 Home screen After transitioning past the startup screen the home screen is displayed Figure 27 Home screen The home screen is spli...

Page 30: ...n alarm and a status message which indicates the current floor number Figure 29 Elevator panel Finally the last section in the bottom left contains two icons showing that NXP voice and vision are acti...

Page 31: ...lt state which means no face is detected yet Figure 31 No user detected The icon turns red when a face is detected but is not registered This state indicates that the following floor can be associated...

Page 32: ...floor The application has six default floors that can be chosen with either voice or touch simulating a real elevator which can travel to different floors Touch controls simply require pressing the f...

Page 33: ...ine If the microphone icon is grey use the Hey NXP wake word or wake word associated with your preferred language to enable voice command recognition A confirmation tone and a green light confirm when...

Page 34: ...color Figure 36 Screen displaying You Have Arrived The kit then reenables the standby screen waiting once again for user interaction to reawaken the kit 5 3 2 3 Saving floor selection Users can associ...

Page 35: ...Confirm or Cancel a new floor can be selected Or after a 15 seconds timeout with no activity detected the demo resets the floor number back to a default state and once again allow recognizing saved f...

Page 36: ...sks the user which floor they would like to go to instead 5 3 2 5 Deleting a saved face If the application recognizes a face the user may delete it using either a voice command or onscreen touch contr...

Page 37: ...ce recognition debugging information To activate debugging information that can help with debugging issues recognizing your face tap the face icon next to the NXP logo icon 3 times Figure 40 Enabling...

Page 38: ...e help screen is a useful reference tool to retrieve a list of the available voice commands To enable the help screen simply press the Help Screen button in the lower left hand corner of the home scre...

Page 39: ...which language to use For example waking the board using Hey NXP configures the system language to English while Salut NXP configures the system language to French On the home screen the system langu...

Page 40: ...rrent floor to the main lobby floor one Premier Etage Sets elevator current floor to floor one Deuxi me Etage Sets elevator current floor to floor two Troisi me Etage Sets elevator current floor to fl...

Page 41: ...oor two Sets elevator current floor to floor three Sets elevator current floor to floor four Sets elevator current floor to floor five Sets elevator current floor to floor six 1 of 2 responses used fo...

Page 42: ...stem language serial_number Prints the device serial number 6 Troubleshooting 6 1 Unable to save a face This section describes the steps that can be taken to help debug potential points of failure whe...

Page 43: ...side the bounding box Stare straight at the camera Move slowly from left to right and up to down to ensure the proper face angle as shown in Figure 44 Be under proper lighting conditions as shown in F...

Page 44: ...screen debugging information to help diagnose issues recognizing a face Enabling the face recognition debugging information differs slightly between different applications In both the coffee machine a...

Page 45: ...I User Guide Figure 46 Face recognition debugging information in coffee machine MCU SMHMI UG All information provided in this document is subject to legal disclaimers 2022 NXP B V All rights reserved...

Page 46: ...Message Meaning Solution OK A face is detected N A Small Face is too far away Come closer to camera Light Face is too bright Adjust environment brightness Dark Face is too dark Adjust environment bri...

Page 47: ...speak clearly There are also occasional problems with recognizing commands given in a strong accent To account for these issues see the Smart HMI Software Development User Guide for good practices in...

Page 48: ...done to this document since the initial release Revision number Date Substantive Changes 0 29 September 2022 Initial release Table 14 Revision history MCU SMHMI UG All information provided in this doc...

Page 49: ...nd conditions of commercial sale NXP Semiconductors products are sold subject to the general terms and conditions of commercial sale as published at http www nxp com profile terms unless otherwise agr...

Page 50: ...nd logos are registered trademarks owned by Bluetooth SIG Inc and any use of such marks by NXP Semiconductors is under license MCU SMHMI UG All information provided in this document is subject to lega...

Page 51: ...25 5 2 6 2 French 25 5 2 6 3 German 26 5 2 6 4 Chinese 27 5 2 7 Shell commands 27 5 3 Elevator 28 5 3 1 Overview 29 5 3 2 Home screen 29 5 3 2 1 Recognition status icon 31 5 3 2 2 Selecting a floor 32...

Reviews:

No comments

Brands by name

Popular brands

Load more brands