Can you control your HoloLens app with voice commands? Cortana listens to global commands but what are your options to execute your code by speaking? In this tutorial you will learn how to use the Mixed Reality Toolkit to recognize phrases and how to respond to them.
Control HoloLens with voice commands using Unity3D and the mixed reality toolkit (MRTK).
From the toolkit, drag UITextPrefab to the Hierarchy
Set the Render Mode to Screen Space - Overlay
Expand UITextPrefab and select Text
Set Text to Say 'create' or 'new'
Set Color to Red
Create an empty Object GameObject
From the toolkit, drag SpeechInputSource and SpeechInputHandler to GameObject
Add keywords ‘new’ and ‘create’ to the Speech Input Source keywords collection.
In Speech Input Handler check Is Global Listener
Drag the toolkit InputManager to the Hierarchy
Create a script to respond to voice commands
The Speech Input Handler is able to trigger functions but in this example we are going to use the ISpeechHandler interface.
Add a new script to the GameObject and call it SpeechController
Open in Visual Studio.
Here is the code for the script:
public class SpeechController : MonoBehaviour, ISpeechHandler
public Text Text;
public void OnSpeechKeywordRecognized(SpeechEventData eventData)
Text.text = eventData.RecognizedText;
Save and go back to Unity
Drag Text to the property in the Speech Controller script:
Click Play and if your microphone is on, the app should already respond to the two commands:
Before deploying to HoloLens
If you want to deploy this to the HoloLens, you have to enable the microphone capability.
Open the Player Settings
Go to the Publish Settings
Check Microphone capability
Now you can build and deploy to the HoloLens
Written by Loek van den Ouweland on 2017-10-20.
Questions regarding this artice? You can send them to the address below.