Researchers at the University of Washington have designed "Vocal Joystick," an alternative to a handheld mouse based on the human voice.
The interface software can detect sounds 100 times per second and instantaneously turn the sound into movement on the screen. For example, different vowel sounds such as "ah," "ee," "aw," and "oo" move the cursor one of eight different directions. The sounds "k" and "ch" simulate clicking the mouse, while louder sounds make the cursor move faster.
Doctoral student Brandi House uses Vocal Joystick to control the movement of a robotic arm.
Doctoral student Jonathan Malkin, who helped develop the tool, said that it takes about two minutes to train the software to recognize an individual's voice. Then Vocal Joystick can be used for Web browsing, drawing, and playing video games. Early tests with the device have shown that a user has an equivalent amount of control as if using a mouse or other handheld device.
The group, headed by UW associate professor of electrical engineering Jeffrey Blimes, explains that the human voice is a very flexible instrument, with precise control and ability to make different sounds besides words and sentences. The researchers said they hoped to incorporate more advanced controls that use more aspects of the human voice, such as repeated vocalizations, vibrato, degree of nasality and trills.
Blimes said they also investigated whether people would feel self-conscious using the Vocal Joystick, and found that it soon feels so natural that people nearly forget that they're using it. The device could potentially prove very useful for people with spinal cord injuries who don't have use of their hands.
The group has also applied the Vocal Joystick technology to control a robotic arm, with the pitch of the voice used to move the arm up and down. Blimes said that this is the first time that vocal commands have been used to control a three-dimensional object in this manner.
A demonstration of Vocal Joystick for Web surfing can be found here .