The device uses two gloves and a foot pedal to control a speech-generating synthesizer. The right glove forms vowels, the left glove forms “stops” (such as the sound for the letter “b”), and the pitch of the hand affects the pitch of the sound. The foot pedal controls volume. It takes practice to learn to coordinate the gestures to make intelligible speech — about 100 hours of practice, says Fels.
The researchers have created a simplified version of the device that can be used on a tablet computer, but the complete version requires a backpack full of equipment. Several musical compositions have been performed and recorded using the device. A video discussion and demonstration of the device, including a woman singing a duet with herself, is available at Gizmodo.