The VUI project allows for novel human-computer interaction by using a real-time video stream for gestural user input. The visual field is divided into sensing regions that may or may not be registered with objects or interface representations in the physical environment allowing for scalable arbitrary spatial UIs. One goal is to leverage existing widely-accessible low-cost technologies, such as webcams, to facilitate deployment of application software to end users. Applications include practical context-specific user interfaces, gaming, interactive art installations, and performance art.
Virtual Musical Instruments are one type of implementation of an application of Virtual User Interface that allows for the creation of music through gestural input, mapping musical features to spatial locations and hand gestures. A user can create music in real-time by gesturing in space in front of a VUI camera or by using a registered pattern, such as a printable paper piano keyboard.