Nonverbal vocal interface, meaning the use of non-speech vocal sounds such as "oooh" and "ahhh" as input to a computer, provides an interesting and useful input modality for a graphical user interface. Nonverbal vocal interface is a novel improvement over speech-based solutions because voiced sounds may be smoothly modulated, meaning they are well suited to control of continuous variables such as cursor position, while spoken commands are inherently discrete. A graphical user interface is an excellent environment for vocal input because instantaneous visual feedback is crucial to usability, enabling users to see the results of their vocalizations and learn the interface very quickly. Continuously voiced sounds may be easily and independently modulated in dimensions such as volume, pitch, and vowel. These dimensions may be used to augment a familiar input device such as the mouse, adding another degree of freedom to the interaction. For example, a mouse-based painting program may be improved by using vocal volume to control brush size while painting. Vocal input may alternatively be used without other input devices, for example to control the cursor in two dimensions. This offers an opportunity to improve access to computing for users unable to operate a mouse. In this thesis, the use of nonverbal vocal interface for graphical interaction is explored. Vocal dimensions of volume, pitch, and vowel are detected in real time using input from a simple USB microphone and used to affect parameters in several example graphical applications. Effectiveness of the interactive method is tested via measurement of user performance with these example applications.

Library of Congress Subject Headings

User interfaces (Computer systems); Human-computer interaction; Voice--Analysis--Data processing; Intonation (Phonetics)--Analysis--Data processing; Vowels--Analysis--Data processing

Publication Date


Document Type


Student Type


Degree Name

Computer Science (MS)

Department, Program, or Center

Computer Science (GCCIS)


Joe Geigel

Advisor/Committee Member

Warren Carithers

Advisor/Committee Member

Carl Reynolds


Physical copy available from RIT's Wallace Library at QA76.9.U83 M87 2006


RIT – Main Campus