Physical Sciences and Mathematics Commons^™

An Implementation And Usability Study Of A Natural User Interface Virtual Piano, Zackery Frazier

Williams Honors College, Honors Research Projects

We present an implementation of a Natural User Interface (NUI) virtual piano keyboard. Using the Unity game engine and a Leap Motion hand controller, users are able to use their hands to interact with and play music in a virtual scene. Various approaches were attempted in refining the user experience of the virtual piano, and the successes and shortcomings of each implementation method are outlined and elaborated upon. Additionally, the most successful method, which was achieved by using Unity’s physics engine to detect rigid body collisions between virtual fingers and keys, was used to complete a usability study involving both …