MirrorFugue is a set of interfaces for the piano to visualize the gesture of a performance. Based on the idea that the visibility of gesture contributes to learning and synchronization, MirrorFugue displays the hand and body movements of piano playing using metaphors from the physical world to connect musicians from disparate spaces and times.
We designed two modes to visualize the hand gesture of a performance, which we term "Reflection" and "Organ". Inspired by the reflective surface on a lacquered grand piano that mirrors the keyboard and player's hands, Reflection mode shows the mirrored keyboard and hands of a performance. Organ mode displays the unaltered, top-town image of the collaborator's keyboard like the aligned and offset keyboards of an organ.
MirrorFugue can be used in remote lessons to enable teachers and students to see each other's playing. It could also be used to playback recordings of the self and others. A student can use MirrorFugue for practice support by recording a portion of a piece playing a duet with the recording. By displaying the body and the hands at full scale on the surface of the instrument, MirrorFugue evokes the presence of a pianist with a sense of intimate emotional expression.
[pdf] Xiao Xiao and Hiroshi Ishii. 2010. MirrorFugue: communicating hand gesture in remote piano collaboration. In Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction (TEI '11). ACM, New York, NY, USA, 13-20.[pdf] Xiao Xiao and Hiroshi Ishii. 2011. Duet for solo piano: MirrorFugue for single user playing with recorded performances. In Proceedings of the 30th international conference extended abstracts on Human factors in computing systems(CHI EA '11). ACM, New York, NY, USA [pdf] Xiao Xiao. MirrorFugue: Communicating Presence in Musical Collaboration Across Space and Time. MAS Thesis. Massachusetts Institute of Technology, USA. 2011.