Media Lab Europe
Human Connectedness research group


a responsive virtual mirror for interpersonal communication

Cian Cullinan, Stefan Agamanolis

Reflexion is an interpersonal video communication system that operates like a "magic mirror" in which you see a reflection of yourself together with the reflections of other participants in remote locations. The system responds to visual and auditory cues to appropriately compose the scene and emphasize the center of attention.

A Reflexion station consists of a camera and video display connected to a computer. Each participant, of which there can be several, uses a separate Reflexion station. Using an in-house image segmentation algorithm, the computer extracts an image of the participant from his background and transmits a mirror image of it over the network to the other stations. The computer also receives extracted participant images from the other stations and combines them all together into a single video scene. The effect is one of a "digital mirror" in which the participant sees a reflection of himself as well as the reflections of the other remotely-located participants.

The system automatically monitors auditory cues and uses them to compose the scene in a way that enhances the interaction. For example, the current prototype tracks which participants are speaking in order to judge who is the "center of attention". Active participants are rendered opaque and in the foreground to emphasize their visual presence, while other less-active participants appear slightly faded in the background in a manner that maintains awareness of their state without drawing undue attention. The system smoothly transitions the layering and appearance of the participants as their interactions continue. Every participant sees exactly the same composition, enhancing the sense of inhabiting a "shared space".

The system uses a peer-to-peer networking strategy for audio and video transmission to achieve low latency. A central server handles control messages that synchronize the screen compositions at each station.

Participants may navigate and interact with media objects that appear in the background, such as documents, images, movies, or live television feeds. We are exploring new forms of object and gesture recognition to make this interaction as natural and seamless as possible in different application scenarios.

Traditional multi-party video conferencing systems often display participants in separate windows, in scenes that often look like the title sequence from the TV show "The Brady Bunch". The visual separation characteristic of these designs introduces a confrontational dynamic before a meeting or interaction even begins. The fact that participants in Reflexion are layered together and can "touch" and interact with each other directly in the virtual video scene creates a space with a unique and intimate social dynamic, one that is more appropriate for many kinds of applications.

Future experiments will include scaling the system to work fluidly with very large numbers of participants, developing new techniques for more natural interaction, and applying the base system in other scenarios, such as a distance learning or multi-user remote interactive theatre.

Publications and Links

  • Stefan Agamanolis, At the intersection of broadband and broadcasting: How ITV technologies can support Human Connectedness, Proceedings of the 4th EuroITV Conference, Athens, 25 - 26 May 2006, pp. 17-22. (PDF)

  • Cian Cullinan and Stefan Agamanolis, Reflexion: a responsive virtual mirror for interpersonal communication, Conference Supplement, ECSCW 2003 8th European Conference on Computer Supported Cooperative Work, Helsinki, 14 - 18 September 2003. (PDF)

  • Cian Cullinan and Stefan Agamanolis, Reflexion: a responsive virtual mirror, Conference Companion, UIST 2002 Symposium on User Interface Software and Technology, Paris, 27 - 30 October 2002. (PDF)

  • Reflexion demo movie (Quicktime, 40 MB).

    The Reflexion interface concept is inspired by an earlier prototype named Reflection of Presence created several years prior at the MIT Media Lab. Below are some links to the old project:

  • Old Reflection of Presence web page.

  • Stefan Agamanolis, Alex Westner, and V. Michael Bove, Jr., Reflection of Presence: Toward More Natural and Responsive Telecollaboration, Proceedings of SPIE Multimedia Networks, vol. 3228A, Dallas, 4 November 1997.