Media Lab
Europe
Human Connectedness
research group
Reflexion
a responsive virtual mirror for interpersonal
communication
Cian Cullinan, Stefan Agamanolis
Reflexion is an interpersonal video communication system that operates
like a "magic mirror" in which you see a reflection of yourself
together with the reflections of other participants in remote
locations. The system responds to visual and auditory cues to
appropriately compose the scene and emphasize the center of attention.
A Reflexion station consists of a camera and video display connected
to a computer. Each participant, of which there can be several, uses
a separate Reflexion station. Using an in-house image segmentation
algorithm, the computer extracts an image of the participant from his
background and transmits a mirror image of it over the network to the
other stations. The computer also receives extracted participant
images from the other stations and combines them all together into a
single video scene. The effect is one of a "digital mirror" in which
the participant sees a reflection of himself as well as the
reflections of the other remotely-located participants.
The system automatically monitors auditory cues and uses them to
compose the scene in a way that enhances the interaction. For
example, the current prototype tracks which participants are speaking
in order to judge who is the "center of attention". Active
participants are rendered opaque and in the foreground to emphasize
their visual presence, while other less-active participants appear
slightly faded in the background in a manner that maintains awareness
of their state without drawing undue attention. The system smoothly
transitions the layering and appearance of the participants as their
interactions continue. Every participant sees exactly the same
composition, enhancing the sense of inhabiting a "shared space".
The system uses a peer-to-peer networking strategy for audio and video
transmission to achieve low latency. A central server handles control
messages that synchronize the screen compositions at each station.
Participants may navigate and interact with media objects that appear
in the background, such as documents, images, movies, or live
television feeds. We are exploring new forms of object and gesture
recognition to make this interaction as natural and seamless as
possible in different application scenarios.
Traditional multi-party video conferencing systems often display
participants in separate windows, in scenes that often look like the
title sequence from the TV show "The Brady Bunch". The visual
separation characteristic of these designs introduces a
confrontational dynamic before a meeting or interaction even begins.
The fact that participants in Reflexion are layered together and can
"touch" and interact with each other directly in the virtual video
scene creates a space with a unique and intimate social dynamic, one
that is more appropriate for many kinds of applications.
Future experiments will include scaling the system to work fluidly
with very large numbers of participants, developing new techniques for
more natural interaction, and applying the base system in other
scenarios, such as a distance learning or multi-user remote
interactive theatre.
Publications and Links
Stefan Agamanolis, At the intersection of broadband and
broadcasting: How ITV technologies can support Human
Connectedness, Proceedings of the 4th EuroITV Conference,
Athens, 25 - 26 May 2006, pp. 17-22. (PDF)
Cian Cullinan and Stefan Agamanolis, Reflexion: a responsive
virtual mirror for interpersonal communication, Conference
Supplement, ECSCW 2003 8th European Conference on Computer
Supported Cooperative Work, Helsinki, 14 - 18 September 2003. (PDF)
Cian Cullinan and Stefan Agamanolis, Reflexion: a responsive
virtual mirror, Conference Companion, UIST 2002 Symposium on
User Interface Software and Technology, Paris, 27 - 30 October
2002. (PDF)
Reflexion demo movie
(Quicktime, 40 MB).
The Reflexion interface concept is inspired by an earlier prototype
named Reflection of Presence created several years prior at the MIT
Media Lab. Below are some links to the old project:
Old Reflection of
Presence web page.
Stefan Agamanolis, Alex Westner, and V. Michael Bove, Jr.,
Reflection of Presence: Toward More Natural and Responsive
Telecollaboration, Proceedings of SPIE Multimedia Networks,
vol. 3228A, Dallas, 4 November 1997.
|