Nitin Sawhney, Sean Wheeler and Chris Schmandt

Speech Interface Group

MIT Media Lab

20 Ames St., Cambridge, MA

People wish to maintain a level of awareness of timely information and activity of others in a variety of social settings at different times of day. Access to situationally relevant information in the shared environment of the participants provides a means for better exchange, coordination and negotiated order in a community. One approach to enhance community awareness combines sensing of audio/visual context with filtering and display of relevant information in a shared workspace. The challenge is to develop means for lightweight interaction and communication using these 'shared information appliances'.

In this paper, we describe the design of an experimental project that explores such issues within a community environment at the MIT Media Lab. We are investigating peripheral interfaces that can be used in a casual manner in transitional spaces. Such methods can be designed in the context of casual workplace domains, distributed workgroups, infant/senior caretakers, and everyday public spaces.

To understand the role of shared community appliances and interaction in a public context, we pose a number of questions that may suggest different design approaches. Many such issues are encountered in everyday community appliances which purposely draw attention in public spaces, such as information kiosks in train stations, or an electronic whiteboard in a meeting room. Such appliances tend to be situated in the heart of an area, surrounded by peripheral and transitional spaces, which are typically highly frequented but under-utilized. We are interested in exploring means for enabling brief encounters with contextually relevant information in such transitional spaces.

Why is information about context necessary within a community? One view is that information permits co-ordination and negotiation between community members, clarifying and democratizing their decisions. Another view is that it provides assurances about the social order and one's role in the community. It allows one to stay in the loop. Hence transparent public access to situationally relevant information about the community is a desirable goal. The Portholes project [Dourish92] provided distributed awareness via periodic images of others, helping maintain a sense of community. Piazza [Issacs96] supported spontaneous encounters based on shared tasks. We believe shared information in a community space may trigger richer interactions.

Whose needs in the community should the system address? A community is held together partially by spatial proximity, established social orders, and a set shared interests and goals. The question of who belongs in a community could be broadly defined to include its current physical inhabitants, as well as on-line individuals and visitors who may engage in it briefly. Hence shared displays must allow others a level of access, while preserving community standards for privacy and anonymity if desired.

When should a system in transitional space draw attention to itself? An information system that provides continuous alerts or requires active user engagement is distracting. Designing graceful shared systems that coexist with the environment, requires a means to detect when an appropriate interruption is meaningful, based on the context of its participants. [Sawhney 99]

How does interface and modality affect transitional use of space? Many systems tend to utilize high-bandwidth interaction or modalities in a manner that asserts their presence in the environment. We take the converse view that a system, which inhibits the original use of its environment, may have undesirable effects. Hence the presence of the system itself may draw undue attention or distract from the primary social activity of the environment. Lightweight interaction and ambient modalities allow greater cohesion with the environment. Use of human intuitions about social distance [Goffman63], as well as our kinesthetic senses, is underutilized in current systems. We propose using movement and proximity as a peripheral interface.

Let's now consider how these questions could be addressed in the design of a shared information appliance for a casual workplace environment such as the 'Garden'.

The MIT Media Lab is an active research environment and a casual social space inhabited by a diverse group of students and faculty, and frequented by many visitors during the day. The lab has several open spaces for workgroups, and thrives on the interaction among and between such groups to maintain a fluid and social environment. We chose to utilize one such workspace and 'social collective', called the 'Garden' as the primary environment for our exploratory project. The Garden inhabitants are involved in responsive media and interaction research, hence are somewhat more receptive to using prototype HCI systems. The Garden workspace (see figure 1) was recently renovated to further open-up the space and install digital media infrastructure, including computer-controlled lighting, sound, video projections and some cameras (in focused areas). The space consists of clusters of workstations, a large HDTV display, a glass-walled conference room, offices, and an expanded hallway. The Garden hallway faces the entire space and is visible by most people there. This presents a good opportunity to utilize this transitional area for dynamic information display.

An existing shared appliance called the Garden-Box, a digital audio jukebox developed at the lab, has been in use here for some time. The system plays MP3 music in the environment, selected via a web-based interface. Shared use of such an appliance requires implicit negotiation among participants regarding the type and volume of music played at different times of day. Continuous use and the kind of music played also serves to define the changing interests of the community and a subtle awareness of others in the space.

We attempted to leverage the infrastructure and negotiated social order within the Garden community to propose an on-going design of shared appliances for display of situated information and awareness patterns.

We set out to explore a number of key themes and issues for designing a shared media appliance that utilizes short-term awareness of visual activity and long-term community interests, Some of these themes were further developed in an early prototype.

It is important to note that the phased presentation is not directly mapped to actual movement, but it gradually transitions between timed display modes to provide a fluid interaction. In browse mode the size of the article is reduced to allow easier reading at a closer distance. The visual activity is detected via image differencing and thresholding techniques in ISIS using real-time data from networked cameras (see figure 3). The timing and duration of information glances and previews have to be carefully devised and synchronized with movement to provide fluid presentation and interaction without being overtly distracting or prolonging beyond a user's interest. As the system has only recently been up and running, we have not conducted formal evaluations. However we hope to share preliminary observations of the system's usage at the workshop.

As people carry mobile devices like PDAs in their environment, they may be able to implicitly exchange information between the aware portals in these spaces. This may allow them to post personal messages easily or store updated information (news or changes in schedules) they recently glanced on the display. In this manner, humans act as carriers of contextually relevant information between spaces and communities.

By providing more personal information to a shared portal, one may get more relevant information from the system and allow known others to get better awareness, however there is also a greater potential intrusion of ones privacy [Hudson96]. We need to consider negotiated protocols and interfaces that perceive and retain the level of privacy one demands in different situations. Abstract representations of visual activity (such as shadow-views [Hudson96]) and our own experiments with garbled audio techniques point to several design approaches.

Most communication devices assume binary and absolute modes for interaction. Telephones can disrupt recipients and require a time-limited conversation between parties rather than an on-going awareness. Video and audio-conferencing permit open connections between participants but typically require complete engagement or attention. Communication technology can now be used to create more graceful means for peripheral awareness and background conversation. The awareness portal may act as a link between distributed spaces, and the proximity of individuals in both spaces may trigger gradual awareness and communication.

One must consider the granularity at which audio and visual events must be segmented and classified for meaningful abstraction of activity information. There are several techniques for establishing context from audio/visual scenes [Clarkson98] as well as extracting high-level activities from motion templates [Bobick97]. There should be an emphasis on modeling and representation of activity derived from both audio and visual features of everyday environments.

We are currently exploring the use of aware information appliances in a casual workplace scenario. However in public environments such as hospitals, airports and train stations, there will be varying modes of interaction, privacy concerns and benefits of access to timely information or awareness of routines. Awareness provides subtle cues about people's availability, rhythms, and regularity of activity (or lack thereof) [Mynatt99]. One's expectations regarding the social and temporal structure of the environment, allows better coordination with others in a workgroup and enhances cognitive well being in a community. Awareness portals placed within elderly communities may provide cognitive support and enhanced mutual awareness within the community or with caretakers and loved ones.

Publicly available information is predominantly accessed through private information appliances. For information of broad community interest, however, it may be more appropriate to integrate the information within the very spaces we occupy [Dourish97]. Providing a platform for shared information access and awareness within a community is a valuable means to support coordination, negotiation and a sense of belonging.

The design of shared information appliances requires consideration of community-driven authoring of content, low-overhead interaction techniques and a framework that takes into account frequent but transitional usage by participants. We have demonstrated an exploratory prototype that provides community-filtered information based on peripheral interaction using visual proximity and motion.

A fluid and situationally sensitive interaction requires appropriate use of modality to the context of use, a protocol for synchronization of information 'actors' with human activity, and sensitivity to privacy norms in the group. Such systems must have an unobtrusive means for sensing user intention and activity patterns while representing timely information relevant to the community.

We wish to thank Stefan Agmanolis and Chris McEniry for supporting us with the ISIS programming environment and their prior work with the projection display.

Agamanolis, S., Westner, A., and Bove, Jr. V. M. "Reflection of Presence: Toward More Natural and Responsive Telecollaboration," Proc. SPIE Multimedia Networks, 3228A, 1997.

Bobick, A. "Movement, Activity and Action: The Role of Knowledge in the Perception of Motion." Phil. Trans. R. Soc. Lond. B (1997) 352, 1257-1265.

Cassell, J. and Vilhjalmsson, H. "Fully Embodied Conversational Avatars: Making Communicative Behaviors Autonomous." Autonomous Agents and Multi-Agent Systems, 2(1), June 1999.

Clarkson, B., Sawhney, N., Pentland, A. "Auditory Context Awareness via Wearable Computing". 1998 Workshop on Perceptual User Interfaces (PUI'98). Nov 4-6, 1998. pp. 37-43.

Dourish, P., and Bly, S. "Portholes: Supporting Awareness in a Distributed Work Group." Proceedings of CHI '92.

Dourish, Paul. "Extending Awareness Beyond Synchronous Collaboration." Position Paper for CHI '97 Workshop on Awareness in Collaborative Systems, 1997.

Goffman, E. 1963. Behavior in Public Places: Notes on the Social Organization of Gatherings. The Free Press.

Hudson, S. E. and Smith, I. "Techniques for Addressing Fundamental Privacy and Disruption: Tradeoffs in Awareness Support Systems." Proceedings of CSCW '96. pp. 238-247.

Issacs, Ellen, Tang, J.C., Morris, T. "Piazza: A Desktop Environment Supporting Impromptu and Planned Interactions." Proceedings of CSCW '96. pp. 315-324.

Mynatt, E.D., Adler, A., Ito, M., Linde, C., O'Day, V.L. "The Network Communities of SeniorNet." Proceedings of ECSCW '99.

Sawhney, N., and Schmandt, C. 1999. Nomadic Radio: Scaleable and Contextual Notification for Wearable Audio Messaging. Proceedings of CHI’99, 96-103.