Awareness and Communication

Nomadic Radio will provide listeners with an ability to have an awareness of their colleagues and communicate with them using the wearable audio platform. Spatial auditory awareness cues will indicate when people login, move to other rooms or logout, detected via the Position Server that keeps track of a selected user community. The listener can ask "who's there" and the system will speak the names of people currently available or use auditory cues to indicate their presence and varying levels of privacy. The listener could also ask to track one or more users, such that their activity is continuously conveyed to them for a certain period of time. Currently this functionality is being prototyped on a map-based client application called RadioSpace, being developed by Natalia Narmasse (a graduate student in the Speech Group). RadioSpace is a test-bed for audio interaction techniques and client-server protocols via the Position Server developed by John Holmes (Speech Group UROP). A limited subset of the awareness functionality and protocols will be integrated in Nomadic Radio with a speech and audio-only interface.

The nomadic user should also be able to let others listen into her conversation to know if she can be interrupted, via a garbled audio recording. The GarblePhone, an application currently being developed by Mike Jacknis (Speech Group UROP) will be incorporated within the Nomadic Radio architecture. A client application for audio recording, garbling and network communication will need to be developed for the PC platform. The user could choose to send an asynchronous voice message to a colleague or initiate voice conversation with them, depending on their current level of interruption. This message could be broadcast to all users, and others should be able to asynchronously reply at their own pace. This form of communication is not unlike textual MUDs (multi-user domains) or Zephyr (a real-time text-based messaging service at MIT), with the added spontaneity and intonational properties of voice.

We are investigating the use of UDP (User Datagram Protocol - a connection-less network transport protocol) for synchronous audio communication using public-domain or commercially available Internet telephony APIs. Speak Freely is a public-domain application that support real-time voice data over the network between Windows and UNIX platforms using GSM compression and PGP encryption. We are evaluating audio quality and performance of Speak Freely for use in Nomadic Radio. In summary, we plan to utilize the following modes of awareness/communication:

• Abstract Awareness: spatial auditory cues indicating activity of users tracked by the Position Server.
• Garbled Awareness: playback of garbled audio via GablePhone using UDP network protocol.
• Asynchronous Voice Messaging: audio recording and playback on client with server messaging.
• Synchronous Voice Communication: use of Internet telephony API for reliable half-duplex audio.

Interaction Protocols for Communication Transitions

Currently Nomadic Radio provides only incoming-asynchronous messaging. We plan to integrate incoming and outgoing-synchronous communication. Several interface issues must be considered for notification and to permit users to seamlessly transition between different communication modes:

1. What form of notification will be provided for asynchronous messaging vs. synchronous communication? For example how would a notification for a voice mail differ from a request by a caller to initiate a voice conversation with the user? How will this notification indicate the urgency level of a message or communication from a caller?
2. How will a user request garbled audio awareness of another user's auditory channel? How will the user be notified when another party wishes to hear their garbled audio stream? How should privacy levels be set for specific users and groups?
3. In what way should users be notified (or made continuously aware) that they have an open audio channel / garbled audio filter with another caller or audio multicast to several remote sites?
4. Some users may wish to filter their messaging/communication such that they can hear the party leave a voice message, while having the ability to intercept the caller to initiate a voice conversation. How should such a transaction be handled such that the caller is notified that the user has suddenly become available? How would the user place an existing party on hold or easily interleave between them?
5. What interface protocol must be used to notify/interrupt a user who is currently conversing with a caller, when a 2^nd caller wishes to initiate communication with the user? In such a scenario, the user could be notified via audio cue or synthesized speech regarding the caller and the 2^nd caller could receive some speech/audio notification that the user is busy. Alternatively the caller could also receive a synchronous/asynchronous garbled audio of the conversation if allowed by the user.

This indicates a somewhat complex set of transaction protocols and interface issues that must be handled in an unobtrusive manner without confusing either the user or the caller while maintaining an desirable level of privacy and notification. Related studies in audio only communication environments may offer some insights [Hindus96][Watts96]. These issues are particularly challenging since notifications and complex transactions between users must be provided via speech and audio-only interaction techniques.

Audio Awareness and Communication

[Cohen94] Cohen, J. Monitoring background activities. Auditory Display: Sonification, Audification, and Auditory Interfaces. Reading, MA: Addison-Wesley, 1994.

[Hindus96] Hindus, Debby, Mark S. Ackerman, Scott Mainwaring, and Brian Starr. "Thunderwire: A Field Study of an Audio-Only Media Space". Proceedings of CSCW'96, pp. 238-247. November 1996.

[Watts96] Watts, Jennifer C., David D. Woods, James M. Corban, Emily S. Patterson. "Voice Loops as Cooperative Aids in Space Shuttle Mission Control". Proceedings of CSCW'96, pp. 48-247. November 1996.

Next: Adaptive Notification

Back: Simultaneous and Spatial Listening

Nomadic Radio Website