Adaptive Notification of Messaging and Communication

Selected messages are presented in the user's listening space, based on a presentation level explicitly set by the user. Since users can be actively engaged in conversations or meetings, they may not wish to be distracted by audio playing in the foreground or the full text of the message spoken to them, at certain times of the day. Instead auditory cues or message previews can be provided. Several levels of notification are used to determine the appropriate form of message delivery and communication.

Notification Levels

Currently the following notification levels can be explicitly set by the user via speech commands:

• Silence: No notification is provided and all auditory cues are turned-off.
• Ambient Audio: Continuous ambient sound of water indicates the current state of the system and network activity. When messages are being downloaded, the sound of water is heard flowing faster.
• Auditory Cues: A unique cue is played when a message arrives to distinguish the type of message.
• Background/Foreground: pushes the spatial audio to the background or foreground of the listening space and lowers or increases the volume of the synthesized speech.
• Message Header: provides the sender and subject of email messages (via speech synthesis) or plays the first 2-3 seconds of a voice message or audio news summary.
• Message Preview: provides a short preview of an email message (first 100 characters) or plays a fifth of the audio message. Use of text summarization techniques could allow more flexible means of providing message content. Audio can be played faster using time compression techniques.
• Message Body: plays the entire audio file or reads the full text of the message. Some parsing of the text will be necessary to eliminate redundant phrases and format tags.

Adaptive Notification Model

Different combinations of notification levels can be useful for varying degrees of feedback to the user, based on her current state of awareness/interruptability. It is clear that in some cases the users will be able to set their awareness state explicitly, however in most cases users will desire a means for the system to automatically infer the notification level based on the message priority and user context.

CLUES is a filtering and prioritization system developed by Matt Marx at the Media Lab [Marx96]. It detects the timely nature of messages by finding correlation between a user's calendar, rolodex, to-do list, as well as a record of outgoing messages and phone calls. Currently CLUES only prioritizes email messages, yet it can be extended to handle voice messages, based on caller identification (within the MIT phone exchange). Content-based email filtering using CLUES has been integrated in Nomadic Radio to determine the priority level of email messages as timely, personal or very important.

A system with some domain knowledge and an ability to learn the user's regular listening patterns would be ideal for adaptive notification of messaging and communication. Implicit feedback from the user provided by ignoring or actively foregrounding messages presented should allow the system to reinforce certain kinds of notification and develop a listening profile (LP) over time. The goal for the system would be to maximize quality and amount of information presented and minimize the cost of delivery due to loss of user attention from current activity. Several machine learning techniques such as Memory-based Reasoning (MBR) [Maes94] or Bayesian Networks can be explored to mediate the dynamic auditory notification, based on the user context and prior listening patterns. Bayesian Networks have been used in the past to control display of information or 3D graphics renderings, based on the resources available, perceptual cost of degradation and attentional focus of viewers [Horvitz 95, 97]. A notification model for wearable computing must utilize some form of machine learning and user modeling to provide an effective means for adapting to a user's specific behavioral patterns and focus of attention.

Message Filtering and Adaptive Information Display

[Horvitz95] Horvitz, Eric and Jed Lengyel, "Perception, Attention, and Resources: A Decision-Theoretic Approach to Graphics Rendering", Proceedings of the Thirteenth Conference on Uncertainty in Artificial Intelligence (UAI'97), Providence, RI, Aug. 1-3, 1997, pp. 238-249.

[Horvitz97] Horvitz, Eric and Matthew Barry. "Display of Information for Time-Critical Decision Making", Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence (UAI'95), Montreal, August 1995.

[Maes94] Maes, Pattie. "Agents that Reduce Work and Information Overload" Communications of the ACM, July 1994/Vol.34, No.7, pp. 31-41.

[Marx96] Marx, Matthew and Chris Schmandt. "CLUES: Dynamic Personalized Message Filtering". Proceedings of CSCW '96, pp. 113-121, November 1996.

[Papp96] Papp, Albert and Meera M. Blattner, "Dynamic Presentation of Asynchronous Auditory Output", Proceedings of ACM Multimedia'96, November 1996, pp.109-116.

Next: The Future of Wearable Audio Computing

Back: Awareness and Communication

Nomadic Radio Website