Current mobile communication devices do not grab our attention in a socially appropriate way. They could be disrespectful of ongoing social activity such as an important meeting or private dinner. To improve on this, I have built the Cellular Squirrel, a system where the agent that controls my cellphone is embodied in a small portable animatronic device, as a personal 'companion' for the user. This embodiment is able to use the same subtle but still public non-verbal cues to get our attention and interrupt us like humans would do (like eye gaze and small gestures), instead of ringing or vibration. The user can whisper and listen to her squirrel, receiving and replying to voice instant messages. If the user wishes, she can also bypass the Intermediary altogether and get into a synchronous voice communication with the caller by simply talking to the embodiment.

The Cellular Squirrel is the latest prototype of an Autonomous Interactive Intermediary, which is a software and robotic agent that helps the user manage her mobile communication channels. This is my dissertation project, built when I was part of the Speech Interface Group, headed by Chris Schmandt, at MIT's Media Lab.


Squirrel with back open

Today's cellphones are passive communication portals. They do not 'care' about our ongoing conversations, about the relationship between the caller and the user, nor what the purpose of the call is, and often interrupt us at inappropriate times. The Cellular Squirrel is part of a larger project that aims at adding elements of human style social intelligence to our mobile communication devices in order to make them more socially acceptable to both user and local others. It is an Autonomous Interactive Intermediary that assumes the role of an actively mediating party between caller, callee, and co-located people.

This project explores ways to make mobile communication devices socially intelligent, both in their internal reasoning and in how they interact with people, trying to avoid, e.g., that our advanced communication devices interrupt us at completely inappropriate times.

The Intermediary is a conversational as well as robotic agent.

The conversational agent is able to converse with caller and callee—at the same time, mediating between them, and possibly suggesting modality crossovers. It deals with incoming communication attempts when the user cannot or does not want to. It's a dual conversational agent since it can converse with both user and caller simultaneously, mediating between them.

The robotic agent, an animatronic device, uses socially strong non-verbal cues like gaze, posture, and gestures, to alert and interact with the user and co-located people in a subtle but public way. I have built three working prototypes of Intermediaries: a parrot, a bunny, and a squirrel.

An Intermediary, however, is more than just an interactive animatronic device that listens to you and whispers into your ear. When a call comes in, it detects face-to-face conversations to determine social groupings, may invite input ("vetos") from the local others, consults memory of previous interactions stored in the location, and tries to assess the importance of the incoming communication by conversing with the caller.


Animatronic skeleton

The current implementation of the Cellulars Squirrel is a remote brain robot. It has a duplex audio and duplex data connection to via a class 1 Bluetooth module. It is controlled by a PC with a Bluetooth dongle that runs all the necessary control processes (conversational agent, speech recognition, animatronics control server, etc). Mechanically, it consists of a brass skeleton with micro servos that control four degrees of freedom (neck up/down, left/right, left and right eyelids). There are also sensors in its extremities so that it detects the owners touches.

Earlier embodiments included a parrot and a cute bunny.

Videos

Videos of the Squirrel and other embodiments can be found here.

Publications

Marti, S. & Schmandt, C. (2005). Physical Embodiments for Mobile Communication Agents. In Proceedings of UIST 2005, Seattle/WA, 2005.

Marti. S. (2005). Autonomous Interactive Intermediaries: Social Intelligence for Mobile Communication Agents. Ph.D. dissertation, MIT Media Laboratory, June 2005.

Marti, S. (2004). Autonomous Interactive Intermediaries: Social Intelligence for Mobile Communication Agents. In Doctoral Colloquium Companion Booklet of ACM conference on Computer supported Cooperative Work (CSCW '04), Chicago, 2004.

Press
New York Times Magazine article, December 11, 2005.
ITU News magazine article, October 2005.
The Age article (Australia), October 1, 2005.
Kölner Stadtanzeiger article (Germany), September 29, 2005.
PC Chip (Croatia), September 2005.
BBC News Online (and blogs based on it), August 26, 2005.
Technology Review article, August 2005.
FRAMES article, May/June 2005.
Mobileburn article (and other blogs), July 15, 2005.
Toronto Star article, July 24, 2005.

© 1997 - 2008 by Stefan Marti & MIT Media Lab. Send comments to: . Last updated: October 11, 2005
home | research | writings | courses | personal