At the age of 2, Hiroshi Ishii experienced his first PDA - an abacus. Even though this calculator was used primarily to manage Mrs. Ishii's budget, Hiroshi found many more interesting "dual uses." In fact, he used the abacus as a musical instrument, a toy train, and a back-scratcher. His mother didn't mind, and Hiroshi soon learned the "music" of addition and multiplication at the simplest level: the tune meant that the beads - predigital bits - were in use.

Forty years later, Ishii is determined to carry the idea of tangible bits forward and into the problem of making the human-computer interface seamless with the physical world. This mission obviously includes sound, dual uses, and - most important - the engagement of muscles and motor skills. Although somewhat less apparent, it includes attending to the peripheral senses - call these the ambience.

Feely-touchy
Transforming human-computer interaction from abstract mousings and keystrokes into hands-on engagement is the challenge Ishii and his students are addressing. They are building interfaces in which bits are embodied and grasped as physical objects and surfaces. Physical icons, or "phicons,"are small objects that serve as both handles for, and containers of, information. A prototype of such an interface exists in the form of a horizontal display surface that senses the physical objects placed on it.

This interface allows computer-generated video, graphics, and 3-D models to be accessed by placing a phicon on the display surface, for example. You interact with the content by manipulating the phicons, inspecting the space with "lenses,"or probing the space with "instruments."

World as interface
Yet there is a world of sensation beyond that which can be grasped with our hands or stared at with our eyes. Ishii dreams of mapping Earth's nuances of warm and cold fronts, trade winds, and tidal waves into the circulating currents of his hot tub. Why? Because, he explains, experiencing a hurricane in the Bahamas as a whirlpool around your ankle or a monsoon in Asia as a warm spot on your shoulder blade allows your skin to become the interface between the meteorological world and you.

Caught within the "painted bits" of glowing pixels, Ishii returns to his childhood abacus for a vision of future interface design, intent on using everyday physical objects and surfaces - a world full of incense bottles, writing desks, and window glass. As humans, we have myriad skills for processing information through tactile interaction with physical objects.

The idea of tangible bits includes peripheral senses. Note that you often close your eyes when "feeling" something or while trying to determine the source of a particular sound. This process of concentration extends to ambience itself. You know something without "looking" at it.

But while computing, all we normally touch are transducers, and what we see is always "in our face." Yet our peripheral senses and the surrounding activity are equally important. Why has the use of background displays been lost in computer interaction? Could this stem from a cultural divide that Ishii's Eastern perspective reveals?

Making real and virtual seamless
Once the virtual world and the real world interpenetrate, the interface disappears. The Macintosh is easy to use because knowing one action set - see-point-click - allows users to navigate the menus. Ishii's interface is even simpler. His idea of interface is seamlessly coupled with the physical world. You manipulate things using your innate knowledge of the physical world. If you can pick up a mothball, you can run Ishii's computer.

His "computer" is a small room that is augmented with computer-controlled lights, shadows, sounds, airflow, and water movements. These communicate information to the user's peripheral attention - at the background of awareness - leaving the user free to concentrate on other tasks in the foreground, those which we refer to as "at hand."

In this space, light reflecting off rippling water moves gently across the ceiling to communicate the activity of a loved one (e.g., the lab's pet hamster skittering on its wheel). Changes in lighting and an audio space that features birdcalls and thunder convey email or information on Net traffic. Past activity can be reviewed by turning back the hands of a physical wall clock.

Sicilian kitchen
The curtain rises; it is 2020. Quantum computing made the metric of qips (quadrillions of instructions per second) obsolete years ago, and computer interfaces are equivalent to the approximate age and maturity of the 1978 automobile interface. To view the true state of the art, we visit a Sicilian kitchen and look to the center table, only to find…

Bread. Pasta. Olive oil and an overripe tomato. Perhaps the bread knives are edged with guaranteed-never-to-go-dull nanoceramic and the oven is fusion fired. The only glass screens in the kitchen are found on a window overlooking the garden and the oven door (both nanocleaning, of course). The only keyboard resides on the faux-vintage typewriter. And all the mice play tag with the cats.

This Sicilian kitchen is digital, of course, but it is also intimate and inescapably physical. While a few frantic folk and workers of the midnight hour consume energy pills, the Sicilians take pleasure in their food and embrace its substance and its preparation. Ishii and I join the Sicilians in this quest to maintain the primacy of the physical world as interface - and strive to make the recipe books, green peppers, and wine bottles of the future proud.

This article was written with Professor Hiroshi Ishii (ishii@media.mit.edu), who founded and directs the Tangible Media Group at the MIT Media Lab, and his graduate student assistant, Brygg Ulmer (brygg@media.mit.edu).

Next Issue: 2B1

[Copyright 1997, WIRED Ventures Ltd. All Rights Reserved. Issue 5.05 May 1997.]