The fact that, in one year, a 34-year-old former Michigan cheerleader generated sales in excess of $1.2 billion did not go unnoticed by Time Warner, which signed Madonna to a $60 million "multimedia" contract last year. At the time, I was startled to see "multimedia" used to describe a collection of unrelated traditional print, record, and film productions. Since then, I see the word almost every day in the Wall Street Journal, often used as an adjective to mean anything from interactive to digital to broadband. It would seem that if you are an information and entertainment provider who does not plan to be in the multimedia business, you will soon be out of business. What is this all about?

It is about both new content and looking at old content in different ways. It's about one intrinsically interactive medium, made possible by a digital lingua franca: bits. And it's about the decreasing costs, increasing power, and exploding presence of computing in our daily lives: 47 percent of all PCs sold in 1992 went to the home market.

This technological push is augmented by an aggressive pull from media companies, which are selling and reselling as many bits as possible, including Madonna's (which sell so well). This not only means reuse of data, music, and film libraries but also the expanded use of text, audio, and video for as many purposes as possible, in multiple packages and through diverse channels.

The Golden Fleece to the Golden Goose
In 1975, Richard Bolt, Andy Lippman, and I submitted a proposal called "Multimedia Computing" to the Defense Advanced Research Projects Agency (previously ARPA, then DARPA, now once again ARPA). It was accepted on the condition that we change the title to avoid the possibility of receiving the Golden Fleece Award from Senator Proxmire, an annual prize given to the most gratuitously funded government project. I had been nominated on several occasions but never tarred and feathered by the dubious honor. (In December, 1979 the Office of Education won the Fleece for spending $219,592 to develop a "curriculum package" to teach college students how to watch television.)

Anyhow, it is interesting to observe that during the 1970s, "multimedia" meant "nightclubs." It carried the connotation of rock music plus light show. In 1978, when we showed a full-color, illustrated page of text on a computer screen, people gasped in astonishment when an illustration turned into a sound-synch movie at the touch of a finger. Some of today's best multimedia titles, like Robert Winter's Mozart, are high production value renditions of sloppy but seminal experiments from the 1970s.

What today's titles share with the past is the simple idea that three discrete streams of data - audio, video, and text - explicitly meet on the screen with an order imposed by astute synchronization. The current challenge in designing multimedia product is very much the organization of time, or what might be called "page layout" in the space of X, Y, and T. But multimedia can mean more.

The Message is the Medium
Modern multimedia, at least our thinking about it, must include the automatic transcoding from one medium into another, or the translation of a single representation into many media. Namely, modern multimedia should redefine our notions of a medium. WIRED's patron saint, Marshall McLuhan, was right about the medium being the message in the 1960s and 1970s. But that is not the case today. In a digital world the message is the message, and the message, in fact, may be the medium.

Multimedia needs to include fluid movement from one medium to another, saying the same thing in different ways, calling upon one human sense or another, depending on what you are doing. Books that read themselves when you are dozing off, or movies that explain themselves with text are good examples. The salient still, a recent breakthrough at the Media Lab, is an even better illustration of transcoding in multimedia.

The original problem addressed by Walter Bender, a founding member of the MIT Media Lab, was: How could video be printed in such a way that the resolution of the still image would be an order of magnitude greater than any one frame? A single frame of video has very low resolution in comparison to photos. The answer, clearly, was to pull resolution out of time and look at many frames both forward and backward in time. Today, Bender makes high-quality video prints from crummy 8mm video. These stills have in excess of 5,000 lines of resolution. This means that any frame from the billions of hours of 8mm home movies stored in the shoeboxes of American homes can be turned into a Christmas card or printed for a photo album with as much or more resolution as a normal 35mm snapshot.

However, something much more than resolution results. The print captures an image that never existed. Instead, it represents a static window of many seconds of time. During that time the camera may zoom and pan, and objects in the scene may move. The image is nonetheless crisp and perfectly resolved. Its contents reflect the filmmaker's intentions by putting more resolution in places where the camera zoomed or by widening the scene if it panned. Quickly moving elements, like a person walking across a stage, drop out in favor of the temporarily stable ones.

What occurs in this example of "multimedia" is important: Movement from one medium to the next requires transcoding one dimension (time) into another dimension (space). We have simple examples in our daily lives, where, for instance, a speech (the acoustic domain) is transcribed with punctuation (the text domain) to render a small semblance of intonation. In the script for a play, much more is added in parentheses to characterize action.

True multimedia, not all of which has to be explicit sound and light on the screen (some of it can be in your head), will include the automation of transcoding from one medium to the next because people will not be satisfied with the assumption that they only can be seated in front of an array of playback machines lashed together by a gaggle of wires. We are just as likely to want teleconferencing output, for example, on a Personal Digital Assistant as we are on a full-blown "virtual reality" system worn over our heads. In short, ubiquity is more important to multimedia than is explicit immersion.

Next Issue: Virtual Reality - Oxymoron or Pleonasm?

[Copyright 1993, WIRED Ventures Ltd. All Rights Reserved. Issue 1.05 October 1993.]