Debugging and the Experience of Immediacy

David Ungar

Sun Microsystems Labs

Mountain View, CA, USA

Henry Lieberman

Media Laboratory

Massachusetts Institute of Technology

Cambridge, MA, USA

Christopher Fry

PowerScout Corp.

Boston, MA, USA


A good user interface brings you face-to-face with whatever is being manipulated and experienced. For example, the steering in a sports car, especially with a mid-engine layout, lets you feel the road surface so you can tell when the road is slippery, just by the feeling of the steering. A good tool, like a high-quality wrench becomes a part of your hand, so you feel that you are turning the bolt yourself. The machinery disappears, and you feel connected to the object in question. A programming environment can also convey an experience of immediacy, drawing the programmer in closer to his program. When that happens, debugging becomes easier. This principle of immediacy can serve as a guide, keeping programming environment builders on the path to productive environments.

Like your favorite pizza, a programming environment can be described as a layered yet synergistic stack of ingredients:

The principle of immediacy can be brought to bear on each of these layers.

For example, in the user interface, it will help if each chunk of information (e.g. each procedure) shows up as a visually distinguished and clearly legible unit. Here, at the low level of visual perception, designers frequently employ cues that mimic the real visual world in order to exploit a user's preexisting perceptual abilities. That is why, for example, the Self user interface [1] renders each object as a slab with stereotypical stage lighting, from above and left. Careful attention to perceptual legibility can lighten the burden on the programmer, so that more attention can be focused on finding bugs and less attention on finding objects on the screen. The user interface must allow programmers to manipulate programs as well as see them. Here again, immediacy can be achieved by employing affordances that match user's prior experiences, or that reduce the need for the user to change focus. A bad user interface draws so much attention to itself that it distracts from debugging, and a good one lets a tired programmer debug much later into the night (if need be!).

But the UI is only the lowest-level of the system. The programming environment provides the lenses and levers that let the programmer inspect, manipulate, and debug the program. But if the lenses and levers feel like lenses and levers, the programmer is distanced from the program, has to mentally work harder to keep track of it, and is encumbered in attempts to fix it.

The principle of immediacy leads us to understand that the environment must make it effortless to examine a program, with all of its connections. If the programming environment you use cannot rapidly and effortlessly show you all the callers of a function, or all the definitions of a message, or all the places that can change a particular variable, it is distancing you from the logical structure of your program, and making it that much harder for you to debug it.

And when debugging, what goes for a static program also goes for a (running or suspended) process. When you find an argument that has an erroneous value, the environment should be able to take you to where the value was ultimately passed in, even if it is ten stack frames up (although I know of none that can do this today). Immediacy says that the important relationships must be made manifest.

Immediacy and debugging

We'll illustrate how the principle of immediacy applies to debugging environments by describing ZStep 95 [3, 4], a program debugging environment based on the principle of immediacy. Debugging is often a search that starts from observed effects of running a program and proceeds to deduce the possible causes in the underlying program code. So ZStep 95 strives to facilitate debugging by bringing the relationship between cause and effect closer to the programmer.

The separation between cause and effect can occur in at least three dimensions: time, space and semantics, thus designers can strive for three kinds of immediacy that are important for debugging: temporal immediacy, spatial immediacy and semantic immediacy.

Temporal immediacy

Human beings recognize causality without concious effort only when the time between causally related events is kept to a minimum. If there is a delay between a change in a steering wheel and the response of a car, we describe the steering as "mushy" and it destroys the feeling of immediacy of control of the car. In programming, delay between an effect and observing related events or data in the program puts a strain on the programmer's short term memory to hold all the relevant information in their head while waiting for the programming environment to "catch up". In Eisenstadt's paper in this issue [2], we see that a full 15% of bugs are directly attributable to temporal distance between cause and effect, the largest single source of error in his survey.

In debugging, it is important to reason backwards from effects to causes, so temporal immediacy backward in time is just as important, if not more important, than temporal immediacy in the forward direction. ZStep 95 is built upon the notion of reversibility. It keeps a history of the computation and can be run either forward or backward using "video recorder" controls. The history includes expressions, values, and the graphic output of the program. Important also is fine control over the level of detail displayed. It is not only immediacy between temporally adjacent events that is important, but immediacy between temporally relevant events. We provide both "fast" and "slow" stepping speeds in both directions, manual and automatic control. By allowing the programmer to choose speed and direction, ZStep makes it easier for him to find a mode where the temporal distance between relevant causal connectionns occurs on the appropriate scale for preconcious perception.

We also provide a novel "Graphic Step Forward" and "Graphic Step Backward" that works in terms of events where something is drawn on the screen, rather than in terms of expressions in the code. This allows you to step the behavior of the program rather than step the code, and supports temporal immediacy between graphic events and their causes in the code.

Whether you choose to step expression-by-expression, graphic event by graphic event, forward or backward, all views of the program -- code, value, stack, and graphics are kept in sync and present a consistent view of the program execution. Self also stives to keep the information presented to the user consistent with the dynamic state of the system.

The principal of immediacy explains why these and other designers made this choice: by providing automatic display updates, the link between the system's state and the display state is removed from the programmer's concious attention and becomes immediate and unconcious; the programmer unconciously comes to identify one with the other, reducing the cognive burden of the debugging task.

Spatial Immediacy

Spatial immediacy means that the physical distance between causally related events should be kept to a minimum. The reason is the same as for temporal immediacy -- events that are widely separated by space on the screen force the user to devote more concious effort to link them, forcing the user to shift attention and putting a strain on he user's short-term memory. Many steppers show the value of the current expression in a fixed window off to the side of the window containing the program code display. This causes a "ping-pong" effect as the user's attention bounces from the code to the values and back again. And the existence of a separate, constant area that seqentially reflects the values of different expressions adds cognitive distance as well as spatial by forcing the user to deal with the changing link between the value display region and the expression being displayed.

ZStep 95 takes the approach of showing a floating window that follows the point of program execution. The code is displayed in an editor buffer, and as the program runs, either forward or backward, fast or slow, the expression currently being evaluated is highlighted, and, right next to it, a floating window shows the code's value, if it has been evaluated.

The user's attention never need wander to see the value of the current expression. The user need never devote precious short-term memory to the task of remembering which expression's value is being shown. If the user is working backwards in the history, the notation "Will be:" along with the value is displayed, to indicate that the value lies further along in the history. "Working on it..." is shown if the value is only partially computed, so that the event is situated in the temporal event stream.

Spatial immediacy is important because it maintains the visual context. Seeing two related pieces of information next to each other fosters making semantic connections in the mind between those two pieces of information. If they are spatially separated, "out of sight" becomes "out of mind". In ZStep 95, a click on the floating window containing the value brings up a data inspector. The code, the resulting value, and an inspector on that value are all spatially co-located, reinforcing the feeling of immediacy.

If an error occurs, the error message is shown in the floating window, again, spatially located near where the error is manifest, so both the cause and effect can be seen at once. Another click on the error message brings up documentation about the error, and another click still evokes documentation for related functions.

Semantic Immediacy

Semantic immediacy means that information that the conceptual distance between pieces of information that are semantically related is kept to a minimum. In interactive interfaces, the conceptual distance between two pieces of information is often represented by how many user interface operations, such as mouse clicks, it takes you to get from one to the other. We have already seen some examples of semantic immediacy, between code expressions and their values, and between error messages and documentation. There are others.

Because ZStep 95 keeps a history of all the graphic events in the program, we can associate each graphic object drawn on the screen with the event that drew it. This makes it possible to click on a graphic object, and go immediately to the place in the execution where that object was drawn. Not only do we see the source code that drew the object, but the entire state of the stepper is backed up to the point in time where that object was drawn, including the appearance of the screen at the time, the values of expressions, the stack. Similarly, we can point to an expression and jump the stepper immediately to that point in time, keeping all views consistent, including the graphics.

To summarize, the process of debugging a program involves understanding the relationships between the following objects:

An expression in the source code [not "lines of code"!]

The value of that expression at a given point in time

The graphic output, if any, of that expression

Expressions that are executed before or after that expression

Other expressions or values that are semantically related

[for example, the stack, inspecting data values, etc.]

ZStep's interface is designed to allow you to go from any one of these objects to any one of the others with practially no concious effort. It never takes more than one mouse click. That's semantic immediacy.

Immediacy and programming language design

An environment creates the experience of immediacy by allowing the programmer to make any change at any time, and by taking less than half a second to do so, which keeps response time below the level of perceivable delay. That way, the program feels right in your hands. I once attended a panel session representing three great programming environments at PARC: InterLisp, Smalltalk, and Cedar. The Cedar panelist boasted that, since multitasking had been added to Cedar, he now had the luxury of reading his e-mail while waiting for compiles. But the Smalltalk user had the greater luxury of having any change take effect in the time it took to press the accept button and see the resulting screen flash. He or she could then go on and continue debugging without loss of continuity. Just as immediacy applies to both stimulus and response sides of the UI, it also applies-at a higher level of abstraction-to both browsing and changing code and data in the programming environment.

Like Smalltalk, Self, APL, and most Lisps let the programmer breakpoint a program, and insert extra code to print information, or to check conditions, or even to attempt a bug fix-while the program is suspended. The program can be continued without restarting, the change takes effect immediately, runs at full speed, can be browsed the same way as any other part of the program. Few other environments-especially those in wide use-can boast this level of manipulative immediacy. Many C++ systems take minutes to change class definitions in a large program, forcing the programmer to waste mental effort devising workarounds, or leading to either boredom (leaving the flow state) or distraction (reading e-mail, etc.). Immediacy at the programming-environment manipulation level is crucial for debugging productivity, and underlies many of the good results reported on in the rest of this issue.

Finally, programming language design is often viewed as high art, mystical religion, or as putting together a menu for a restaurant (this is the principle of orthogonality-- the diner can choose sauce independently of meat to give everyone something that they like). One can also appraise the design of a language by returning to the principle of immediacy. In order for a language to bring programs closer to the user, it must be built around a small number of concepts. For example, APL embodies linear algebra with its arrays, functions, and functions-on-functions (a.k.a operators). As a result, to an engineer already familiar with linear algebra, there is hardly any learning curve at all! Just as you can write any matrix equation on a piece of paper, you can write almost any APL expression, unlike many languages that impose extra constraints. For example, APL though typesafe, has no static type checking. Although the absence of static checking requires the programmer to find and fix some bugs at runtime (the traditional rationale for static typing), its presence compels the programmer to reason much more indirectly about his program. Like a scratch on a skipping CD (or LP), static semantics such as type checking ram home the disturbing fact that the program does not execute as written; there is a translation and checking stage, which "executes" the program in its own way, so that what finally runs is less directly related to what the programmer can write or see.

As another example, macros are a popular variety of static translation on programs. They often help the programmer make up for a lack in the language design itself. But what happens when there is a bug in the macro? The programmer may wind up staring at "preprocessor output" in order to understand some particularly abstruse and unexpected behavior of the program. In effect the poor programmer has to debug two programs, the one he wrote, and the one that eventually ran. The principle of immediacy dictates that the programmer need not worry about static invariants unless he or she wants to, but many existing languages force doing this all the time.

By bringing the programmer closer to the program, an experience of immediacy helps the programmer understand, change, and ultimately debug. Attaining a reasonable level of immediacy is a precondition for effective debugging and can serve as a useful guide to the design of all aspects of programming environments.


[1] Bay-Wei Chang, Objective reality for self: concreteness and animation in the Seity user interface. Ph.D. Dissertation, Electrical Engineering Dept., Stanford University, 1996.

[2] Marc Eisenstadt, "My Hairiest Bug War Stories", Communcations of the ACM, April 1997.

[3] Henry Lieberman and Christopher Fry, Bridging the Gap Between Code and Behavior in Programming, ACM Conference on Computers and Human Interface, CHI '95, Denver, Colorado, April 1995.

[4] Henry Lieberman and Christopher Fry, ZStep 95, A Reversible, Animated Source Code Stepper, in Software Visualization: Programming as a Multimedia Experience, John Stasko, John Domingue, Marc Brown, and Blaine Price, eds., MIT Press, Cambridge, MA, 1997.

David Ungar is a Senior Staff Scientist at Sun Microsystems Laboratories. Author's present address: Sun Microsystems Laboratories, 2550 Garcia Ave, MTV29-117, Mountain View, CA 94043; e-mail:

Henry Lieberman is a Research Scientist at the Media Laboratory of the Massachusetts Institute of Technology. His interests lie at the intersection of artificial intelligence and human interface concerns, and among his topics of interest are intelligent agents for the Web, software visualization, visual programming and debugging, machine learning, and programming by example. Author's present address: Media Laboratory, MIT, 20 Ames St., Room 305 A, Cambridge, MA 02139 USA. E-Mail:

Christopher Fry is the Chief Technical Officer of PowerScout Corporation in Boston. His current primary interest is the visualization of complex data structures as diverse as the World Wide Web or intermediate values within a program development environment. Over the past couple of decades, Fry has hacked at MIT and numerous MIT spin-offs. Author's Present Address: PowerScout Corp., 21 Crescent Rd., Lexington, MA 02173 USA. Email: